[Rasch] item bank

Tim Pelton tpelton at uvic.ca
Thu Oct 12 07:39:16 EST 2006

While I agree that a larger item pool may reduce the effects of  
wholesale exposure (item bank transparency), I am not convinced that  
it will be testing the same thing as secure item bank might test.

A problem that is presented fresh is a real problem that can test  
conceptual understanding, procedural mastery, and domain power (e.g.,  
mathematical power).   When you expose the item bank, you are now  
confounding this domain power with memory and recall.  In  
mathematics, a problem that has been reviewed and a solution examined  
is no longer a real problem - rather it is an exercise - often with a  
significantly diminished difficulty.

I would expect that the items would drift in the negative direction,  
and also that the discrimination of the items would degrade.  The  
latter would be as a result of inconsistent review of items and  
inconsistent memories.  These changes would also be inconsistent from  
item to item and depend upon how memorable the items and their  
solutions were.


On 11-Oct-06, at 11:30 AM, Agustin Tristan wrote:

> Hello Iasonas.
> I think the main question is not if the items are multiple choice  
> or open responses with an item bank. In many countries (including  
> Mexico and other Latin American countries), some people say that  
> there is no reason to have the item bank under so high security  
> measures, they believe that if you publish (in a book, on the web  
> or whatever other media, that is not the issue) for instance a bank  
> of 300 questions (or 100 or 1000, that is not the issue either) and  
> if the students are able to "learn" (mostly to memorize) the 300  
> questions this is a good thing, they say this is preferable than to  
> have a student who doesn't know anything....300 memorized questions  
> is better than nothing, so when they will face the test (a subset  
> of these questions) they will obtain high notes, they will show the  
> higher stakes, and our national education will improve.
> I hope you may see the falacies of this reasoning.
> If the national education system tries to develop abilities,  
> knowledge and competencies in our students, a certain part of this  
> knowledge is produced using the memory, BUT not to memorize 300  
> items, education is more than that.
> For instance:
> If the system wishes to develop critical thinking, and the items in  
> the bank concern this ability, BUT the student did memorize the  
> items, they will use the memory but will not develop critical  
> thinking.
> Repeat the same argument with other abilities and objectives of the  
> national education system and you will see that an "open" item bank  
> is a wrong solution.
> One of the main falacies is that people needs to know THE ITEMS, I  
> agree they need to know the areas, subjects, topics, level of  
> complexity, but not the item. For instance if the doctor asks me to  
> do a blood analysis to measure several substances and organic  
> parameters, I have to know several things concerning the test: I  
> don't have to eat at least for 12 hours before the test (or 6 or  
> whatever previous time), and I don't have to drink alcohol or some  
> other substances in a specific time, but if you drink or eat  
> something, you may provide false results (in favor or against you).  
> I hope to have a healthy situation, but I don't do anything  
> specific to obtain false good results.  Why we expect this in our  
> health and not in education? The educational test is like a medical  
> test: I wish to have a real and good "picture" of what I am able to  
> do, to know, to produce...in many countries and places, persons  
> wants to get good results, not a good picture. If everybody obtain  
> good results, but this is not true, how can we improve the  
> educational situation of our students?  The test is the mean not  
> the goal.
> We have the idea to educate people and evaluate people for a  
> specific purpose, I expect that to get high scores in a test is not  
> the main purpose of education.
> Hope this helps.
> Agustin.
> Mariano Jiménez 1830 A
> Col. Balcones del Valle
> 78280, San Luis Potosí, S.L.P. México
> TEL (52) 44-4820 37 88, 44-4820 04 31
> FAX (52) 44-4815 48 48
> web page (in Spanish AND ENGLISH): http://www.ieesa-kalt.com
> Get your own web address for just $1.99/1st yr. We'll help. Yahoo!  
> Small Business.
> _______________________________________________
> Rasch mailing list
> Rasch at acer.edu.au
> http://mailinglist.acer.edu.au/mailman/listinfo/rasch

Tim Pelton,
Department of Curriculum and Instruction
University of Victoria
PO Box 3010 STN CSC
Victoria BC V8W 3N4
tel 250-721-7803, fax 250-721-7598
"Any fool can know. The point is to understand." -Albert Einstein

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20061011/f6190e11/attachment.html 

More information about the Rasch mailing list