[Rasch] item bank
tpelton at uvic.ca
Thu Oct 12 07:39:16 EST 2006
While I agree that a larger item pool may reduce the effects of
wholesale exposure (item bank transparency), I am not convinced that
it will be testing the same thing as secure item bank might test.
A problem that is presented fresh is a real problem that can test
conceptual understanding, procedural mastery, and domain power (e.g.,
mathematical power). When you expose the item bank, you are now
confounding this domain power with memory and recall. In
mathematics, a problem that has been reviewed and a solution examined
is no longer a real problem - rather it is an exercise - often with a
significantly diminished difficulty.
I would expect that the items would drift in the negative direction,
and also that the discrimination of the items would degrade. The
latter would be as a result of inconsistent review of items and
inconsistent memories. These changes would also be inconsistent from
item to item and depend upon how memorable the items and their
On 11-Oct-06, at 11:30 AM, Agustin Tristan wrote:
> Hello Iasonas.
> I think the main question is not if the items are multiple choice
> or open responses with an item bank. In many countries (including
> Mexico and other Latin American countries), some people say that
> there is no reason to have the item bank under so high security
> measures, they believe that if you publish (in a book, on the web
> or whatever other media, that is not the issue) for instance a bank
> of 300 questions (or 100 or 1000, that is not the issue either) and
> if the students are able to "learn" (mostly to memorize) the 300
> questions this is a good thing, they say this is preferable than to
> have a student who doesn't know anything....300 memorized questions
> is better than nothing, so when they will face the test (a subset
> of these questions) they will obtain high notes, they will show the
> higher stakes, and our national education will improve.
> I hope you may see the falacies of this reasoning.
> If the national education system tries to develop abilities,
> knowledge and competencies in our students, a certain part of this
> knowledge is produced using the memory, BUT not to memorize 300
> items, education is more than that.
> For instance:
> If the system wishes to develop critical thinking, and the items in
> the bank concern this ability, BUT the student did memorize the
> items, they will use the memory but will not develop critical
> Repeat the same argument with other abilities and objectives of the
> national education system and you will see that an "open" item bank
> is a wrong solution.
> One of the main falacies is that people needs to know THE ITEMS, I
> agree they need to know the areas, subjects, topics, level of
> complexity, but not the item. For instance if the doctor asks me to
> do a blood analysis to measure several substances and organic
> parameters, I have to know several things concerning the test: I
> don't have to eat at least for 12 hours before the test (or 6 or
> whatever previous time), and I don't have to drink alcohol or some
> other substances in a specific time, but if you drink or eat
> something, you may provide false results (in favor or against you).
> I hope to have a healthy situation, but I don't do anything
> specific to obtain false good results. Why we expect this in our
> health and not in education? The educational test is like a medical
> test: I wish to have a real and good "picture" of what I am able to
> do, to know, to produce...in many countries and places, persons
> wants to get good results, not a good picture. If everybody obtain
> good results, but this is not true, how can we improve the
> educational situation of our students? The test is the mean not
> the goal.
> We have the idea to educate people and evaluate people for a
> specific purpose, I expect that to get high scores in a test is not
> the main purpose of education.
> Hope this helps.
> FAMILIA DE PROGRAMAS KALT.
> Mariano Jiménez 1830 A
> Col. Balcones del Valle
> 78280, San Luis Potosí, S.L.P. México
> TEL (52) 44-4820 37 88, 44-4820 04 31
> FAX (52) 44-4815 48 48
> web page (in Spanish AND ENGLISH): http://www.ieesa-kalt.com
> Get your own web address for just $1.99/1st yr. We'll help. Yahoo!
> Small Business.
> Rasch mailing list
> Rasch at acer.edu.au
Department of Curriculum and Instruction
University of Victoria
PO Box 3010 STN CSC
Victoria BC V8W 3N4
tel 250-721-7803, fax 250-721-7598
"Any fool can know. The point is to understand." -Albert Einstein
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch