[Rasch] item bank
MarkM at eddata.com
Thu Oct 12 07:00:30 EST 2006
Iasonas et. al.,
To support Don's point a different way, to the degree pre-knowledge of
individual items influences a student's performance on the test, the item
becomes less independent of the examinee, violating the parameter
separability requirement of Rasch.
However, this effect can be expected to diminish the larger the size of the
published item bank since performance on the test as a whole becomes less
related to previous experience with any individual item. How large a bank
is large enough? I don't know. Ideally, the bank should be as "large" as
the content domain as a whole -- the set of all possible non-duplicating
items -- certainly larger than any single student can reasonably memorize.
In practice, a bank of 1000 items or more feels like it might be sufficient.
The side-effect of published test preparation materials is not trivial. I
might counter by reserving for the test a version of the item which is
written differently, to foil rote memorization, but which can be expected to
be of equivalent difficulty. In this context, open-ended items are less
prone to rote memorization than multiple choice, but test-specific variants
of the question would still be helpful.
Educational Data Systems
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf
Of Donald Bacon
Sent: Wednesday, October 11, 2006 1:28 PM
To: Agustin Tristan; -Rasch
Subject: RE: [Rasch] item bank
Hi Iasonas (and others) --
I agree with Agustin. Items on a test are assumed to be sampled from a
domain of all possible questions. If you distribute the items in advance,
we are no longer taking a random sample of knowledge, so we can't make
inferences about the knowledge domain. I recommend distributing a detailed
description of the knowledge domain (a.k.a., learning objectives), but not
the questions themselves.
Associate Professor of Marketing
Daniels College of Business
University of Denver
dbacon at du.edu, 303-871-2707
From: rasch-bounces at acer.edu.au on behalf of Agustin Tristan
Sent: Wed 10/11/2006 12:30 PM
Subject: [Rasch] item bank
I think the main question is not if the items are multiple choice or open
responses with an item bank. In many countries (including Mexico and other
Latin American countries), some people say that there is no reason to have
the item bank under so high security measures, they believe that if you
publish (in a book, on the web or whatever other media, that is not the
issue) for instance a bank of 300 questions (or 100 or 1000, that is not the
issue either) and if the students are able to "learn" (mostly to memorize)
the 300 questions this is a good thing, they say this is preferable than to
have a student who doesn't know anything....300 memorized questions is
better than nothing, so when they will face the test (a subset of these
questions) they will obtain high notes, they will show the higher stakes,
and our national education will improve.
I hope you may see the falacies of this reasoning.
If the national education system tries to develop abilities, knowledge and
competencies in our students, a certain part of this knowledge is produced
using the memory, BUT not to memorize 300 items, education is more than
If the system wishes to develop critical thinking, and the items in the bank
concern this ability, BUT the student did memorize the items, they will use
the memory but will not develop critical thinking.
Repeat the same argument with other abilities and objectives of the national
education system and you will see that an "open" item bank is a wrong
One of the main falacies is that people needs to know THE ITEMS, I agree
they need to know the areas, subjects, topics, level of complexity, but not
the item. For instance if the doctor asks me to do a blood analysis to
measure several substances and organic parameters, I have to know several
things concerning the test: I don't have to eat at least for 12 hours before
the test (or 6 or whatever previous time), and I don't have to drink alcohol
or some other substances in a specific time, but if you drink or eat
something, you may provide false results (in favor or against you). I hope
to have a healthy situation, but I don't do anything specific to obtain
false good results. Why we expect this in our health and not in education?
The educational test is like a medical test: I wish to have a real and good
"picture" of what I am able to do, to know, to produce...in many countries
and places, persons wants to get good results, not a good picture. If
everybody obtain good results, but this is not true, how can we improve the
educational situation of our students? The test is the mean not the goal.
We have the idea to educate people and evaluate people for a specific
purpose, I expect that to get high scores in a test is not the main purpose
Hope this helps.
FAMILIA DE PROGRAMAS KALT.
Mariano Jiménez 1830 A
Col. Balcones del Valle
78280, San Luis Potosí, S.L.P. México
TEL (52) 44-4820 37 88, 44-4820 04 31
FAX (52) 44-4815 48 48
web page (in Spanish AND ENGLISH): http://www.ieesa-kalt.com
Get your own web address for just $1.99/1st yr
We'll help. Yahoo! Small Business
Rasch mailing list
Rasch at acer.edu.au
More information about the Rasch