[Rasch] Threshol Disordering
jimsick at iname.com
Wed Dec 20 00:31:48 EST 2006
Cloze tests are prone to item dependence, and in my experience
dichotomously-scored cloze items tend to overfit (i.e., Infit MNSQ below
.70). The likely reason is that context provides clues and the more
items one gets, the more context one has to deduce the remaining items,
lending an advantage to the more able test takers and inflating item
discrimination. That said, I think the primary determinants of
difficulty are lexical frequency, morphology, the overall complexity of
the passage, and the ratio of deleted words to total words. I would
treat the four passages as a single, 100 item test. Just be aware that
reliability indicators, especially sample-based statistics such as
alpha, could be inflated and misleading.
Also, are responses selected from a list of words, or open ended? In the
example given, I can think of at least three acceptable answers for the
second deletion (entities, bodies, countries).
Trevor Bond wrote:
> Clearly I've eaten too many dim sum and have become a little slow.
> But in what sense could a 25 response cloze passage be seen as one item of a four
> item test?
> With either the PCM or the RSM the model requires ordered categories. So, surely
> with the RSM we must have the same 25 missing words in each passage and in the
> PCM...? or we are expecting local dependence due to one stem, so using the super-
> item approach?
> so please tell me what I've missed . . .
> Rasch mailing list
> Rasch at acer.edu.au
More information about the Rasch