[Rasch] Help with equating partial credit items in pre - post test

Mike Linacre (RMT) rmt at rasch.org
Sun Jun 1 22:43:56 EST 2008


Good questions, Gregory.

You wrote: "Overfitting items ... does the fit help us with detecting 
them?  What happens if you find them?"

Fit is relative so, in any usual set of items, about half the items will 
overfit and half will underfit. Underfit and overfit are usually easy to 
detect in Rasch analysis.

Conspicuous underfit damages the usefulness of the measures through 
unmodeled noise (unpredictability). This lack of predictability also 
lessens the usefulness  of an equating relationship in an empirical 
situation. We are not really sure that measure X on one instrument 
corresponds to measure Y on the other instrument.

Conspicuous overfit (Guttman patterns) stretches out the measures along the 
logit variable, so overstating reliability. But, in equating situations of 
the Fahrenheit-Celsius variety, we know that one set of logit measures 
stretches out the variable relative to the other set of measures, so 
overfit really doesn't matter. It will merely change the equating slope. In 
fact, we will be more sure that a measure of X on one instrument 
corresponds to a measure of Y on the other instrument in empirical situations.

It would be interesting to find a Paper that demonstrates that overfit to a 
Rasch model really does lead to misleading inferences in empirical 
situations (apart from reliability coefficients).

Mike L.

At 6/1/2008, Stone, Gregory wrote:
>I've done this several times when the N is too small.  My question regards 
>local independence.  When I've done this in the past, I've (we've) 
>assessed the items for overfit and exceptionally high point-biserials.  It 
>isn't perfect, but it can give us an idea as to whether or not the 
>requirement of local independence is met, even for the calculation of item 
>difficulties.  Some journals are requiring this to be done.  So ...
>
>Is this really important in the estimation process for the purpose of 
>equating?  (local independence)
>
>Overfitting items ... does the fit help us with detecting them?  What 
>happens if you find them?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20080601/f74daba5/attachment.html 


More information about the Rasch mailing list