[Rasch] Help with equating partial credit items in pre - post test
Mike Linacre (RMT)
rmt at rasch.org
Sun Jun 1 22:43:56 EST 2008
Good questions, Gregory.
You wrote: "Overfitting items ... does the fit help us with detecting
them? What happens if you find them?"
Fit is relative so, in any usual set of items, about half the items will
overfit and half will underfit. Underfit and overfit are usually easy to
detect in Rasch analysis.
Conspicuous underfit damages the usefulness of the measures through
unmodeled noise (unpredictability). This lack of predictability also
lessens the usefulness of an equating relationship in an empirical
situation. We are not really sure that measure X on one instrument
corresponds to measure Y on the other instrument.
Conspicuous overfit (Guttman patterns) stretches out the measures along the
logit variable, so overstating reliability. But, in equating situations of
the Fahrenheit-Celsius variety, we know that one set of logit measures
stretches out the variable relative to the other set of measures, so
overfit really doesn't matter. It will merely change the equating slope. In
fact, we will be more sure that a measure of X on one instrument
corresponds to a measure of Y on the other instrument in empirical situations.
It would be interesting to find a Paper that demonstrates that overfit to a
Rasch model really does lead to misleading inferences in empirical
situations (apart from reliability coefficients).
At 6/1/2008, Stone, Gregory wrote:
>I've done this several times when the N is too small. My question regards
>local independence. When I've done this in the past, I've (we've)
>assessed the items for overfit and exceptionally high point-biserials. It
>isn't perfect, but it can give us an idea as to whether or not the
>requirement of local independence is met, even for the calculation of item
>difficulties. Some journals are requiring this to be done. So ...
>Is this really important in the estimation process for the purpose of
>equating? (local independence)
>Overfitting items ... does the fit help us with detecting them? What
>happens if you find them?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch