[Rasch] DIF size and significance
Mike Linacre (RMT)
rmt at rasch.org
Tue Jul 8 12:01:45 EST 2008
There are two concerns with DIF:
1. That is big enough to make a substantive impact on the measures.
2. That it is statistically improbably enough ("significant enough") that
it is not likely merely to be an accident of the natural randomness in the
So, the ETS criteria are "both ... and ..."
Yes, you always want to investigate the reason for DIF because it may
indicate a flaw in test construction or scoring.
If the DIF effect is roughly uniform within the focus group, then changing
the sample size will not alter the DIF size, but will alter its
At 7/7/2008, you wrote:
>My question is this: are the ETS size/significance criteria 'either-or'?
>I'm finding many items exhibiting DIF where the DIF contrast is small in
>absolute size (0.25 logits or less) but nonetheless statistically
>significant at the 0.01 level.
>I'm not sure whether I should...
>-simply use a cut-off point based on size, such as ETS's 0.43 'moderate
>-investigate the causes of the high DIF significance/low DIF size items
>-randomly select a smaller sample of, say 1000 students per test, and see
>if the items still show DIF.
>You can see the ETS criteria at
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch