[Rasch] DIF and multiple comparisons

Mike Linacre (RMT) rmt at rasch.org
Tue Feb 7 05:23:16 EST 2006


Dear Fred:

Thank you for your email. Here's my understanding of this, but there is 
probably something more authoritative in the literature somewhere .... and, 
if anyone on the list is better informed, please share ....

In Rasch analysis, the usual Bonferroni rules apply, so it depends what 
hypothesis you are testing.

If the hypothesis to be tested is: "None of these 20 items exhibit DIF". 
The Bonferroni applies to each of the 20 items in turn, because we would 
expect that at least one of the items would show a significance p<.05 under 
these conditions.

If the hypothesis to be tested is: "Item 1 has no DIF". Bonferroni does not 
apply. We would not expect item 1 to have a significance p<.05 under this 
condition.

So, a procedure would be to test the items one at a time, but then 
interpret the significance tests conservatively, because even if, for 20 
items, we expect one of them to have p<.05, we are still surprised by which 
item it happens to be. The items are not perfectly exchangeable or a random 
selection from a population of items. Each item was individually crafted 
and stands by itself.

OK?

Cordially,

Mike L.

At 10:11 AM 2/6/2006 -0600, you wrote:
>In performing item analysis for DIF, there were 20 items (scored 0-3). The 
>question came up as to whether one should correct for multiple 
>comparisons. Using the Bonferroni adjustment, that would convert the 
>significant p value from 0.5 to 0.025. However, the items are not entirely 
>independent. Within the Rasch schema, are there data, rules, suggestions 
>as to whether correction for multiple comparisons should be undertaken?

Mike Linacre
Editor, Rasch Measurement Transactions
rmt at rasch.org www.rasch.org/rmt/ Latest RMT: 19:3 Winter


-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20060206/6c138b67/attachment.html 


More information about the Rasch mailing list