[Rasch] Re: Rasch Digest, Vol 36, Issue 7
ici_kalt at yahoo.com
Mon Jul 7 23:05:04 EST 2008
Hello Joe. The problem you are mentioning is related to sample size. The problem is when you choose a sub-sample of 1000, why not 500? why not 2000? As we had the same problem in the past, I´ve proposed a normalization, to have always the same reference, please see http://www.rasch.org/rmt/rmt203e.htm where I show DIF analysis with huge and small samples once they are normalized.
Hope this helps.
FAMILIA DE PROGRAMAS KALT.
Ave. Cordillera Occidental No. 635
Colonia Lomas 4ª San Luis Potosí, San Luis Potosí.
C.P. 78216 MEXICO
(52) (444) 8 25 50 76
(52) (444) 8 25 50 77
(52) (444) 8 25 50 78
web page (in Spanish AND ENGLISH): http://www.ieesa-kalt.com
--- On Mon, 7/7/08, J.P. Lavallee <lavallee.jp at gmail.com> wrote:
From: J.P. Lavallee <lavallee.jp at gmail.com>
Subject: [Rasch] Re: Rasch Digest, Vol 36, Issue 7
To: rasch at acer.edu.au
Date: Monday, July 7, 2008, 12:14 AM
Thanks, Mike, for the link to the ETS DIF criteria. I hope I can ask a further question on the use of the DIF 'size' and 'significance' criteria.
I'm currently equating exams originally created by test editors drawing from a pool of uncalibrated items. Each test was taken by more than 2000 students. I'm using WINSTEPS MForms to merge two forms at a time into a single matrix, then using the DIF table to check the common items for DIF.
My question is this: are the ETS size/significance criteria 'either-or'? I'm finding many items exhibiting DIF where the DIF contrast is small in absolute size (0.25 logits or less) but nonetheless statistically significant at the 0.01 level.
I'm not sure whether I should...
-simply use a cut-off point based on size, such as ETS's 0.43 'moderate DIF' standard
-investigate the causes of the high DIF significance/low DIF size items
-randomly select a smaller sample of, say 1000 students per test, and see if the items still show DIF.
I'd be grateful for any and all input on this problem!
Ming Chuan University, Taiwan
From: "Mike Linacre (RMT)" rmt at rasch.org
Invariance of item difficulties across different person groups can be
investigated by means of DIF analysis. ETS have developed rules-of-thumb
for identifying when their items exhibit enough DIF for remedial action to
be taken. You can see the ETS criteria at http://www.rasch.org/rmt/rmt32a.htm
Rasch mailing list
Rasch at acer.edu.au
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch