[Rasch] DIF,Uni-dimensionality, and testlets
svkr at sund.ku.dk
Fri Oct 3 06:13:21 EST 2014
True, DIF can be caused by multidimensionality, but since we have other means to test for multidimensionality, we usually do not think about multidimensionality when we find evidence of DIF.
Evidence of multidimensionality may on the other hand be caused by DIF. If two or three items have DIF relative to Age and if Age is not included in the analysis, then Age will place the role of an unobserved "latent" variable. This possibility is often forgotten by people using exploratory factor analysis.
A testlet in PISA consists of a text (a reading unit if it is a reading text) with several questions. It is obvious that there is a risk (not a certainty) of local response dependence in such test. Our analyses that this was the case.
Adding responses from such items into one superitem may under certain conditions create items fitting the PCM. The condition is that the local dependence is uniform in the sense that the strength of the local association does not depend on the person parameter (like in uniform DIF). Karl and I have several papers on that topic.
From: rasch-bounces at acer.edu.au [rasch-bounces at acer.edu.au] on behalf of Thomas Salzberger [thomas.salzberger at gmail.com]
Sent: Thursday, October 02, 2014 5:57 PM
To: rasch at acer.edu.au
Subject: Re: [Rasch] DIF,Uni-dimensionality, and testlets
2014-10-02 16:08 GMT+02:00 Fabio La Porta <fabiolaporta at mail.com<mailto:fabiolaporta at mail.com>>:
First: DIF can also be viewed as a violation of invariance. The Rasch model tells that ability estimates are independent from the difficulty estimates and vice versa. This must hold for the sample as a whole but also for any subsamples drawn from the main sample, irrispective of the factor used to make such partitions.
Agree, strictly speaking DIF contradicts invariance.
You were talking about reasons for DIF.
Multidimensionality is just one possible reason for DIF. Generally speaking, there is no 1:1 relationship between any symptom pointing at problems and a particular reason or type of problem. This makes it hard to figure out what the issue really is.
All available evidence of your Rasch analysis, other qualitative information you might have and conceptual considerations usually help narrow the search area.
If you believe DIF is due to a quantitatively different interpretation of the item (something is harder for one group of participants and easier for another), you may account for DIF by splitting the item. The difference in location reflects, and accounts for, the difference in meaning. If there is a qualitative difference, the split versions (or at least one of them) won't work properly. This should not be done automatically. A reasonable explanation as to why DIF occurs is desirable.
But as Fabio said it involves a violation of invariance. You need to decide whether it is acceptable to include the extra information of group membership.
On the one hand, DIF-free scales are preferable. On the other hand, if an item appears to be important but shows DIF, one might want to retain it.
Keep in mind that (real) DIF may also imply artificial DIF (DIF due to a mathematical necessity). Look for papers by Curt Hagquist.
Second: I think that the word 'testlet' may have different meaning. For instance, in PISA, the word 'testlets' seems to refer to the fact that items are organized in some sort of hierarchical structure (i.e. booklet --> testlet --> items). In such a context, items within a testlet may display local dependence if they have a similar content.
On the other hand, within Rasch analysis, it is possible to create 'testlets", that is to group locally dependent items into 'superitems" with the specific aim of absorbing local dependency. So, for instance, if you have to dichotomous locally dependent item, you can create a superitem from those two, with a scoring structure 0-1-2. This is a technique frequently used with some Rasch softwares (e.g. RUMM) as an alternative way to item deletion for dealing with local dependency when item reduction is not a desiderable option.
I hope this helps.
As Fabio, I had difficulties to comprehend the terminology.
You said "test-lets are a big violation of Rasch modeling assumptions because they create local dependence."
Test-lets understood as a combination of items (subtest, superitem, etc) do not create LD, they try to overcome it.
But yes, if you refer to items that share some specific content (they have more in common than they share with items from other "testlets"), then they create LD.
Sorry for the redundancy. Fabio said it all before.
Sent: Thursday, October 02, 2014 at 2:17 PM
From: "Steve Kramer" <skramer1958 at verizon.net<mailto:skramer1958 at verizon.net>>
To: rasch at acer.edu.au<mailto:rasch at acer.edu.au>
Subject: [Rasch] DIF,Uni-dimensionality, and testlets
I participate in these conversations mainly as a student, trying to learn
from more expert Rasch modelers. Dr. Kreiner's response, below, to my
earlier query raises two points I only partially understand, and I would
First: I had always viewed DIF as a special case of uni-dimensionality
violation. That is the way I have explained it to others. Am I wrong? How
should I think about it?
Second: I had been taught that test-lets are a big violation of Rasch
modeling assumptions because they create local dependence. In the email
below, Dr. Kreiner notes that PISA uses testlets. So: when are testlets
"not too bad", i.e., when is Rasch a close-enough approximation or reality
that it can be used despite the presence of teslets? Is there a way to tell
or a rule of thumb to use? (Honestly, I'd prefer the rule-of-thumb because
I'm often too pressed for time to do an extra set of complex analyses.) This
is a practical question for me, because I have to ammend some tests that
both want to use a Rasch model and use testlets.
The 21st Century Partnership for STEM Education
Rasch mailing list
email: Rasch at acer.edu.au<mailto:Rasch at acer.edu.au>
Thomas.Salzberger at gmail.com<mailto:Thomas.Salzberger at gmail.com>
Thomas.Salzberger at wu.ac.at<mailto:Thomas.Salzberger at wu.ac.at>
More information about the Rasch