[Rasch] DIF,Uni-dimensionality, and testlets

Thomas Salzberger thomas.salzberger at gmail.com
Fri Oct 3 01:57:45 EST 2014


Steve,

2014-10-02 16:08 GMT+02:00 Fabio La Porta <fabiolaporta at mail.com>:

> Hi Steve.
>
> First: DIF can also be viewed as a violation of invariance. The Rasch
> model tells that ability estimates are independent from the difficulty
> estimates and vice versa. This must hold for the sample as a whole but also
> for any subsamples drawn from the main sample, irrispective of the factor
> used to make such partitions.
>
>

Agree, strictly speaking DIF contradicts invariance.
You were talking about reasons for DIF.
Multidimensionality is just one possible reason for DIF. Generally
speaking, there is no 1:1 relationship between any symptom pointing at
problems and a particular reason or type of problem. This makes it hard to
figure out what the issue really is.
All available evidence of your Rasch analysis, other qualitative
information you might have and conceptual considerations usually help
narrow the search area.

If you believe DIF is due to a quantitatively different interpretation of
the item (something is harder for one group of participants and easier for
another), you may account for DIF by splitting the item. The difference in
location reflects, and accounts for, the difference in meaning. If there is
a qualitative difference, the split versions (or at least one of them)
won't work properly. This should not be done automatically. A reasonable
explanation as to why DIF occurs is desirable.

But as Fabio said it involves a violation of invariance. You need to decide
whether it is acceptable to include the extra information of group
membership.
On the one hand, DIF-free scales are preferable. On the other hand, if an
item appears to be important but shows DIF, one might want to retain it.

Keep in mind that (real) DIF may also imply artificial DIF (DIF due to a
mathematical necessity). Look for papers by Curt Hagquist.



> Second: I think that the word 'testlet' may have different meaning. For
> instance, in PISA, the word 'testlets' seems to refer to the fact that
> items are organized in some sort of hierarchical structure (i.e. booklet
> --> testlet --> items). In such a context, items within a testlet may
> display local dependence if they have a similar content.
> On the other hand, within Rasch analysis, it is possible to create
> 'testlets", that is to group locally dependent items into 'superitems" with
> the specific aim of absorbing local dependency. So, for instance, if you
> have to dichotomous locally dependent item, you can create a superitem from
> those two, with a scoring structure 0-1-2. This is a technique frequently
> used with some Rasch softwares (e.g. RUMM) as an alternative way to item
> deletion for dealing with local dependency when item reduction is not a
> desiderable option.
> I hope this helps.
>


As Fabio, I had difficulties to comprehend the terminology.

You said "test-lets are a big violation of Rasch modeling assumptions
because they create local dependence."

Test-lets understood as a combination of items (subtest, superitem, etc) do
not create LD, they try to overcome it.
But yes, if you refer to items that share some specific content (they have
more in common than they share with items from other "testlets"), then they
create LD.
Sorry for the redundancy. Fabio said it all before.

Thomas





>
> Fabio
>
> *Sent:* Thursday, October 02, 2014 at 2:17 PM
> *From:* "Steve Kramer" <skramer1958 at verizon.net>
> *To:* rasch at acer.edu.au
> *Subject:* [Rasch] DIF,Uni-dimensionality, and testlets
>  Dear List-serve,
> I participate in these conversations mainly as a student, trying to learn
> from more expert Rasch modelers. Dr. Kreiner's response, below, to my
> earlier query raises two points I only partially understand, and I would
> appreciate help.
>
> First: I had always viewed DIF as a special case of uni-dimensionality
> violation. That is the way I have explained it to others. Am I wrong? How
> should I think about it?
>
> Second: I had been taught that test-lets are a big violation of Rasch
> modeling assumptions because they create local dependence. In the email
> below, Dr. Kreiner notes that PISA uses testlets. So: when are testlets
> "not too bad", i.e., when is Rasch a close-enough approximation or reality
> that it can be used despite the presence of teslets? Is there a way to tell
> or a rule of thumb to use? (Honestly, I'd prefer the rule-of-thumb because
> I'm often too pressed for time to do an extra set of complex analyses.)
> This
> is a practical question for me, because I have to ammend some tests that
> both want to use a Rasch model and use testlets.
>
>
> Steve Kramer
> The 21st Century Partnership for STEM Education
> Rasch mailing list
> email: Rasch at acer.edu.au
> web:
> https://mailinglist.acer.edu.au/mailman/options/rasch/thomas.salzberger%40gmail.com
>



-- 
___________________________________
Thomas.Salzberger at gmail.com
Thomas.Salzberger at wu.ac.at
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20141002/23441cf1/attachment.html 


More information about the Rasch mailing list