# [Rasch] complex sampling Rasch

Agustin Tristan ici_kalt at yahoo.com
Wed Feb 17 03:18:19 EST 2010

```Hello Ray, you wrote:

"To control for this our test design, which has multiple test booklets, makes sure that each item is located once in each of four locations: first 30-minute cluster, second 30-minute cluster, third 30-minute cluster and fourth 30-minute cluster."

Does the position of the anchor item in each cluster matters? i.e. The first anchor item may occupy positions 13 (cluster1), 4 (cluster2), 8 (cluster3), 21 (cluster4) or is it needed that it occupies position 13 in the four clusters?
It is clear that the position must be inside its own tesltlet, but as items must be independent I think they may be interchaged in position inside the testlet.
I hope position will not be important on the item calibration, but please let me know.

Thank you.

Agustin Tristan

Subject: RE: [Rasch] complex sampling Rasch
To: "Agustin Tristan" <ici_kalt at yahoo.com>, "rasch" <rasch at mailinglist.acer.edu.au>
Cc: "Randy & Shelley MacIntosh" <srmac at bluebottle.com>
Date: Monday, February 15, 2010, 4:56 PM

(a)  This is effectively what happens.  Item parameters in PISA are estimate using a “calibration sample”.  The calibration sample is a subsample of 500 students from each OECD country.  The subsample is drawn using weights so that schools and students are appropriately represented, and then with 500 per country each country has the same influence on the item parameter estimates.

(b) PISA illustrates a potential for quite strong position effects.  To control for this our test design, which has multiple test booklets, makes sure that each item is located once in each of four locations: first 30-minute cluster, second 30-minute cluster, third 30-minute cluster and fourth 30-minute cluster.

Ray

From: Agustin Tristan [mailto:ici_kalt at yahoo.com]
Sent: Tuesday, 16 February 2010 1:58 AM
To: rasch
Cc: Adams, Ray; Randy & Shelley MacIntosh; rasch
Subject: RE: [Rasch] complex sampling Rasch

Hello! Thank your for the explanation concerng the sample design and the Rasch item's parameters, it is useful because we are also interested to use some ideas from the PISA project . In addition to Randy's question, I wonder if some of the following aspects are considered as important for the item and test calibration for the PISA project:

a) Could it be convenient to get the item calibration for each sub-sample  (schools) and after that to combine the calibrations using the weight for each sample unit to obtain a "general" item calibration?

b) As the Rasch model is sample free (or without a practical implication as Ray did explain), can we use the item calibration of the anchor items independently of the position of the items  in the test?

Regards

Agustin Tristan

Subject: RE: [Rasch] complex sampling Rasch
To: "Randy & Shelley MacIntosh" <srmac at bluebottle.com>, "rasch" <rasch at mailinglist.acer.edu.au>
Date: Sunday, February 14, 2010, 7:31 PM

Randy,

The answer is a bit complicated. Let's put fit aside and assume the
Rasch model and your data are compatible.  Then, it depends upon the
modelling assumptions you make and your estimation method.  If you use
conditional maximum likelihood or a pairwise routine then the item
parameter estimates are sample free and so the sample design does not
influence the estimates.  If you use unconditional or marginal maximum
likelihood then "in theory" there may be some minor effects, but I've
never seen any evidence that it has any practical consequence.

In PISA we do not take the complex sample design into account when
estimating item parameters.

Ray

-----Original Message-----
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On
Behalf Of Randy & Shelley MacIntosh
Sent: Monday, 15 February 2010 7:42 AM
To: rasch
Subject: [Rasch] complex sampling Rasch

I am interested in applying the Rasch model to data from a complex
sample design.
I was wondering if this has been done?
For example, is the nature of the PISA sample design explicitly taken
into account when the Rasch estimates are produced?

Thanks,
Randy MacIntosh

-------------------------------------------------
Please consider the environment before you print
_______________________________________________
Rasch mailing list
Rasch at acer.edu.au
https://mailinglist.acer.edu.au/mailman/listinfo/rasch

-------------------------------------------------
Please consider the environment before you print
_______________________________________________
Rasch mailing list
Rasch at acer.edu.au
https://mailinglist.acer.edu.au/mailman/listinfo/rasch
-------------------------------------------------
Please consider the environment before you print

-------------------------------------------------
Please consider the environment before you print
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20100216/da9e13c2/attachment.html
```