# [Rasch] complex sampling Rasch

Tue Feb 16 09:56:49 EST 2010

```(a)  This is effectively what happens.  Item parameters in PISA are
estimate using a "calibration sample".  The calibration sample is a
subsample of 500 students from each OECD country.  The subsample is
drawn using weights so that schools and students are appropriately
represented, and then with 500 per country each country has the same
influence on the item parameter estimates.

(b) PISA illustrates a potential for quite strong position effects.  To
control for this our test design, which has multiple test booklets,
makes sure that each item is located once in each of four locations:
first 30-minute cluster, second 30-minute cluster, third 30-minute
cluster and fourth 30-minute cluster.

Ray

From: Agustin Tristan [mailto:ici_kalt at yahoo.com]
Sent: Tuesday, 16 February 2010 1:58 AM
To: rasch
Cc: Adams, Ray; Randy & Shelley MacIntosh; rasch
Subject: RE: [Rasch] complex sampling Rasch

Hello! Thank your for the explanation concerng the sample design and the
Rasch item's parameters, it is useful because we are also interested to
use some ideas from the PISA project . In addition to Randy's question,
I wonder if some of the following aspects are considered as important
for the item and test calibration for the PISA project:

a) Could it be convenient to get the item calibration for each
sub-sample  (schools) and after that to combine the calibrations using
the weight for each sample unit to obtain a "general" item calibration?

b) As the Rasch model is sample free (or without a practical implication
as Ray did explain), can we use the item calibration of the anchor items
independently of the position of the items  in the test?

Regards

Agustin Tristan

Subject: RE: [Rasch] complex sampling Rasch
To: "Randy & Shelley MacIntosh" <srmac at bluebottle.com>, "rasch"
<rasch at mailinglist.acer.edu.au>
Date: Sunday, February 14, 2010, 7:31 PM

Randy,

The answer is a bit complicated. Let's put fit aside and assume
the
Rasch model and your data are compatible.  Then, it depends upon
the
modelling assumptions you make and your estimation method.  If
you use
conditional maximum likelihood or a pairwise routine then the
item
parameter estimates are sample free and so the sample design
does not
influence the estimates.  If you use unconditional or marginal
maximum
likelihood then "in theory" there may be some minor effects, but
I've
never seen any evidence that it has any practical consequence.

In PISA we do not take the complex sample design into account
when
estimating item parameters.

Ray

-----Original Message-----
From: rasch-bounces at acer.edu.au
<http://us.mc1115.mail.yahoo.com/mc/compose?to=rasch-bounces@acer.edu.au
>  [mailto:rasch-bounces at acer.edu.au
<http://us.mc1115.mail.yahoo.com/mc/compose?to=rasch-bounces@acer.edu.au
> ] On
Behalf Of Randy & Shelley MacIntosh
Sent: Monday, 15 February 2010 7:42 AM
To: rasch
Subject: [Rasch] complex sampling Rasch

I am interested in applying the Rasch model to data from a
complex
sample design.
I was wondering if this has been done?
For example, is the nature of the PISA sample design explicitly
taken
into account when the Rasch estimates are produced?

Thanks,
Randy MacIntosh

-------------------------------------------------
Please consider the environment before you print
_______________________________________________
Rasch mailing list
Rasch at acer.edu.au
<http://us.mc1115.mail.yahoo.com/mc/compose?to=Rasch@acer.edu.au>
https://mailinglist.acer.edu.au/mailman/listinfo/rasch

-------------------------------------------------
Please consider the environment before you print
_______________________________________________
Rasch mailing list
Rasch at acer.edu.au
<http://us.mc1115.mail.yahoo.com/mc/compose?to=Rasch@acer.edu.au>
https://mailinglist.acer.edu.au/mailman/listinfo/rasch

-------------------------------------------------
Please consider the environment before you print
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20100216/838ddd98/attachment.html
```