[Rasch] a scaling issue?
svkr at sund.ku.dk
Thu Aug 30 15:47:33 EST 2012
Have they checked the models or are they just estimating parameters? In principle results should be the same if items fit the Rasch model except that estimates from the 3pl model will be much more imprecise because they estimate parameters that do no exist.
If items do not fit the Rasch model anything can happen and using a 3pl model will not automatically solve the problem because some of the claims of these models make less than sense.
From: rasch-bounces at acer.edu.au [rasch-bounces at acer.edu.au] on behalf of Treder, David [dtreder at geneseeisd.org]
Sent: Thursday, August 30, 2012 3:35 AM
To: rasch at acer.edu.au
Subject: [Rasch] a scaling issue?
Not sure that this is the right place to bring this up, but though it might be a good place to start.
As a requirement of the federal NCLB Waiver, states were required to address the "achievement gap." As their answer, Michigan decided to identify schools with the largest "gap." The gap is based on the State's standardized tests: this is computed by subtracting the mean score top 30% percent of students from the mean score of the bottom 30% of students ( for each subject-area test).
The issue, and my question:
1) Some of the tests are scaled using 1 parameter IRT, while others are scaled using the 3 parameter IRT,.
2) Based on tests scaled with 1 P/IRT, the schools with "high achievement" (highest mean scale score) are far more likely to have a large gap (correlation between achievement and gap, around .70); for the tests scaled with 3 P/IRT, there's an opposite relationship - low achievement schools are far more likely to have a large gap (correlations around -.70)
Would anyone care to weigh in on why tests scaled with 1 P/IRT would identify diametrically different schools, on the "top minus bottom" index, than tests scaled with 3 P/IRT?
(or, point me to a place in the literature that talks about the differences in the way scores will be "spread out" in 1 P/IRT, vs 3 P/IRT).
Rasch mailing list
Rasch at acer.edu.au
More information about the Rasch