[Rasch] a scaling issue?
dtreder at geneseeisd.org
Thu Aug 30 11:35:34 EST 2012
Not sure that this is the right place to bring this up, but though it might be a good place to start.
As a requirement of the federal NCLB Waiver, states were required to address the "achievement gap." As their answer, Michigan decided to identify schools with the largest "gap." The gap is based on the State's standardized tests: this is computed by subtracting the mean score top 30% percent of students from the mean score of the bottom 30% of students ( for each subject-area test).
The issue, and my question:
1) Some of the tests are scaled using 1 parameter IRT, while others are scaled using the 3 parameter IRT,.
2) Based on tests scaled with 1 P/IRT, the schools with "high achievement" (highest mean scale score) are far more likely to have a large gap (correlation between achievement and gap, around .70); for the tests scaled with 3 P/IRT, there's an opposite relationship - low achievement schools are far more likely to have a large gap (correlations around -.70)
Would anyone care to weigh in on why tests scaled with 1 P/IRT would identify diametrically different schools, on the "top minus bottom" index, than tests scaled with 3 P/IRT?
(or, point me to a place in the literature that talks about the differences in the way scores will be "spread out" in 1 P/IRT, vs 3 P/IRT).
More information about the Rasch