[Rasch] IRT based reliability
mike at winsteps.com
Wed Jun 26 22:16:59 EST 2013
Thank you for your questions, Ou Zhang.
1. What index do you use in calculating IRT based reliability?
Reply: We all use Spearman (1910) Reliability coefficients = "true"
variance / observed variance, computed for the current person sample.
In CTT, these are known as KR-20 and Cronbach Alpha and are computed
from raw scores. In Rasch methodology, Reliability is computed from the
estimated person measures (locations) and their standard errors. The
"Test" Reliability = (Observed Person Measure Variance - Mean Person
Error Variance) / (Observed Person Measure Variance)
2. How to model the reliability in CAT?
Reply: We can compute CAT Rasch reliability using the formula in 1., but
the Reliability coefficient will be lower than for a fixed-length test
because the CAT tests are shorter and so the Error Variances are larger.
If we want higher CAT Reliability, then we administer more items to each
person. However, rather than Reliability for a person sample, CAT test
design focuses on the precise measurement of persons near cut-points or
in crucial measurement ranges. There is usually no advantage in
attempting to obtain high precision (= high Reliability) for very low or
very high performers.
Can anyone suggest good References?
More information about the Rasch