[Rasch] IRT based reliability

Mike Linacre mike at winsteps.com
Wed Jun 26 22:16:59 EST 2013


Thank you for your questions, Ou Zhang.

  1. What index do you use in calculating IRT based reliability?

Reply: We all use Spearman (1910) Reliability coefficients = "true" 
variance / observed variance, computed for the current person sample.
In CTT, these are known as KR-20 and Cronbach Alpha and are computed 
from raw scores. In Rasch methodology, Reliability is computed from the 
estimated person measures (locations) and their standard errors. The 
formula is:
"Test" Reliability = (Observed Person Measure Variance - Mean Person 
Error Variance) / (Observed Person Measure Variance)

2. How to model the reliability in CAT?

Reply: We can compute CAT Rasch reliability using the formula in 1., but 
the Reliability coefficient will be lower than for a fixed-length test 
because the CAT tests are shorter and so the Error Variances are larger. 
If we want higher CAT Reliability, then we administer more items to each 
person. However, rather than Reliability for a person sample, CAT test 
design focuses on the precise measurement of persons near cut-points or 
in crucial measurement ranges. There is usually no advantage in 
attempting to obtain high precision (= high Reliability) for very low or 
very high performers.

Can anyone suggest good References?

Mike L.






More information about the Rasch mailing list