[Rasch] Reliability in Facets

Stuart Luppescu slu at ccsr.uchicago.edu
Thu Feb 2 10:01:08 EST 2012


Hello fellow Raschies, I'm confused about the notion of reliability in a
facets analysis where teacher ratings are moderated by observers. As I
see it, there are two ways to conceptualize reliability: 

1) true variation among teachers / (true variation among teachers +
measurement error)
2) true variation among teachers / (variation among teachers + variation
among raters (severity) + variation among other facets + residual
variation)

(For the latter, I used the measure S.D. from the Facets output squared
for each source of variance, plus the variance of the residuals from the
residual file.)

In the work I'm doing analyzing teacher ratings data using the Danielson
Framework for Teaching, reliability 1) is .93, but reliability 2)
is .35. How do I reconcile these two? Are they two different kinds of
reliability?

-- 
Stuart Luppescu -=- slu .at. ccsr.uchicago.edu        
University of Chicago -=- CCSR 
才文と智奈美の父 -=-    Kernel 3.2.1-gentoo-r2                
Jason Horn: Can anyone think of a way to create a
 pretty() sequence that excludes zero? Or a way to 
 remove the zero from a sequence after using
 pretty()? Peter Dalgaard: The former is rather
 hard because zero is generally considered just
 about the prettiest number around...    -- Jason
 Horn and Peter Dalgaard       R-help (October
 2005)  
 




More information about the Rasch mailing list