[Rasch] Calibrating ratee ability and rater leniency/severity with data not connected

Mike Linacre rmt at rasch.org
Wed Dec 21 18:08:15 EST 2011


Thank you for your questions, Kenji Yamazaki.

Your experimental design includes a 20-item instrument that was used 
everywhere (good). But there are 12 separate resident programs. Each has 
its own attending physicians (raters) and residents (ratees). The main 
purpose is to compare resident performance levels.

In this design, there are 12 disjoint subsets of ratings corresponding to 
the 12 resident programs. To make the measures of the 12 programs 
comparable, we must make some assumptions:

If we want to compare ratee abilities across programs, then we assume that 
the 12 samples of raters have the same mean leniency. In Facets, 
group-anchor the 12 groups of raters (attending physicians) at the same 
value, usually zero. Non-center the ratee facet.

Since the same 20 items are used everywhere, no additional action is 
required to make the item difficulties comparable everywhere.

Alternatively, if we want to compare raters severities/leniencies across 
programs, then we assume that the 12 samples of ratees have the same mean 
ability.

OK?

Mike L.

Mike Linacre
Editor, Rasch Measurement Transactions
rmt at rasch.org www.rasch.org/rmt/ Latest RMT: 25:3 Winter 2011
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20111221/4aa131e9/attachment.html 


More information about the Rasch mailing list