[Rasch] Calibrating ratee ability and rater leniency/severity with data not connected
kyamazaki at acgme.org
Thu Dec 22 08:01:17 EST 2011
I was wondering if you could lead me to studies that have used group-anchoring so that ratee abilities can be comparable across disjoin subsets. I tried to find an example, but have hard time to find any because of my unfamiliarity of this method.
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf Of Mike Linacre
Sent: Wednesday, December 21, 2011 1:08 AM
To: rasch at acer.edu.au
Subject: Re: [Rasch] Calibrating ratee ability and rater leniency/severity with data not connected
Thank you for your questions, Kenji Yamazaki.
Your experimental design includes a 20-item instrument that was used everywhere (good). But there are 12 separate resident programs. Each has its own attending physicians (raters) and residents (ratees). The main purpose is to compare resident performance levels.
In this design, there are 12 disjoint subsets of ratings corresponding to the 12 resident programs. To make the measures of the 12 programs comparable, we must make some assumptions:
If we want to compare ratee abilities across programs, then we assume that the 12 samples of raters have the same mean leniency. In Facets, group-anchor the 12 groups of raters (attending physicians) at the same value, usually zero. Non-center the ratee facet.
Since the same 20 items are used everywhere, no additional action is required to make the item difficulties comparable everywhere.
Alternatively, if we want to compare raters severities/leniencies across programs, then we assume that the 12 samples of ratees have the same mean ability.
Editor, Rasch Measurement Transactions
rmt at rasch.org www.rasch.org/rmt/<http://www.rasch.org/rmt/> Latest RMT: 25:3 Winter 2011
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch