[Rasch] Calibrating ratee ability and rater leniency/severity with data not connected

Kenji Yamazaki kyamazaki at acgme.org
Wed Dec 21 08:26:41 EST 2011

Dear Rasch professionals:

I have a question for you.  I hope you can help me with this matter.  My question is about how to calibrate items, persons (i.e., ratee), and raters using FACETS on performance rating data, although the data are not connected.

For the sake of argument, I describe the situation less complicated.  The dataset I analyze includes resident performance data on their professionalism/communication skills, which were collected from 12 independent resident programs.  Across all programs, 72 residents' performance scores were provided by their respective attending physicians (n = 171) using a 20-item performance rating instrument.  Because no residents or attending physicians belonged to more than one resident program, both facets were completed nested within resident programs.  In other words, there were no connections of performance rating data across 12 resident programs.

Given this condition, what I would like to do is to place resident abilities, item difficulties, and attending physicians' severity/leniency on the same scale so that resident performance levels in particular can be comparable across 12 resident programs.

The website (http://www.rasch.org/rn3.htm ) explains solutions under 13.3.1 Connecting final data saying, "Either the tasks are 'said to be alike' or the people are 'said to be alike'".
So, my questions would be:

(1)    What should I do for my specific case, if I follow the solution, "the tasks are 'said to be alike'"?  Does that mean that I fix item difficulties of the 20 items, and calibrate resident ability and attending physician leniency/severity for each of 12 programs?

(2)    Could you explain what to do, if I follow the solution, "the people are 'said to be alike'"?  In my view, because residents or attending physicians do not belong to more than one program, I have difficulty in understanding this instruction.

(3)    Is there any other way to achieve my analysis goal?
Thank you very much in advance.

Kenji Yamazaki
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20111220/5568253f/attachment.html 

More information about the Rasch mailing list