[Rasch] Calibrating ratee ability and rater leniency/severity with data not connected
irothnie at usyd.edu.au
Sun Jan 8 14:16:57 EST 2012
Apologies not sure of correct email for posting to Rasch forum. Trying
to send below message.
I have a similar problem, and the group anchor doesn't seem to have
resolved the subset issue, so I am wondering if there is something else
I can do or how I need to interpret the analysis.
I have applicants taking an oral exam consisting of 9 seven minute
questions . The questions are very structured from the examiner point of
view and examiners are trained.
Because there are a large number of applicants being examined over a
number of weeks, different subsets of questions are used in 'circuits'.
The subsets of questions are partially crossed, ie one question is in
all circuits and many are in more than one. (there are about 50
questions across 13 circuits). Applicants are obviously completely
nested within circuits but examiners are partially crossed with circuits
(and questions) because many examiners examine on more than one day.
I ran the Facets model with examiner, applicant, qn (item) , and
question circuit as facets. Applicants and circuits were in up to 21
subsets. I group anchored the circuits because we do assume they are of
equal mean difficulty (to be fair) and we certainly don't assume
applicants to be of equal ability (they are ranked on their scores).
I still have 11 or so subsets, and I can't group anchor the applicants
obviously. Is there anything else I can do or is it a case of we must
have more crossing of items across circuits? When I run the analysis
without the 'circuit' facet there is no subset problem, but I'm not sure
how useful this analysis in determining 'fairness' of the experience
Any help much appreciated.
IMOGENE ROTHNIE | Lecturer (Assessment)
Office of Medical Education | Faculty of Medicine
THE UNIVERSITY OF SYDNEY
Rm 108, Edward Ford Bdg A27 | The University of Sydney | NSW | 2006
T +61 2 9036 6434 | F +61 2 9036 7580 | M +61 418 381 359
E imogene.rothnie at sydney.edu.au <mailto:imogene.rothnie at sydney.edu.au>
| W http://ome.med.usyd.edu.au <http://sydney.edu.au>
This email plus any attachments to it are confidential. Any unauthorised
use is strictly prohibited. If you receive this email in error, please
delete it and any attachments.
Please think of our environment and only print this e-mail if necessary.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch