[Rasch] Reliability of assessments run in circuits

Imogene Rothnie irothnie at usyd.edu.au
Fri Feb 22 12:48:47 EST 2013


Thanks very much Mike, I'm not sure what you mean by 'comment out'? Delete?


From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf Of Mike Linacre
Sent: Friday, February 22, 2013 12:47 PM
To: rasch at acer.edu.au
Subject: Re: [Rasch] Reliability of assessments run in circuits

Thank you for your question, Imogene.

1. If you want the examinee reliability for each circuit based on their overall measures:
With Facets, analyze the 24 circuits. Output an Anchorfile=af.txt with anchor values. Comment out all the circuits except one. Use the anchor file af.txt as the specification file of a Facets analysis. The candidate reliability for that circuit is reported.

2. 1. If you want the examinee reliability for each circuit based on their measures for that circuit:
With the Facets specification file for 24 circuits, comment out all the circuits except one. Use that as the specification file of a Facets analysis. The candidate reliability for that circuit is reported.

OK?

Mike L.

On 2/21/2013 17:13 PM, Imogene Rothnie wrote:
I have run a FACETS analysis on an assessment where candidates perform 5 tasks out of a pool of 33 tasks. The 33 tasks are organized into 24 circuits of 5 items, all sharing a common anchor item.
There isn't much spread of difficulty between the tasks, and they are all actually fairly similar, and so the average difficulty of each circuit does not vary by any significant amount (important).
 HOWEVER, I am trying to ascertain the reliability of the assessment, per circuit, as these are the scores we rank candidates on, not the reliability reported for the 33 items overall as a facet of measurement .
 I have included 'circuit' as a group anchor dummy facet, so am assuming the reported (sample) reliability of 0.96 is not to be interpreted as , say, a Cronbach's alpha like we might do for other facets.  Does anyone know how I might be able to calculate or find output in FACETS that can reveal reliability of the assessment, for the candidate experience of 5 items?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20130222/fdbce695/attachment.html 


More information about the Rasch mailing list