[Rasch] Reliability of assessments run in circuits

Mike Linacre mike at winsteps.com
Fri Feb 22 12:46:54 EST 2013


Thank you for your question, Imogene.

1. If you want the examinee reliability for each circuit based on their 
overall measures:
With Facets, analyze the 24 circuits. Output an Anchorfile=af.txt with 
anchor values. Comment out all the circuits except one. Use the anchor 
file af.txt as the specification file of a Facets analysis. The 
candidate reliability for that circuit is reported.

2. 1. If you want the examinee reliability for each circuit based on 
their measures for that circuit:
With the Facets specification file for 24 circuits, comment out all the 
circuits except one. Use that as the specification file of a Facets 
analysis. The candidate reliability for that circuit is reported.

OK?

Mike L.

On 2/21/2013 17:13 PM, Imogene Rothnie wrote:
> I have run a FACETS analysis on an assessment where candidates perform 
> 5 tasks out of a pool of 33 tasks. The 33 tasks are organized into 24 
> circuits of 5 items, all sharing a common anchor item.
>
> There isn't much spread of difficulty between the tasks, and they are 
> all actually fairly similar, and so the average difficulty of each 
> circuit does not vary by any significant amount (important).
>
> HOWEVER, I am trying to ascertain the reliability of the assessment, 
> per circuit, as these are the scores we rank candidates on, not the 
> reliability reported for the 33 items overall as a facet of measurement .
>
> I have included 'circuit' as a group anchor dummy facet, so am 
> assuming the reported (sample) reliability of 0.96 is not to be 
> interpreted as , say, a Cronbach's alpha like we might do for other 
> facets.  Does anyone know how I might be able to calculate or find 
> output in FACETS that can reveal reliability of the assessment, for 
> the candidate experience of 5 items?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20130221/8b55711e/attachment.html 


More information about the Rasch mailing list