[Rasch] Reliability of assessments run in circuits

Mike Linacre mike at winsteps.com
Fri Feb 22 12:46:54 EST 2013

```Thank you for your question, Imogene.

1. If you want the examinee reliability for each circuit based on their
overall measures:
With Facets, analyze the 24 circuits. Output an Anchorfile=af.txt with
anchor values. Comment out all the circuits except one. Use the anchor
file af.txt as the specification file of a Facets analysis. The
candidate reliability for that circuit is reported.

2. 1. If you want the examinee reliability for each circuit based on
their measures for that circuit:
With the Facets specification file for 24 circuits, comment out all the
circuits except one. Use that as the specification file of a Facets
analysis. The candidate reliability for that circuit is reported.

OK?

Mike L.

On 2/21/2013 17:13 PM, Imogene Rothnie wrote:
> I have run a FACETS analysis on an assessment where candidates perform
> 5 tasks out of a pool of 33 tasks. The 33 tasks are organized into 24
> circuits of 5 items, all sharing a common anchor item.
>
> There isn't much spread of difficulty between the tasks, and they are
> all actually fairly similar, and so the average difficulty of each
> circuit does not vary by any significant amount (important).
>
> HOWEVER, I am trying to ascertain the reliability of the assessment,
> per circuit, as these are the scores we rank candidates on, not the
> reliability reported for the 33 items overall as a facet of measurement .
>
> I have included 'circuit' as a group anchor dummy facet, so am
> assuming the reported (sample) reliability of 0.96 is not to be
> interpreted as , say, a Cronbach's alpha like we might do for other
> facets.  Does anyone know how I might be able to calculate or find
> output in FACETS that can reveal reliability of the assessment, for
> the candidate experience of 5 items?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20130221/8b55711e/attachment.html
```