[Rasch] Does Decision consistency = Reliability?

David Andrich david.andrich at uwa.edu.au
Sun Mar 1 18:21:13 AEDT 2020


Yes, but if you are also thinking of precision and consistency of response, then the apparent paradox is that consistency is inversely related to precision. It is explained in

Andrich, D. (1981). Stability of response, reliability, and accuracy of measurement. Educational and Psychological Measurement, 41 (2), 253-262.
David

David Andrich
Chapple Professor, Graduate School of Education
The University of Western Australia
M428, 35 Stirling Highway, Crawley,
Western AUSTRALIA, 6009
Telephone: +61 8 6488 1085;   Fax: +61 8 6488 1052
david.andrich at uwa.edu.au<mailto:david.andrich at uwa.edu.au>
GSE Psychometric Laboratory
http://www.education.uwa.edu.au/ppl/courses
www.matildabayclub.net<http://www.matildabayclub.net>
[cid:image005.gif at 01D4E47E.030505A0]<http://www.uwa.edu.au/university-campaigns-resources/emailsig2015/uwa-logo/>


From: Rasch <rasch-bounces at acer.edu.au> On Behalf Of Mike Linacre
Sent: Saturday, 29 February 2020 11:28 AM
To: rasch <rasch at acer.edu.au>
Subject: Re: [Rasch] Does Decision consistency = Reliability?

Hi Patrick:

Your question is not simple to answer.

Decision consistency?
1) Does a judge always make the same decision (give the same rating) under the same circumstances? = internal fit
2) Does a judge always make the same decision as the other judges under the same circumstances? = inter-judge reliability

Reliability?
1) Can the data reliably separate high performers from low performers? - Spearman reliability (Cronbach Alpha etc.)
2) Do judges reliably agree
(a) with the standard judge?
(b) in awarding the same ratings in the same circumstances?
(c) with which performances are better and which are worse?
These are different types of inter-judge reliability.

And other participants can probably think of more possibilities :-)

Mike L.

On 2/29/2020 5:54 AM, Patrick Fisher wrote:
Greeting fellow Raschies,
 Very simple question: does decision consistency = reliability?
 Thanks,
Patrick

--
Mike Linacre, mike at winsteps.com<mailto:mike at winsteps.com> or winsteps1234 at gmail.com<mailto:winsteps1234 at gmail.com>
Winsteps 4.5.1 and Facets 3.83.0 - www.winsteps.com<http://www.winsteps.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20200301/4ecd653a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 5442 bytes
Desc: image001.gif
URL: <https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20200301/4ecd653a/attachment-0001.gif>


More information about the Rasch mailing list