[Rasch] Is reliability reliable?

Bond, Trevor trevor.bond at jcu.edu.au
Tue Dec 9 11:52:09 EST 2014


Dear PDF
How can a measure discriminate consistently, if it does not discriminate at all?
TGB


Sent from 007's iPad

On 9 Dec 2014, at 8:25 am, "Parisa Daftari Fard" <pdaftaryfard at yahoo.com<mailto:pdaftaryfard at yahoo.com>> wrote:

Thank you Trevor and Thomas ,

I have to retype my reply as I was suddenly disconnected:-)

The sample, I believe, indeed had problems and I agree with you. My friend has coded it wrongly and it was skewed too. But the point is that statistics are mostly in favor of normal distribution not skewed sample.

We have the concept of dependability instead of reliability and to me dependability is subjective just a story like what raters do but the point is that you can test raters in a more objective procedure like FACET but not dependability. It is not objective like reliability. Very easily one can change the sample feature and nature to normal distribution.

Now the question remains. Is reliability an indicator of consistency or discrimination? if consistency is a matter, why should it be changed by sampling nature. If it is for the second purpose, i.e., discrimination, why cannot it be used for negatively skewed which is the most wanted destination in educational setting.

What is bugging me here is not reliability per se But if you surf through oceans of articles in education, you can easily notice that researchers are mostly worried about reliable variance  than result.

Cordially,
Parisa

Parisa Daftarifard
PhD candidate of TEFL
IAU (South Tehran Branch)




On Monday, December 8, 2014 2:45 AM, "Bond, Trevor" <trevor.bond at jcu.edu.au<mailto:trevor.bond at jcu.edu.au>> wrote:


Dear Parisa,
Have a look at two statistics
The range of the person measures...too small?
The SEs of your person measures.... Too big?
Both?
TGB


Sent from 007's iPad

On 8 Dec 2014, at 6:52 am, "Thomas Salzberger" <thomas.salzberger at gmail.com<mailto:thomas.salzberger at gmail.com>> wrote:

Parisa,

I can't refer to your specific problem because it is beyond my expertise, but maybe some very general considerations as to reliabily help.

Although the definition of various reliability coefficients vary (some assume parallel items, some don't, etc), they are all based on the idea of TRUE VARIANCE / TOTAL VARIANCE.

with TRUE VARIANCE + ERROR VARIANCE = TOTAL VARIANCE.

Error variance is a property of the test (maybe even this can be disputed), but true variance is a property of the sample you have.
Thus, reliability is a property of the test AND the sample. It tells you how well the test discriminates between the subjects. If the subjects don't vary, there is nothing to discriminate between.

As you say, adding more extreme persons increases reliability (all other things being equal).

Interpreted as such, reliablity is, I believe, a useful statistic. But it should not be over-interpreted.
It is relative to the sample (population) of interest (or the sample you happen to have).

Skewed distributions are another issue.

Best
Thomas

2014-12-07 16:14 GMT+01:00 Parisa Daftari Fard <pdaftaryfard at yahoo.com<mailto:pdaftaryfard at yahoo.com>>:
Dear Friends, mentors, and colleagues,

I would like to raise a question referring to  my previous question which was based on stunning result on a small sample data and reading through new methodologies in sociocultural camp like measuring dynamicity and growth of different traits in group or individuals.

New articles using dynamic assessment (DA) may frequently report person reliability of zero for construct, as the purpose of DA is to skewed individuals negatively through increasing individuals' Zone of proximal development. Furthermore, Test Low reliability can be ameliorated by having more capable individuals answering questions. Now my question is that is reliability index reliable enough to rely on?

I do apologize if my question sounds novice. But is it possible to say that our interpretation of reliability is relative not fixed? I came across some new articles on DA reporting low reliability as indicator of working model!

Best Regards
Parisa


Parisa Daftarifard
PhD candidate of TEFL
Iran
Faculty member of IAU (South Tehran Branch)

________________________________________
Rasch mailing list
email: Rasch at acer.edu.au<mailto:Rasch at acer.edu.au>
web: https://mailinglist.acer.edu.au/mailman/options/rasch/thomas.salzberger%40gmail.com



--
___________________________________
Thomas.Salzberger at gmail.com<mailto:Thomas.Salzberger at gmail.com>
Thomas.Salzberger at wu.ac.at<mailto:Thomas.Salzberger at wu.ac.at>

________________________________________
Rasch mailing list
email: Rasch at acer.edu.au<mailto:Rasch at acer.edu.au>
web: https://mailinglist.acer.edu.au/mailman/options/rasch/trevor.bond%40jcu.edu.au

________________________________________
Rasch mailing list
email: Rasch at acer.edu.au<mailto:Rasch at acer.edu.au>
web: https://mailinglist.acer.edu.au/mailman/options/rasch/pdaftaryfard%40yahoo.com


________________________________________
Rasch mailing list
email: Rasch at acer.edu.au<mailto:Rasch at acer.edu.au>
web: https://mailinglist.acer.edu.au/mailman/options/rasch/trevor.bond%40jcu.edu.au
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20141209/d50fb380/attachment-0001.html 


More information about the Rasch mailing list