[Rasch] Is reliability reliable?

Thomas Salzberger thomas.salzberger at gmail.com
Mon Dec 8 09:52:03 EST 2014


Parisa,

I can't refer to your specific problem because it is beyond my expertise,
but maybe some very general considerations as to reliabily help.

Although the definition of various reliability coefficients vary (some
assume parallel items, some don't, etc), they are all based on the idea of
TRUE VARIANCE / TOTAL VARIANCE.

with TRUE VARIANCE + ERROR VARIANCE = TOTAL VARIANCE.

Error variance is a property of the test (maybe even this can be disputed),
but true variance is a property of the sample you have.
Thus, reliability is a property of the test AND the sample. It tells you
how well the test discriminates between the subjects. If the subjects don't
vary, there is nothing to discriminate between.

As you say, adding more extreme persons increases reliability (all other
things being equal).

Interpreted as such, reliablity is, I believe, a useful statistic. But it
should not be over-interpreted.
It is relative to the sample (population) of interest (or the sample you
happen to have).

Skewed distributions are another issue.

Best
Thomas

2014-12-07 16:14 GMT+01:00 Parisa Daftari Fard <pdaftaryfard at yahoo.com>:

> Dear Friends, mentors, and colleagues,
>
> I would like to raise a question referring to  my previous question which
> was based on stunning result on a small sample data and reading through new
> methodologies in sociocultural camp like measuring dynamicity and growth of
> different traits in group or individuals.
>
> New articles using dynamic assessment (DA) may frequently report person
> reliability of zero for construct, as the purpose of DA is to skewed
> individuals negatively through increasing individuals' Zone of proximal
> development. Furthermore, Test Low reliability can be ameliorated by having
> more capable individuals answering questions. Now my question is that* is
> reliability index reliable enough to rely on?*
>
> I do apol*ogize if my question sounds novice. But *is it possible to say
> that our interpretation of reliability is relative not fixed? I came across
> some new articles on DA reporting low reliability as indicator of working
> model!
>
> Best Regards
> Parisa
>
>
> Parisa Daftarifard
> PhD candidate of TEFL
> Iran
> Faculty member of IAU (South Tehran Branch)
>
> ________________________________________
> Rasch mailing list
> email: Rasch at acer.edu.au
> web:
> https://mailinglist.acer.edu.au/mailman/options/rasch/thomas.salzberger%40gmail.com
>



-- 
___________________________________
Thomas.Salzberger at gmail.com
Thomas.Salzberger at wu.ac.at
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20141207/1ed97520/attachment.html 


More information about the Rasch mailing list