[Rasch] Is reliability reliable?

Parisa Daftari Fard pdaftaryfard at yahoo.com
Tue Dec 9 12:15:03 EST 2014

Is it possible to say there is no discrimination? We have less or high discrimination. Let me restate my question, In pretest we expect less difference, in posttest , as the model should work, again improvement is what is expected. there is a sheer discrimination but can we say that our result is not reliable? 

Concerning my previous report, when I recode data I come up with the followings 
Number of students: 33minimum = 0maximum = 5number of questions = 8mean= 1.5 which shows the test was not appropriate for the sample; too difficultSTD: 1.14Variance: 1.320STD Error mean: 0.2Alpha: 0.087 which is very very very low but not negative 
Is this what I should read from data. I have done the analysis from the very beginning on the sample data and I recoded the test. I can send it in rasch format in an off list email.
in Rasch person reliability is zero but item reliability is 0.52I hope not to disappoint you :-(


  On Tuesday, December 9, 2014 4:22 AM, "Bond, Trevor" <trevor.bond at jcu.edu.au> wrote:

 Dear PDFHow can a measure discriminate consistently, if it does not discriminate at all?TGB

Sent from 007's iPad
On 9 Dec 2014, at 8:25 am, "Parisa Daftari Fard" <pdaftaryfard at yahoo.com> wrote:

Thank you Trevor and Thomas ,
I have to retype my reply as I was suddenly disconnected:-) 
The sample, I believe, indeed had problems and I agree with you. My friend has coded it wrongly and it was skewed too. But the point is that statistics are mostly in favor of normal distribution not skewed sample. 
We have the concept of dependability instead of reliability and to me dependability is subjective just a story like what raters do but the point is that you can test raters in a more objective procedure like FACET but not dependability. It is not objective like reliability. Very easily one can change the sample feature and nature to normal distribution. 
Now the question remains. Is reliability an indicator of consistency or discrimination? if consistency is a matter, why should it be changed by sampling nature. If it is for the second purpose, i.e., discrimination, why cannot it be used for negatively skewed which is the most wanted destination in educational setting. 
What is bugging me here is not reliability per se But if you surf through oceans of articles in education, you can easily notice that researchers are mostly worried about reliable variance  than result.
Parisa DaftarifardPhD candidate of TEFLIAU (South Tehran Branch)

On Monday, December 8, 2014 2:45 AM, "Bond, Trevor" <trevor.bond at jcu.edu.au> wrote:

Dear Parisa,Have a look at two statisticsThe range of the person measures...too small?The SEs of your person measures.... Too big?Both?TGB

Sent from 007's iPad
On 8 Dec 2014, at 6:52 am, "Thomas Salzberger" <thomas.salzberger at gmail.com> wrote:


I can't refer to your specific problem because it is beyond my expertise, but maybe some very general considerations as to reliabily help.

Although the definition of various reliability coefficients vary (some assume parallel items, some don't, etc), they are all based on the idea of TRUE VARIANCE / TOTAL VARIANCE.


Error variance is a property of the test (maybe even this can be disputed), but true variance is a property of the sample you have.
Thus, reliability is a property of the test AND the sample. It tells you how well the test discriminates between the subjects. If the subjects don't vary, there is nothing to discriminate between.

As you say, adding more extreme persons increases reliability (all other things being equal).

Interpreted as such, reliablity is, I believe, a useful statistic. But it should not be over-interpreted.
It is relative to the sample (population) of interest (or the sample you happen to have).

Skewed distributions are another issue. 


2014-12-07 16:14 GMT+01:00 Parisa Daftari Fard<pdaftaryfard at yahoo.com>:

Dear Friends, mentors, and colleagues,
I would like to raise a question referring to  my previous question which was based on stunning result on a small sample data and reading through new methodologies in sociocultural camp like measuring dynamicity and growth of different traits in group or individuals. 
New articles using dynamic assessment (DA) may frequently report person reliability of zero for construct, as the purpose of DA is to skewed individuals negatively through increasing individuals' Zone of proximal development. Furthermore, Test Low reliability can be ameliorated by having more capable individuals answering questions. Now my question is that is reliability index reliable enough to rely on?
I do apologize if my question sounds novice. But is it possible to say that our interpretation of reliability is relative not fixed? I came across some new articles on DA reporting low reliability as indicator of working model! 
Best RegardsParisa

Parisa DaftarifardPhD candidate of TEFLIranFaculty member of IAU (South Tehran Branch)
Rasch mailing list
email: Rasch at acer.edu.au
web: https://mailinglist.acer.edu.au/mailman/options/rasch/thomas.salzberger%40gmail.com

Thomas.Salzberger at gmail.com
Thomas.Salzberger at wu.ac.at

Rasch mailing list
email: Rasch at acer.edu.au
web: https://mailinglist.acer.edu.au/mailman/options/rasch/trevor.bond%40jcu.edu.au

Rasch mailing list
email: Rasch at acer.edu.au
web: https://mailinglist.acer.edu.au/mailman/options/rasch/pdaftaryfard%40yahoo.com

Rasch mailing list
email: Rasch at acer.edu.au
web: https://mailinglist.acer.edu.au/mailman/options/rasch/trevor.bond%40jcu.edu.au

Rasch mailing list
email: Rasch at acer.edu.au
web: https://mailinglist.acer.edu.au/mailman/options/rasch/pdaftaryfard%40yahoo.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20141209/ecb4d844/attachment.html 

More information about the Rasch mailing list