[Rasch] Fwd: Reliability, separation and strata
Fabio La Porta
fabiolaporta at mail.com
Wed Mar 1 04:14:15 AEDT 2017
Hi guys,
I have calibrated a clinical scale using a right skewed calibration sample
(mean person ability: +2.796 logits), which reflects the distribution of
the population to which the scale is administered in real life.
After some post-hoc modifications, the scale has fitted the Rasch model.
The reported reliability coefficient by RUMM (PSI) was 0.867, corresponding
to the following separation statistics:
- G (separation ratio)=2.553
- H (ability strata)=3.738
These values, based on the assumption that the sample follows a normal
distribution, were calculated using the formulas specified in
http://www.rasch.org/rmt/rmt94n.htm
<https://api.thetopinbox.com/track/v2/link?id=3dcc48b5-cfd7-42aa-80f4-1909c8f0e7e6&url=http%3A%2F%2Fwww.rasch.org%2Frmt%2Frmt94n.htm>
However, in view of the skewed distribution of the sample, I also
calculated the number of statistically distinct levels of performance (G)
using the method suggested in http://www.rasch.org/rmt/rmt144k.htm
<https://api.thetopinbox.com/track/v2/link?id=3dcc48b5-cfd7-42aa-80f4-1909c8f0e7e6&url=http%3A%2F%2Fwww.rasch.org%2Frmt%2Frmt144k.htm>,
in which it is made use of individual estimates and standard errors to
calculate the separation ratio (G) for any given total score value.
Using this method, a completely different picture emerged. Particularly,
the 'new' reliability estimates and coefficients calculated were:
- G (separation ratio)=7
- reliability=0.980
- H (ability strata)=9.667
In other words, unlike to what suggested by the classic method, the scale
seems to have an excellent reliability and separation capability.
So here are my two questions:
1) given the striking difference in values (and clinical meaning) between
the classic method and the second method to what extent can we trust the
first method to calculate G and H? In other words, are there any studies
suggesting when to start using the second method in lieu of the classic
method?
2) So far, I have always reported H, but in practical terms, it is easier
to divide the range of measurement according to the calculated G, as in the
attached table. Thus, I am now a bit confused about the difference between
G and H. Considering that these two statistics give different values, when
is it is useful to trust one rather than the other?
Thanks in advance
Fabio
TotSc / ExpVal
Location
Std Error
Minimum measure for next level
Separation index (G)
0.143
-6.312
1.41
1
1
-5.157
1.04
-2.814
1
2
-4.335
0.83
-3.039
1
3
-3.751
0.72
-3.146
1
4
-3.294
0.65
-3.208
1
5
-2.913
0.6
-3.246
2
6
-2.581
0.57
-1.259
2
7
-2.282
0.54
-1.294
2
8
-2.007
0.52
-1.319
2
9
-1.748
0.51
-1.338
2
10
-1.501
0.5
-1.350
2
11
-1.261
0.49
-1.359
3
12
-1.027
0.49
0.119
3
13
-0.795
0.48
0.115
3
14
-0.566
0.48
0.111
3
15
-0.339
0.48
0.108
3
16
-0.114
0.48
0.105
3
17
0.109
0.48
0.105
4
18
0.333
0.48
1.454
4
19
0.558
0.48
1.457
4
20
0.787
0.48
1.460
4
21
1.019
0.48
1.464
4
22
1.253
0.49
1.468
4
23
1.489
0.49
1.472
5
24
1.726
0.49
2.879
5
25
1.966
0.5
2.889
5
26
2.215
0.51
2.906
5
27
2.478
0.53
2.933
5
28
2.765
0.56
2.970
5
29
3.088
0.59
3.022
6
30
3.457
0.63
4.817
6
31
3.884
0.69
4.907
6
32
4.399
0.78
5.049
6
33
5.089
0.96
5.345
6
33.809
6.015
1.31
5.969
7
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20170228/88b0c0ef/attachment-0001.html>
More information about the Rasch
mailing list