[Rasch] Negative pt-bis and fit of 1.0? How can this be?

Stuart Luppescu slu at ccsr.uchicago.edu
Wed Mar 7 08:36:34 EST 2012


On Tue, 2012-03-06 at 13:21 -0800, Mark Moulton wrote:
> Stuart,
> 
> Great chart!
> 
> Although there is a straight-line relationship between the point biserials
> and fit, the fit statistics only range between 0.85 and 1.15, which all
> look "fitting" to the casual eye and would seem to be "good".  The point
> biserials, ranging from -0.20 to 0.40, tell the more useful story, which is
> that only around a third of the items (if that) are getting any traction on
> this test. 

Yes, good point.

>  The birds-eye explanation is that the data are so drenched in
> error that nothing misfits.

I don't get this at all. Seems oxymoronical. 

> I'm still thinking about the particulars of the probabilities going into
> your fit formula.  I surmise that the overall error of the test has caused
> the person/item distributions to shrink and regress toward each other and
> produce probabilities near 0.50, which would cause the denominators in the
> cell fit formula to be maximized, which would minimize the appearance of
> misfit.  I think if you loaded  ones and zeros randomly into a data matrix
> and Rasch analyzed it you would get a similar pattern.  (I wasn't kidding
> when I said a random number generator would be cheaper!)

Are you saying that if I just generated 1's and 0's randomly and tried
to calibrate them they would all fit? Hmmm. I'm going to have to try
that....

> I think this is a good example of why the usual fit statistics are often
> harder to interpret than good old point-biserials.  The misfit scale
> changes as a function of overall measurement noise and has to be
> interpreted accordingly. 

Yes, I've always wondered about this.

>  The point-biserial scale can be interpreted in
> pretty much the same way regardless of underlying noise.  A zero or
> negative point biserial is always bad.  A point biserial greater than 0.5
> or so is always good.

Stay tuned for my random data calibration experiment results.

-- 
Stuart Luppescu -=- slu .at. ccsr.uchicago.edu        
University of Chicago -=- CCSR 
才文と智奈美の父 -=-    Kernel 3.2.1-gentoo-r2                
Gregor Gorjanc: But imagine how hard would it be to
 have two separate modes ... argh, probably a
 mess^2 or have I missed something obvious. Martin 
 Maechler: Yes, exactly: "Mess ^ 2" -- and if you
 allow both 'drop' and 'na.rm' options, it's "Mess 
 ^ 3" -- not something anyone really wants!    --
 Gregor Gorjanc and Martin Maechler (in a
 discussion whether 'drop' should       be set in
<options(), possibly depending on different user




More information about the Rasch mailing list