[Rasch] Negative pt-bis and fit of 1.0? How can this be?

Stuart Luppescu slu at ccsr.uchicago.edu
Wed Mar 7 08:36:34 EST 2012

```On Tue, 2012-03-06 at 13:21 -0800, Mark Moulton wrote:
> Stuart,
>
> Great chart!
>
> Although there is a straight-line relationship between the point biserials
> and fit, the fit statistics only range between 0.85 and 1.15, which all
> look "fitting" to the casual eye and would seem to be "good".  The point
> biserials, ranging from -0.20 to 0.40, tell the more useful story, which is
> that only around a third of the items (if that) are getting any traction on
> this test.

Yes, good point.

>  The birds-eye explanation is that the data are so drenched in
> error that nothing misfits.

I don't get this at all. Seems oxymoronical.

> I'm still thinking about the particulars of the probabilities going into
> your fit formula.  I surmise that the overall error of the test has caused
> the person/item distributions to shrink and regress toward each other and
> produce probabilities near 0.50, which would cause the denominators in the
> cell fit formula to be maximized, which would minimize the appearance of
> misfit.  I think if you loaded  ones and zeros randomly into a data matrix
> and Rasch analyzed it you would get a similar pattern.  (I wasn't kidding
> when I said a random number generator would be cheaper!)

Are you saying that if I just generated 1's and 0's randomly and tried
to calibrate them they would all fit? Hmmm. I'm going to have to try
that....

> I think this is a good example of why the usual fit statistics are often
> harder to interpret than good old point-biserials.  The misfit scale
> changes as a function of overall measurement noise and has to be
> interpreted accordingly.

>  The point-biserial scale can be interpreted in
> pretty much the same way regardless of underlying noise.  A zero or
> negative point biserial is always bad.  A point biserial greater than 0.5
> or so is always good.

Stay tuned for my random data calibration experiment results.

--
Stuart Luppescu -=- slu .at. ccsr.uchicago.edu
University of Chicago -=- CCSR

Gregor Gorjanc: But imagine how hard would it be to
have two separate modes ... argh, probably a
mess^2 or have I missed something obvious. Martin
Maechler: Yes, exactly: "Mess ^ 2" -- and if you
allow both 'drop' and 'na.rm' options, it's "Mess
^ 3" -- not something anyone really wants!    --
Gregor Gorjanc and Martin Maechler (in a
discussion whether 'drop' should       be set in
<options(), possibly depending on different user

```