[Rasch] Negative pt-bis and fit of 1.0? How can this be?
Mark Moulton
markhmoulton at gmail.com
Wed Mar 7 08:21:19 EST 2012
Stuart,
Great chart!
Although there is a straight-line relationship between the point biserials
and fit, the fit statistics only range between 0.85 and 1.15, which all
look "fitting" to the casual eye and would seem to be "good". The point
biserials, ranging from -0.20 to 0.40, tell the more useful story, which is
that only around a third of the items (if that) are getting any traction on
this test. The birds-eye explanation is that the data are so drenched in
error that nothing misfits.
I'm still thinking about the particulars of the probabilities going into
your fit formula. I surmise that the overall error of the test has caused
the person/item distributions to shrink and regress toward each other and
produce probabilities near 0.50, which would cause the denominators in the
cell fit formula to be maximized, which would minimize the appearance of
misfit. I think if you loaded ones and zeros randomly into a data matrix
and Rasch analyzed it you would get a similar pattern. (I wasn't kidding
when I said a random number generator would be cheaper!)
I think this is a good example of why the usual fit statistics are often
harder to interpret than good old point-biserials. The misfit scale
changes as a function of overall measurement noise and has to be
interpreted accordingly. The point-biserial scale can be interpreted in
pretty much the same way regardless of underlying noise. A zero or
negative point biserial is always bad. A point biserial greater than 0.5
or so is always good.
Mark Moulton
On Tue, Mar 6, 2012 at 12:25 PM, Stuart Luppescu <slu at ccsr.uchicago.edu>wrote:
> On Tue, 2012-03-06 at 11:28 -0800, Mark Moulton wrote:
> > I've kind of come around to the same point of view as you. Fit
> > statistics are much more complex than point-biserials (or their Rasch
> > equivalent) and are driven by lots of factors, making them harder to
> > interpret. Does an item fit the construct of a test? A fit statistic
> > may or may not give a clear answer to that question. Point-biserials
> > almost always do.
>
> In this case, I plotted the point-biserials against the fit statistics
> and came out with a (nearly) perfect straight line. It doesn't look like
> either is any better than the other. (I also did it for the standardized
> fit statistic and that plot looks similar, but slightly curved.) The
> plot can be viewed here:
>
> https://webshare.uchicago.edu/xythoswfs/webui/_xy-4342733_1-t_1rlZ33Vp
> --
> Stuart Luppescu -=- slu .at. ccsr.uchicago.edu
> University of Chicago -=- CCSR
> 才文と智奈美の父 -=- Kernel 3.2.1-gentoo-r2
> Marc R. Feldesman: I'm trying to figure out what
> could have possibly changed [...] that make Brian
> Ripley's posts (in particular) not show up on my
> mail server [...] Peter Dalgaard: Did you remember
> to turn off the Oxford-sarcasm filter? -- Marc
> R. Feldesman and Peter Dalgaard (Marc R. Feldesman
> having some e-mail problems) R-help
> (January 2005)
>
>
> _______________________________________________
> Rasch mailing list
> Rasch at acer.edu.au
> Unsubscribe:
> https://mailinglist.acer.edu.au/mailman/options/rasch/markm%40eddata.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20120306/63ef7536/attachment.html
More information about the Rasch
mailing list