[Rasch] Not a Fan of Lexiles?

Mike Linacre (RMT) rmt at rasch.org
Wed Oct 31 13:04:07 EST 2007


Paul and everyone:

So why haven't those achievements been made prior to Rasch?
"I thought they were."

Yes, we haven't yet experienced a Rasch "killer application".  Isaac Newton 
didn't have a "killer application" until a new comet appeared in the sky. 
Before that, most astronomers said something like "Newton isn't telling us 
anything about astronomy that we couldn't get from Ptolemy".

But many, many times I've shown clients and students the results of a Rasch 
analysis of their own data, and seen them experience an "Aha!" moment, much 
as experienced by the poet John Keats  (On first looking into Chapman's Homer):
"Then felt I like some watcher of the skies
  When a new planet swims into his ken;"

Rasch reanalysis of datasets from major test publishers (who have used 
classical or IRT methods) nearly always reveals an obvious flaw (such as an 
item miskey) that they have overlooked. Of course, their psychometricians 
should have caught the problem. But they didn't. In the fog of their 
analysis the obvious wasn't noticed. In the case of the MMPI-2, their rigid 
thinking deliberately included item miskeys: 
http://www.rasch.org/rmt/rmt81f.htm - Years later some of the MMPI-2 
development team attended one of my workshops, and confirmed the validity 
of that Rasch piece.

And then there are the item and person hierarchies. IRT and Classical 
analyses usually ignore the item content. It is not part of their mindset. 
The Rasch approach attempts to make sense of the item difficulty order in 
order to understand what the test is measuring. Is it measuring what was 
intended? Years ago I consulted with the test developers at a medical 
board. Rasch analysis revealed that their items had no meaningful 
hierarchy. Of course, once I pointed that out, they admitted that was the 
case. They had suspected as much, but it was only a "gut feeling". We 
discovered that their certification examination was more like a game of 
"trivial pursuit" than an investigation into clinician competence. But the 
examination board was locked into what they were doing, so they continued 
their merry way. Fortunately no lawyers knew about it ....

And then there are adjustments for rater severity ... something essentially 
unknown before Rasch ....

Mike L.

Mike Linacre
Editor, Rasch Measurement Transactions
rmt at rasch.org www.rasch.org/rmt/ Latest RMT:  21:1 Summer 2007
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20071030/0063e35c/attachment.html 


More information about the Rasch mailing list