[Rasch] Not a Fan of Lexiles?
Mike Linacre (RMT)
rmt at rasch.org
Wed Oct 31 13:04:07 EST 2007
Paul and everyone:
So why haven't those achievements been made prior to Rasch?
"I thought they were."
Yes, we haven't yet experienced a Rasch "killer application". Isaac Newton
didn't have a "killer application" until a new comet appeared in the sky.
Before that, most astronomers said something like "Newton isn't telling us
anything about astronomy that we couldn't get from Ptolemy".
But many, many times I've shown clients and students the results of a Rasch
analysis of their own data, and seen them experience an "Aha!" moment, much
as experienced by the poet John Keats (On first looking into Chapman's Homer):
"Then felt I like some watcher of the skies
When a new planet swims into his ken;"
Rasch reanalysis of datasets from major test publishers (who have used
classical or IRT methods) nearly always reveals an obvious flaw (such as an
item miskey) that they have overlooked. Of course, their psychometricians
should have caught the problem. But they didn't. In the fog of their
analysis the obvious wasn't noticed. In the case of the MMPI-2, their rigid
thinking deliberately included item miskeys:
http://www.rasch.org/rmt/rmt81f.htm - Years later some of the MMPI-2
development team attended one of my workshops, and confirmed the validity
of that Rasch piece.
And then there are the item and person hierarchies. IRT and Classical
analyses usually ignore the item content. It is not part of their mindset.
The Rasch approach attempts to make sense of the item difficulty order in
order to understand what the test is measuring. Is it measuring what was
intended? Years ago I consulted with the test developers at a medical
board. Rasch analysis revealed that their items had no meaningful
hierarchy. Of course, once I pointed that out, they admitted that was the
case. They had suspected as much, but it was only a "gut feeling". We
discovered that their certification examination was more like a game of
"trivial pursuit" than an investigation into clinician competence. But the
examination board was locked into what they were doing, so they continued
their merry way. Fortunately no lawyers knew about it ....
And then there are adjustments for rater severity ... something essentially
unknown before Rasch ....
Editor, Rasch Measurement Transactions
rmt at rasch.org www.rasch.org/rmt/ Latest RMT: 21:1 Summer 2007
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch