[Rasch] Not a Fan of Lexiles?
commons at tiac.net
Wed Oct 31 13:21:05 EST 2007
We would not know if analytic methods of analyzing task difficult such the
model of hierarchical complexity were right.
Michael Lamport Commons, Ph.D.
Assistant Clinical Professor
Department of Psychiatry
Harvard Medical School
Beth Israel Deaconess Medical Center
234 Huron Avenue
Cambridge, MA 02138-1328
commons at tiac.net
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf
Of Mike Linacre (RMT)
Sent: Tuesday, October 30, 2007 10:04 PM
To: rasch at acer.edu.au
Subject: RE: [Rasch] Not a Fan of Lexiles?
Paul and everyone:
So why haven't those achievements been made prior to Rasch?
"I thought they were."
Yes, we haven't yet experienced a Rasch "killer application". Isaac Newton
didn't have a "killer application" until a new comet appeared in the sky.
Before that, most astronomers said something like "Newton isn't telling us
anything about astronomy that we couldn't get from Ptolemy".
But many, many times I've shown clients and students the results of a Rasch
analysis of their own data, and seen them experience an "Aha!" moment, much
as experienced by the poet John Keats (On first looking into Chapman's
"Then felt I like some watcher of the skies
When a new planet swims into his ken;"
Rasch reanalysis of datasets from major test publishers (who have used
classical or IRT methods) nearly always reveals an obvious flaw (such as an
item miskey) that they have overlooked. Of course, their psychometricians
should have caught the problem. But they didn't. In the fog of their
analysis the obvious wasn't noticed. In the case of the MMPI-2, their rigid
thinking deliberately included item miskeys:
http://www.rasch.org/rmt/rmt81f.htm - Years later some of the MMPI-2
development team attended one of my workshops, and confirmed the validity of
that Rasch piece.
And then there are the item and person hierarchies. IRT and Classical
analyses usually ignore the item content. It is not part of their mindset.
The Rasch approach attempts to make sense of the item difficulty order in
order to understand what the test is measuring. Is it measuring what was
intended? Years ago I consulted with the test developers at a medical board.
Rasch analysis revealed that their items had no meaningful hierarchy. Of
course, once I pointed that out, they admitted that was the case. They had
suspected as much, but it was only a "gut feeling". We discovered that their
certification examination was more like a game of "trivial pursuit" than an
investigation into clinician competence. But the examination board was
locked into what they were doing, so they continued their merry way.
Fortunately no lawyers knew about it ....
And then there are adjustments for rater severity ... something essentially
unknown before Rasch ....
Editor, Rasch Measurement Transactions
rmt at rasch.org www.rasch.org/rmt/ Latest RMT: 21:1 Summer 2007
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch