[Rasch] Rasch analysis of interval data
pbarrett at hoganassessments.com
Tue Jul 22 04:15:13 EST 2008
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf Of Anthony James
Sent: Wednesday, July 16, 2008 7:19 AM
To: rasch at acer.edu.au
Subject: [Rasch] Rasch analysis of interval data
Has anyone ever tried to Rasch analyse a variable for which there’s concatenation-based objective measurement? Suppose we make a height scale with 6 points:
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf Of Andrew Kyngdon
Sent: Wednesday, July 16, 2008 8:13 AM
To: rasch at acer.edu.au
Subject: RE: [Rasch] Rasch analysis of interval data
I think Paul Barrett did something like this once...
Yep - 10 years ago to be exact!
Sorry I haven't replied until now ...
The presentation about the simulation can be downloaded at:
>From my web-page abstract ...
Beyond Psychometrics: the recovery of a standard unit of length: This 50-slide presentation was given at the British Psychological Society's Division of Occupational Psychology conference: Assessment in the Millennium: Beyond Psychometrics, November 1998, at Birkbeck (University of London). The theme of this presentation was about Rasch scaling, and its capacity to construct a standard unit from observational data. This presentation contained a data simulation that attempted to hide a true quantitatively structured latent variable of length behind some poor ordinal observations. All the Rasch scaling did was to construct an equal-interval latent variable of ordinal lengths! This simulation was heavily criticised Ben Wright and others, and I have included these criticisms as an addendum to the presentation - along with my reply. However, recent papers seem to have vindicated my conclusions in some respects.....The reality is that these methods simply construct linear latent variables in complete isolation of any empirical evidence that such variables might indeed be quantitatively structured.. In my opinion, from a scientific perspective, these scaling methods are frankly of little utility, but they are ingenious from a psychometric perspective and do have great utility in a more pragmatic sense. It all comes down to what the purpose is for using such scaling, science or number scaling.
10 years on - with some better understanding of things (!) - the goal and conclusions of the presentation still make sense - but now I fully understand why. Rasch scaling cannot "uncover" a linear latent variable from ordinal measures. It simply scales counts and in effect, the numbers applied to its algorithms, without regard to whether those counts or numbers are drawn from an ordinal or linear scale.
The mistake made by many psychologists is to forget that latent variable theory implies nothing about the measurement properties of the variable of interest - latent variables are simply constructed ad-hoc to possess linear properties of measurement. That is not how normal science proceeds, it is as Michell states a "pathology of science" (2000).
I propose that a key exemplar which shows how to properly model data while invoking a latent variable, is the work done by Metametrics. It is no accident that the initial exploratory work was empirical and based upon much cognitive psychological experimentation, PRIOR to the scaling/modeling exercises. Andrew has already provided excellent explanations of the history of this work, along with another exposition recently in his peer response to Michell's target article in the journal Measurement (references below).
However, if we view edumetrics-psychometrics as largely pragmatic/technical work, which is concerned with the efficiencies to be gained in standards-based testing/examination/cumulative risk-scale environments, then IRT models in general, and the Rasch model make a great deal of sense. I think it is an illusion that the Rasch or any IRT/latent variable model magically produces "fundamental measurement" in any sense of the word. Michell (2004, and now 2008) has put paid to this notion.
I don't think this is a controversial point anymore - from the standpoint of simple logic, the work by Robert Wood, and from my own small and almost stupid simulation, the Rasch model cannot possibly "uncover/discover" the true metric for a "statistically constructed latent variable". It just does what it does given the data with which it is presented. Whether or not that data is an accurate representation/set of observations of the phenomenon of interest (my "bad ruler"), the Rasch scaling will simple create a latent variable anyway - given sufficient stochastic error in the observations (as with Wood's coin-tosses). Which is why I think the Metametrics exemplar is so very important, the scaling is constructed around a wealth of empirical phenomena and magnitude relationships - and not just banks of "item responses".
Regards ... Paul
Paul Barrett 918.749-0632 x 326
Chief Research Scientist Skype: pbar088
Hogan Assessment Systems Inc.
2622 East 21st St., Tulsa, OK 74114
Kyngdon, A. (2008) Treating the Pathology of Psychometrics: An Example from the Comprehension of Continuous Prose Text. Measurement: Interdisciplinary Research & Perspective, 6, 1 & 2, 108-113.
Michell. J. (2000) Normal science, pathological Science, and psychometrics. Theory and Psychology, 10, 5, 639-667.
Michell, J. (2004) Item Response Models, pathological science, and the shape of error. Theory and Psychology, 14, 1, 121-129.
Michell, J. (2008) Is psychometrics pathological science? Measurement: Interdisciplinary Research & Perspective, 6, 1, 7-24
Wood, R. (1978) Fitting the rasch model - a heady tale. British Journal of Mathematical and Statistical Psychology, 31, , 27-32.
The journal "Measurement: Interdisciplinary Research and Perspective"published issues two issues simultaneously - three target articles and commenatries on the issue:
The Conceptual Foundations of Psychological Measurement
The target papers by Denny Borsboom and Keith Markus are also excellent expositions of their respective positions. Very nice position pieces.
I've attached the journal link here so you can look at the paper titles etc.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch