[Rasch] Rasch analysis of interval data

Mark Moulton markhmoulton at gmail.com
Tue Jul 22 08:08:49 EST 2008


Paul, Thank you for your explanations and for your presentation 10 years
ago, which are very helpful to me.  You raise a fundamental issue, still
controversial, still worth visiting in my opinion. You make the case that
Rasch measures are equal-interval representations of counts (ratios of
counts), and that is all, and that they do not necessarily capture a
fundamental unit of measurement in the underlying construct.

I think your point is amplified by a simple desktop experiment.  Lay a ruler
on a sheet of paper.  Draw dots (representing persons, say) in various
distributions on the paper.  For each centimeter increment on the ruler
(representing items), count the dots above and below that increment and
calculate their log ratio.  One finds that the logit spacings of the ruler
increments may be highly unequal (unlike the centimeters), depending on how
one distributes the dots.  If one distributes the dots equally up and down
the ruler, the logit lengths between increments appear to get fatter at the
extremes.  If one clumps the dots in multiple modes, the logit lengths can
be distorted in all sorts of cool ways.  Interestingly, as the distribution
approaches normal, the logit lengths between increments seem to approach a
linear relationship with the centimeters, (I don't have a proof for why this
would be true, but presumably it has something to do with the relationship
between the logistic and normal distributions and may account for the
similarity between independently calibrated scales).

So, I agree that Rasch logits do not capture fundamental units of
measurement, and are sample-dependent in this sense (and in several other
senses, too).  My question is:  What does this do to Rasch claims of
"invariance," aka "special objectivity," the notion that the relative logit
spacings of persons will remain the same regardless of how the items are
spaced?  Strangely, I don't think it has any effect at all.  The
disappearance of the item parameter when calculating the person parameter,
and vice versa, has the same force and implication that it always did.  And
due to how Rasch conjointly calculates persons and items, whatever
distortions may occur affect the persons and items equally.

I am left with a relativistic notion of psychometric spaces.  Each Rasch
analysis erects a unique space.  That space bears no necessary relationship
to any other Rasch space (except perhaps in some topographic one-to-one
homeomorphic kind of way).  However, objects within that space are
distributed in a way that is reproducible, hence objective, with respect to
other objects in that space.  Two Rasch spaces can be reconciled only by
"anchoring" one space to the other via common persons or items.  This forces
the two spaces to share the same "distortions," and thus to become one
space, and to preserve invariance for all objects residing in that space.

Your point about the MetaMetrics lexile scale is well-taken.  All texts and
readers are forced into a common space anchored on the physical properties
represented by word frequency and sentence length (or log transformations
thereof).  This was facilitated by the fact that MetaMetrics discovered and
exploited a linear relationship between textual empirical variables and item
difficulties.  But even without that relationship, the two types of
variables could have been forced (by a method MetaMetrics did not use, or
need to use) into a common space through an anchoring procedure.  This is
still not a guarantee that the lexile unit captures the "true" spacing, but
fortunately it does not seem to matter.

One space is as good as another, so long as they are internally consistent.
 Am I reading this right?

Mark H. Moulton
Educational Data Systems







2008/7/21 Paul Barrett <pbarrett at hoganassessments.com>:

>   *From:* rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] *On
> Behalf Of *Anthony James
> *Sent:* Wednesday, July 16, 2008 7:19 AM
> *To:* rasch at acer.edu.au
> *Subject:* [Rasch] Rasch analysis of interval data
>
>
>
> Hi all,
>
> Has anyone ever tried to Rasch analyse a variable for which there's
> concatenation-based objective measurement? Suppose we make a height scale
> with 6 points:
>
>  ------------------------------
>  *From:* rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] *On
> Behalf Of *Andrew Kyngdon
> *Sent:* Wednesday, July 16, 2008 8:13 AM
> *To:* rasch at acer.edu.au
> *Subject:* RE: [Rasch] Rasch analysis of interval data
> I think Paul Barrett did something like this once…
>
>  ------------------------------
>
> Yep - 10 years ago to be exact!
>
> Sorry I haven't replied until now ...
>
> The presentation about the simulation can be downloaded at:
>
> http://www.pbarrett.net/presentations/BPS-rasch_98.pdf
>
> From my web-page abstract ...
> *Beyond Psychometrics: the recovery of a standard unit of length*: This
> 50-slide presentation was given at the British Psychological Society's
> Division of Occupational Psychology conference: Assessment in the
> Millennium: Beyond Psychometrics, November 1998,  at Birkbeck (University of
> London). The theme of this presentation was about Rasch scaling, and its
> capacity to construct a standard unit from observational data. This
> presentation contained a data simulation that attempted to hide a true
> quantitatively structured latent variable of length behind some poor ordinal
> observations. All the Rasch scaling did was to construct an equal-interval
> latent variable of ordinal lengths! This simulation was heavily criticised
> Ben Wright and others, and I have included these criticisms as an addendum
> to the presentation - along with my reply. However, recent papers seem to
> have vindicated my conclusions in some respects.....The reality is that
> these methods simply construct linear latent variables in complete isolation
> of any empirical evidence that such variables might indeed be quantitatively
> structured.. In my opinion, from a scientific perspective, these scaling
> methods are frankly of little utility, but they are ingenious from a
> psychometric perspective and do have great utility in a more pragmatic
> sense. It all comes down to what the purpose is for using such scaling,
> science or number scaling.
>
> 10 years on - with some better understanding of things (!) - the goal and
> conclusions of the presentation still make sense - but now I fully
> understand why. Rasch scaling cannot "uncover" a linear latent variable from
> ordinal measures. It simply scales counts and in effect, the numbers applied
> to its algorithms, without regard to whether those counts or numbers are
> drawn from an ordinal or linear scale.
>
> The mistake made by many psychologists is to forget that latent variable
> theory implies nothing about the measurement properties of the variable of
> interest - latent variables are simply constructed ad-hoc to possess linear
> properties of measurement. That is not how normal science proceeds, it is as
> Michell states a "pathology of science" (2000).
>
> I propose that a key exemplar which shows how to properly model data
> while invoking a latent variable, is the work done by Metametrics. It is no
> accident that the initial exploratory work was empirical and based upon much
> cognitive psychological experimentation, PRIOR to the scaling/modeling
> exercises. Andrew has already provided excellent explanations of the history
> of this work, along with another exposition recently in his peer response to
> Michell's target article in the journal Measurement (references below).
>
> However, if we view edumetrics-psychometrics as largely pragmatic/technical
> work, which is concerned with the efficiencies to be gained in
> standards-based testing/examination/cumulative risk-scale environments, then
> IRT models in general, and the Rasch model make a great deal of sense. I
> think it is an illusion that the Rasch or any IRT/latent variable
> model magically produces  "fundamental measurement" in any sense of the
> word. Michell (2004, and now 2008) has put paid to this notion.
>
> I don't think this is a controversial point anymore - from the standpoint
> of simple logic, the work by Robert Wood, and from my own small and almost
> stupid simulation, the Rasch model cannot possibly "uncover/discover" the
> true metric for a "statistically constructed latent variable". It just does
> what it does given the data with which it is presented. Whether or not that
> data is an accurate representation/set of observations of the phenomenon of
> interest (my "bad ruler"), the Rasch scaling will simple create a latent
> variable anyway - given sufficient stochastic error in the observations (as
> with Wood's coin-tosses). Which is why I think the Metametrics exemplar is
> so very important, the scaling is constructed around a wealth of empirical
> phenomena and magnitude relationships - and not just banks of "item
> responses".
>
> Regards ... Paul
> __________________________________________________
> Paul Barrett                                                918.749-0632 x
> 326
> Chief Research Scientist                                   Skype: pbar088
> Hogan Assessment Systems Inc.
> 2622 East 21st St., Tulsa, OK 74114
>
>
> *References*
>
> Kyngdon, A. (2008) Treating the Pathology of Psychometrics: An Example from
> the Comprehension of Continuous Prose Text. *Measurement:
> Interdisciplinary Research & Perspective*, 6, 1 & 2, 108-113.
>
> Michell. J. (2000) Normal science, pathological Science, and psychometrics.
> Theory and Psychology, 10, 5, 639-667.
>
> Michell, J. (2004) Item Response Models, pathological science, and the
> shape of error. *Theory and Psychology*, 14, 1, 121-129.
>
> Michell, J. (2008) Is psychometrics pathological science? *Measurement:
> Interdisciplinary Research & Perspective*, 6, 1, 7-24
>
> Wood, R. (1978) Fitting the rasch model - a heady tale. *British Journal
> of Mathematical and Statistical Psychology*, 31, , 27-32.
>
> *An aside*  The journal "Measurement: Interdisciplinary Research and
> Perspective"published issues two issues simultaneously - three target
> articles and commenatries on the issue:
> *The Conceptual Foundations of Psychological Measurement*
>
> The target papers by Denny Borsboom and Keith Markus are also excellent
> expositions of their respective positions. Very nice position pieces.
>
> I've attached the journal link here so you can look at the paper titles
> etc. http://www.informaworld.com/smpp/title~content=g794512699~db=all
>
> _______________________________________________
> Rasch mailing list
> Rasch at acer.edu.au
> http://mailinglist.acer.edu.au/mailman/listinfo/rasch
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20080721/cc62a4b0/attachment.html 


More information about the Rasch mailing list