[Rasch] Rasch analysis of interval data

Andrew Kyngdon akyngdon at lexile.com
Wed Jul 23 04:13:40 EST 2008


How can an ordinal structure be measured with an absolute scale?

 

 

Andrew Kyngdon, PhD

Senior Research Scientist

MetaMetrics, Inc.

1000 Park Forty Plaza Drive

Durham NC 27713 USA

Tel. 1 919 354 3473

Fax. 1 919 547 3401

<http://www.lexile.com/conference2008>  

________________________________

From: Michael Lamport Commons [mailto:commons at tiac.net] 
Sent: Tuesday, July 22, 2008 2:04 PM
To: Andrew Kyngdon
Cc: rasch at acer.edu.au
Subject: Re: [Rasch] Rasch analysis of interval data

 

The orders of hierarchical complexity are ordinal, universal, context,
content, and participant free.  The are an analytic measure of the
hierarchical complexity of tasks.  We know of 15 orders.  You can see a
description on Wikipedia.  There is a non-arbitrary zero.  Rasch
measures performance.  Hierarchical complexity measures tasks
properties.  This is psychophysics.  The y-axis is Rasch Score, the x
axis is order of hierarhical complexity.  The r's are mostly in the .9
-.99 range.

MLC

Andrew Kyngdon wrote: 

Michael,
 
I'm not familiar with your work at all, but I take it by "absolute"
scale that you mean you have a continuous, quantitative attribute that
you can measure with a scale possessing a non-arbitrary zero point? If
so, that is quite a feat in the behavioural sciences, given the lack of
natural concatenation operations.
 
But you state that you can transform Rasch scores into this supposedly
"absolute" scale of "Stage scores". Now, correct me if I am wrong, but
Rasch logits are usually advanced as interval scale measurements
(leaving aside the obvious problem of a lack of a defined unit,
something Steve Humphry has been at pains to point out). Interval scale
measurements cannot be meaningfully transformed into ratio or absolute
scales, unless your substantive theory is sufficiently understood to
enable this, such as in temperature with converting Celsius and
Fahrenheit measurements into the "absolute" Kelvin scale. But if you can
measure something with an absolute scale in the first place, why would
you bother with an interval scale, unless there are historical reasons
(as in temperature) for so doing?
 
 
 
Andrew Kyngdon, PhD
Senior Research Scientist
MetaMetrics, Inc.
1000 Park Forty Plaza Drive
Durham NC 27713 USA
Tel. 1 919 354 3473
Fax. 1 919 547 3401
 
 
 
 
-----Original Message-----
From: Michael Lamport Commons [mailto:commons at tiac.net] 
Sent: Tuesday, July 22, 2008 12:38 PM
To: Andrew Kyngdon
Cc: Mark Moulton; Paul Barrett; rasch at acer.edu.au
Subject: Re: [Rasch] Rasch analysis of interval data
 
We have an absolute scale in order of hierarchical complexity. We 
transform the Rasch scores into Stage scores which are based on the 
absolute scale.
 
Michael Lamport Commons
 
Andrew Kyngdon wrote:
  

	This is still not a guarantee that the lexile unit captures the
"true"
	    

 
  

	spacing, but fortunately it does not seem to matter.
	 
	The Lexile unit is best defined unit I have come across in the 
	behavioural sciences, so by what criteria one judges to be a
"true 
	spacing" (whatever that is) seems to be a mystery...
	 
	Andrew Kyngdon, PhD
	 
	Senior Research Scientist
	 
	MetaMetrics, Inc.
	 
	1000 Park Forty Plaza Drive
	 
	Durham NC 27713 USA
	 
	Tel. 1 919 354 3473
	 
	Fax. 1 919 547 3401
	 
	 
	    

------------------------------------------------------------------------
  

	*From:* rasch-bounces at acer.edu.au
[mailto:rasch-bounces at acer.edu.au] 
	*On Behalf Of *Mark Moulton
	*Sent:* Monday, July 21, 2008 6:09 PM
	*To:* Paul Barrett
	*Cc:* rasch at acer.edu.au
	*Subject:* Re: [Rasch] Rasch analysis of interval data
	 
	Paul,
	 
	Thank you for your explanations and for your presentation 10
years 
	ago, which are very helpful to me. You raise a fundamental
issue, 
	still controversial, still worth visiting in my opinion. You
make the 
	case that Rasch measures are equal-interval representations of
counts 
	(ratios of counts), and that is all, and that they do not
necessarily 
	capture a fundamental unit of measurement in the underlying
construct.
	 
	I think your point is amplified by a simple desktop experiment.
Lay a 
	ruler on a sheet of paper. Draw dots (representing persons, say)
in 
	various distributions on the paper. For each centimeter
increment on 
	the ruler (representing items), count the dots above and below
that 
	increment and calculate their log ratio. One finds that the
logit 
	spacings of the ruler increments may be highly unequal (unlike
the 
	centimeters), depending on how one distributes the dots. If one 
	distributes the dots equally up and down the ruler, the logit
lengths 
	between increments appear to get fatter at the extremes. If one
clumps
	    

 
  

	the dots in multiple modes, the logit lengths can be distorted
in all 
	sorts of cool ways. Interestingly, as the distribution
approaches 
	normal, the logit lengths between increments seem to approach a
linear
	    

 
  

	relationship with the centimeters, (I don't have a proof for why
this 
	would be true, but presumably it has something to do with the 
	relationship between the logistic and normal distributions and
may 
	account for the similarity between independently calibrated
scales).
	 
	So, I agree that Rasch logits do not capture fundamental units
of 
	measurement, and are sample-dependent in this sense (and in
several 
	other senses, too). My question is: What does this do to Rasch
claims 
	of "invariance," aka "special objectivity," the notion that the 
	relative logit spacings of persons will remain the same
regardless of 
	how the items are spaced? Strangely, I don't think it has any
effect 
	at all. The disappearance of the item parameter when calculating
the 
	person parameter, and vice versa, has the same force and
implication 
	that it always did. And due to how Rasch conjointly calculates
persons
	    

 
  

	and items, whatever distortions may occur affect the persons and
items
	    

 
  

	equally.
	 
	I am left with a relativistic notion of psychometric spaces.
Each 
	Rasch analysis erects a unique space. That space bears no
/necessary/ 
	relationship to any other Rasch space (except perhaps in some 
	topographic one-to-one homeomorphic kind of way). However,
objects 
	within that space are distributed in a way that is reproducible,
hence
	    

 
  

	objective, with respect to other objects in that space. Two
Rasch 
	spaces can be reconciled only by "anchoring" one space to the
other 
	via common persons or items. This forces the two spaces to share
the 
	same "distortions," and thus to become one space, and to
preserve 
	invariance for all objects residing in that space.
	 
	Your point about the MetaMetrics lexile scale is well-taken. All
texts
	    

 
  

	and readers are forced into a common space anchored on the
physical 
	properties represented by word frequency and sentence length (or
log 
	transformations thereof). This was facilitated by the fact that 
	MetaMetrics discovered and exploited a linear relationship
between 
	textual empirical variables and item difficulties. But even
without 
	that relationship, the two types of variables could have been
forced 
	(by a method MetaMetrics did not use, or need to use) into a
common 
	space through an anchoring procedure. This is still not a
guarantee 
	that the lexile unit captures the "true" spacing, but
fortunately it 
	does not seem to matter.
	 
	One space is as good as another, so long as they are internally 
	consistent. Am I reading this right?
	 
	Mark H. Moulton
	 
	Educational Data Systems
	 
	2008/7/21 Paul Barrett <pbarrett at hoganassessments.com 
	<mailto:pbarrett at hoganassessments.com>
<mailto:pbarrett at hoganassessments.com> >:
	 
	*From:* rasch-bounces at acer.edu.au
<mailto:rasch-bounces at acer.edu.au> <mailto:rasch-bounces at acer.edu.au>  
	[mailto:rasch-bounces at acer.edu.au
<mailto:rasch-bounces at acer.edu.au> <mailto:rasch-bounces at acer.edu.au> ] 
	*On Behalf Of *Anthony James
	*Sent:* Wednesday, July 16, 2008 7:19 AM
	*To:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
<mailto:rasch at acer.edu.au> 
	*Subject:* [Rasch] Rasch analysis of interval data
	 
	Hi all,
	 
	Has anyone ever tried to Rasch analyse a variable for which
there's 
	concatenation-based objective measurement? Suppose we make a
height 
	scale with 6 points:
	 
	 
	    

------------------------------------------------------------------------
  

	*From:* rasch-bounces at acer.edu.au
<mailto:rasch-bounces at acer.edu.au> <mailto:rasch-bounces at acer.edu.au>  
	[mailto:rasch-bounces at acer.edu.au
<mailto:rasch-bounces at acer.edu.au> <mailto:rasch-bounces at acer.edu.au> ] 
	*On Behalf Of *Andrew Kyngdon
	*Sent:* Wednesday, July 16, 2008 8:13 AM
	 
	 
	*To:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
<mailto:rasch at acer.edu.au> 
	 
	*Subject:* RE: [Rasch] Rasch analysis of interval data
	 
	I think Paul Barrett did something like this once...
	 
	 
	    

------------------------------------------------------------------------
  

	Yep - 10 years ago to be exact!
	 
	Sorry I haven't replied until now ...
	 
	The presentation about the simulation can be downloaded at:
	 
	http://www.pbarrett.net/presentations/BPS-rasch_98.pdf
	 
	From my web-page abstract ...
	 
	**Beyond Psychometrics: the recovery of a standard unit of
length**: 
	This 50-slide presentation was given at the British
Psychological 
	Society's Division of Occupational Psychology conference:
Assessment 
	in the Millennium: Beyond Psychometrics, November 1998, at
Birkbeck 
	(University of London). The theme of this presentation was about
Rasch
	    

 
  

	scaling, and its capacity to construct a standard unit from 
	observational data. This presentation contained a data
simulation that
	    

 
  

	attempted to hide a true quantitatively structured latent
variable of 
	length behind some poor ordinal observations. All the Rasch
scaling 
	did was to construct an equal-interval latent variable of
ordinal 
	lengths! This simulation was heavily criticised Ben Wright and
others,
	    

 
  

	and I have included these criticisms as an addendum to the 
	presentation - along with my reply. However, recent papers seem
to 
	have vindicated my conclusions in some respects.....The reality
is 
	that these methods simply construct linear latent variables in 
	complete isolation of any empirical evidence that such variables
might
	    

 
  

	indeed be quantitatively structured.. In my opinion, from a
scientific
	    

 
  

	perspective, these scaling methods are frankly of little
utility, but 
	they are ingenious from a psychometric perspective and do have
great 
	utility in a more pragmatic sense. It all comes down to what the

	purpose is for using such scaling, science or number scaling.
	 
	10 years on - with some better understanding of things (!) - the
goal 
	and conclusions of the presentation still make sense - but now I
fully
	    

 
  

	understand why. Rasch scaling cannot "uncover" a linear latent 
	variable from ordinal measures. It simply scales counts and in
effect,
	    

 
  

	the numbers applied to its algorithms, without regard to whether
those
	    

 
  

	counts or numbers are drawn from an ordinal or linear scale.
	 
	The mistake made by many psychologists is to forget that latent 
	variable theory implies nothing about the measurement properties
of 
	the variable of interest - latent variables are simply
constructed 
	ad-hoc to possess linear properties of measurement. That is not
how 
	normal science proceeds, it is as Michell states a "pathology of

	science" (2000).
	 
	I propose that a key exemplar which shows how to properly model
data 
	while invoking a latent variable, is the work done by
Metametrics. It 
	is no accident that the initial exploratory work was empirical
and 
	based upon much cognitive psychological experimentation, PRIOR
to the 
	scaling/modeling exercises. Andrew has already provided
excellent 
	explanations of the history of this work, along with another 
	exposition recently in his peer response to Michell's target
article 
	in the journal Measurement (references below).
	 
	However, if we view edumetrics-psychometrics as largely 
	pragmatic/technical work, which is concerned with the
efficiencies to 
	be gained in standards-based testing/examination/cumulative
risk-scale
	    

 
  

	environments, then IRT models in general, and the Rasch model
make a 
	great deal of sense. I think it is an illusion that the Rasch or
any 
	IRT/latent variable model magically produces "fundamental
measurement"
	    

 
  

	in any sense of the word. Michell (2004, and now 2008) has put
paid to
	    

 
  

	this notion.
	 
	I don't think this is a controversial point anymore - from the 
	standpoint of simple logic, the work by Robert Wood, and from my
own 
	small and almost stupid simulation, the Rasch model cannot
possibly 
	"uncover/discover" the true metric for a "statistically
constructed 
	latent variable". It just does what it does given the data with
which 
	it is presented. Whether or not that data is an accurate 
	representation/set of observations of the phenomenon of interest
(my 
	"bad ruler"), the Rasch scaling will simple create a latent
variable 
	anyway - given sufficient stochastic error in the observations
(as 
	with Wood's coin-tosses). Which is why I think the Metametrics 
	exemplar is so very important, the scaling is constructed around
a 
	wealth of empirical phenomena and magnitude relationships - and
not 
	just banks of "item responses".
	 
	Regards ... Paul
	 
	__________________________________________________
	Paul Barrett 918.749-0632 x 326
	Chief Research Scientist Skype: pbar088
	Hogan Assessment Systems Inc.
	2622 East 21st St., Tulsa, OK 74114
	 
	**References**
	 
	Kyngdon, A. (2008) Treating the Pathology of Psychometrics: An
Example
	    

 
  

	from the Comprehension of Continuous Prose Text. //Measurement: 
	Interdisciplinary Research & Perspective//, 6, 1 & 2, 108-113.
	 
	Michell. J. (2000) Normal science, pathological Science, and 
	psychometrics. Theory and Psychology, 10, 5, 639-667.
	 
	Michell, J. (2004) Item Response Models, pathological science,
and the
	    

 
  

	shape of error. //Theory and Psychology//, 14, 1, 121-129.
	 
	Michell, J. (2008) Is psychometrics pathological science? 
	//Measurement: Interdisciplinary Research & Perspective//, 6, 1,
7-24
	 
	Wood, R. (1978) Fitting the rasch model - a heady tale.
//British 
	Journal of Mathematical and Statistical Psychology//, 31, ,
27-32.
	 
	**An aside**
	 
	The journal "Measurement: Interdisciplinary Research and 
	Perspective"published issues two issues simultaneously - three
target 
	articles and commenatries on the issue:
	 
	//The Conceptual Foundations of Psychological Measurement//
	 
	The target papers by Denny Borsboom and Keith Markus are also 
	excellent expositions of their respective positions. Very nice 
	position pieces.
	 
	I've attached the journal link here so you can look at the paper

	titles etc.
	 
	http://www.informaworld.com/smpp/title~content=g794512699~db=all

	
<http://www.informaworld.com/smpp/title%7Econtent=g794512699%7Edb=all>
<http://www.informaworld.com/smpp/title%7Econtent=g794512699%7Edb=all> 
	 
	 
	_______________________________________________
	Rasch mailing list
	Rasch at acer.edu.au <mailto:Rasch at acer.edu.au>
<mailto:Rasch at acer.edu.au> 
	http://mailinglist.acer.edu.au/mailman/listinfo/rasch
	 
	 
	    

------------------------------------------------------------------------
  

	_______________________________________________
	Rasch mailing list
	Rasch at acer.edu.au
	http://mailinglist.acer.edu.au/mailman/listinfo/rasch
	  
	    

 
 
 
  

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20080722/d5ccdc08/attachment.html 


More information about the Rasch mailing list