[Rasch] Rasch analysis of interval data

Andrew Kyngdon akyngdon at lexile.com
Wed Jul 23 06:56:32 EST 2008


"It is absolute in the sense that it does not require norms, it does not

need to be tested on people to know what the hierarchical complexity of 
a task is".

It does not logically follow from these premises that you can measure an
ordinal structure with an "absolute" scale.
 

 
-----Original Message-----
From: Michael Lamport Commons [mailto:commons at tiac.net] 
Sent: Tuesday, July 22, 2008 4:47 PM
To: Andrew Kyngdon
Cc: rasch at acer.edu.au
Subject: Re: [Rasch] Rasch analysis of interval data

Andrew Kyngdon wrote:
>
> How can an ordinal structure be measured with an absolute scale?
>

It is absolute in the sense that it does not require norms, it does not 
need to be tested on people to know what the hierarchical complexity of 
a task is. 

MLC
>
>  
>
>  
>
> Andrew Kyngdon, PhD
>
> Senior Research Scientist
>
> MetaMetrics, Inc.
>
> 1000 Park Forty Plaza Drive
>
> Durham NC 27713 USA
>
> Tel. 1 919 354 3473
>
> Fax. 1 919 547 3401
>
>  
>
>
------------------------------------------------------------------------
>
> *From:* Michael Lamport Commons [mailto:commons at tiac.net]
> *Sent:* Tuesday, July 22, 2008 2:04 PM
> *To:* Andrew Kyngdon
> *Cc:* rasch at acer.edu.au
> *Subject:* Re: [Rasch] Rasch analysis of interval data
>
>  
>
> The orders of hierarchical complexity are ordinal, universal, context,

> content, and participant free.  The are an analytic measure of the 
> hierarchical complexity of tasks.  We know of 15 orders.  You can see 
> a description on Wikipedia.  There is a non-arbitrary zero.  Rasch 
> measures performance.  Hierarchical complexity measures tasks 
> properties.  This is psychophysics.  The y-axis is Rasch Score, the x 
> axis is order of hierarhical complexity.  The r's are mostly in the .9

> -.99 range.
>
> MLC
>
> Andrew Kyngdon wrote:
>
> Michael,
>  
> I'm not familiar with your work at all, but I take it by "absolute"
> scale that you mean you have a continuous, quantitative attribute that
> you can measure with a scale possessing a non-arbitrary zero point? If
> so, that is quite a feat in the behavioural sciences, given the lack
of
> natural concatenation operations.
>  
> But you state that you can transform Rasch scores into this supposedly
> "absolute" scale of "Stage scores". Now, correct me if I am wrong, but
> Rasch logits are usually advanced as interval scale measurements
> (leaving aside the obvious problem of a lack of a defined unit,
> something Steve Humphry has been at pains to point out). Interval
scale
> measurements cannot be meaningfully transformed into ratio or absolute
> scales, unless your substantive theory is sufficiently understood to
> enable this, such as in temperature with converting Celsius and
> Fahrenheit measurements into the "absolute" Kelvin scale. But if you
can
> measure something with an absolute scale in the first place, why would
> you bother with an interval scale, unless there are historical reasons
> (as in temperature) for so doing?
>  
>  
>  
> Andrew Kyngdon, PhD
> Senior Research Scientist
> MetaMetrics, Inc.
> 1000 Park Forty Plaza Drive
> Durham NC 27713 USA
> Tel. 1 919 354 3473
> Fax. 1 919 547 3401
>  
>  
>  
>  
> -----Original Message-----
> From: Michael Lamport Commons [mailto:commons at tiac.net] 
> Sent: Tuesday, July 22, 2008 12:38 PM
> To: Andrew Kyngdon
> Cc: Mark Moulton; Paul Barrett; rasch at acer.edu.au
<mailto:rasch at acer.edu.au>
> Subject: Re: [Rasch] Rasch analysis of interval data
>  
> We have an absolute scale in order of hierarchical complexity. We 
> transform the Rasch scores into Stage scores which are based on the 
> absolute scale.
>  
> Michael Lamport Commons
>  
> Andrew Kyngdon wrote:
>   
>> This is still not a guarantee that the lexile unit captures the
"true"
>>     
>  
>   
>> spacing, but fortunately it does not seem to matter.
>>  
>> The Lexile unit is best defined unit I have come across in the 
>> behavioural sciences, so by what criteria one judges to be a "true 
>> spacing" (whatever that is) seems to be a mystery...
>>  
>> Andrew Kyngdon, PhD
>>  
>> Senior Research Scientist
>>  
>> MetaMetrics, Inc.
>>  
>> 1000 Park Forty Plaza Drive
>>  
>> Durham NC 27713 USA
>>  
>> Tel. 1 919 354 3473
>>  
>> Fax. 1 919 547 3401
>>  
>>  
>>     
>
------------------------------------------------------------------------
>   
>> *From:* rasch-bounces at acer.edu.au <mailto:rasch-bounces at acer.edu.au>
[mailto:rasch-bounces at acer.edu.au] 
>> *On Behalf Of *Mark Moulton
>> *Sent:* Monday, July 21, 2008 6:09 PM
>> *To:* Paul Barrett
>> *Cc:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
>> *Subject:* Re: [Rasch] Rasch analysis of interval data
>>  
>> Paul,
>>  
>> Thank you for your explanations and for your presentation 10 years 
>> ago, which are very helpful to me. You raise a fundamental issue, 
>> still controversial, still worth visiting in my opinion. You make the

>> case that Rasch measures are equal-interval representations of counts

>> (ratios of counts), and that is all, and that they do not necessarily

>> capture a fundamental unit of measurement in the underlying
construct.
>>  
>> I think your point is amplified by a simple desktop experiment. Lay a

>> ruler on a sheet of paper. Draw dots (representing persons, say) in 
>> various distributions on the paper. For each centimeter increment on 
>> the ruler (representing items), count the dots above and below that 
>> increment and calculate their log ratio. One finds that the logit 
>> spacings of the ruler increments may be highly unequal (unlike the 
>> centimeters), depending on how one distributes the dots. If one 
>> distributes the dots equally up and down the ruler, the logit lengths

>> between increments appear to get fatter at the extremes. If one
clumps
>>     
>  
>   
>> the dots in multiple modes, the logit lengths can be distorted in all

>> sorts of cool ways. Interestingly, as the distribution approaches 
>> normal, the logit lengths between increments seem to approach a
linear
>>     
>  
>   
>> relationship with the centimeters, (I don't have a proof for why this

>> would be true, but presumably it has something to do with the 
>> relationship between the logistic and normal distributions and may 
>> account for the similarity between independently calibrated scales).
>>  
>> So, I agree that Rasch logits do not capture fundamental units of 
>> measurement, and are sample-dependent in this sense (and in several 
>> other senses, too). My question is: What does this do to Rasch claims

>> of "invariance," aka "special objectivity," the notion that the 
>> relative logit spacings of persons will remain the same regardless of

>> how the items are spaced? Strangely, I don't think it has any effect 
>> at all. The disappearance of the item parameter when calculating the 
>> person parameter, and vice versa, has the same force and implication 
>> that it always did. And due to how Rasch conjointly calculates
persons
>>     
>  
>   
>> and items, whatever distortions may occur affect the persons and
items
>>     
>  
>   
>> equally.
>>  
>> I am left with a relativistic notion of psychometric spaces. Each 
>> Rasch analysis erects a unique space. That space bears no /necessary/

>> relationship to any other Rasch space (except perhaps in some 
>> topographic one-to-one homeomorphic kind of way). However, objects 
>> within that space are distributed in a way that is reproducible,
hence
>>     
>  
>   
>> objective, with respect to other objects in that space. Two Rasch 
>> spaces can be reconciled only by "anchoring" one space to the other 
>> via common persons or items. This forces the two spaces to share the 
>> same "distortions," and thus to become one space, and to preserve 
>> invariance for all objects residing in that space.
>>  
>> Your point about the MetaMetrics lexile scale is well-taken. All
texts
>>     
>  
>   
>> and readers are forced into a common space anchored on the physical 
>> properties represented by word frequency and sentence length (or log 
>> transformations thereof). This was facilitated by the fact that 
>> MetaMetrics discovered and exploited a linear relationship between 
>> textual empirical variables and item difficulties. But even without 
>> that relationship, the two types of variables could have been forced 
>> (by a method MetaMetrics did not use, or need to use) into a common 
>> space through an anchoring procedure. This is still not a guarantee 
>> that the lexile unit captures the "true" spacing, but fortunately it 
>> does not seem to matter.
>>  
>> One space is as good as another, so long as they are internally 
>> consistent. Am I reading this right?
>>  
>> Mark H. Moulton
>>  
>> Educational Data Systems
>>  
>> 2008/7/21 Paul Barrett <pbarrett at hoganassessments.com
<mailto:pbarrett at hoganassessments.com> 
>> <mailto:pbarrett at hoganassessments.com>>:
>>  
>> *From:* rasch-bounces at acer.edu.au <mailto:rasch-bounces at acer.edu.au>
<mailto:rasch-bounces at acer.edu.au> 
>> [mailto:rasch-bounces at acer.edu.au <mailto:rasch-bounces at acer.edu.au>]

>> *On Behalf Of *Anthony James
>> *Sent:* Wednesday, July 16, 2008 7:19 AM
>> *To:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
<mailto:rasch at acer.edu.au>
>> *Subject:* [Rasch] Rasch analysis of interval data
>>  
>> Hi all,
>>  
>> Has anyone ever tried to Rasch analyse a variable for which there's 
>> concatenation-based objective measurement? Suppose we make a height 
>> scale with 6 points:
>>  
>>  
>>     
>
------------------------------------------------------------------------
>   
>> *From:* rasch-bounces at acer.edu.au <mailto:rasch-bounces at acer.edu.au>
<mailto:rasch-bounces at acer.edu.au> 
>> [mailto:rasch-bounces at acer.edu.au <mailto:rasch-bounces at acer.edu.au>]

>> *On Behalf Of *Andrew Kyngdon
>> *Sent:* Wednesday, July 16, 2008 8:13 AM
>>  
>>  
>> *To:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
<mailto:rasch at acer.edu.au>
>>  
>> *Subject:* RE: [Rasch] Rasch analysis of interval data
>>  
>> I think Paul Barrett did something like this once...
>>  
>>  
>>     
>
------------------------------------------------------------------------
>   
>> Yep - 10 years ago to be exact!
>>  
>> Sorry I haven't replied until now ...
>>  
>> The presentation about the simulation can be downloaded at:
>>  
>> http://www.pbarrett.net/presentations/BPS-rasch_98.pdf
>>  
>> From my web-page abstract ...
>>  
>> **Beyond Psychometrics: the recovery of a standard unit of length**: 
>> This 50-slide presentation was given at the British Psychological 
>> Society's Division of Occupational Psychology conference: Assessment 
>> in the Millennium: Beyond Psychometrics, November 1998, at Birkbeck 
>> (University of London). The theme of this presentation was about
Rasch
>>     
>  
>   
>> scaling, and its capacity to construct a standard unit from 
>> observational data. This presentation contained a data simulation
that
>>     
>  
>   
>> attempted to hide a true quantitatively structured latent variable of

>> length behind some poor ordinal observations. All the Rasch scaling 
>> did was to construct an equal-interval latent variable of ordinal 
>> lengths! This simulation was heavily criticised Ben Wright and
others,
>>     
>  
>   
>> and I have included these criticisms as an addendum to the 
>> presentation - along with my reply. However, recent papers seem to 
>> have vindicated my conclusions in some respects.....The reality is 
>> that these methods simply construct linear latent variables in 
>> complete isolation of any empirical evidence that such variables
might
>>     
>  
>   
>> indeed be quantitatively structured.. In my opinion, from a
scientific
>>     
>  
>   
>> perspective, these scaling methods are frankly of little utility, but

>> they are ingenious from a psychometric perspective and do have great 
>> utility in a more pragmatic sense. It all comes down to what the 
>> purpose is for using such scaling, science or number scaling.
>>  
>> 10 years on - with some better understanding of things (!) - the goal

>> and conclusions of the presentation still make sense - but now I
fully
>>     
>  
>   
>> understand why. Rasch scaling cannot "uncover" a linear latent 
>> variable from ordinal measures. It simply scales counts and in
effect,
>>     
>  
>   
>> the numbers applied to its algorithms, without regard to whether
those
>>     
>  
>   
>> counts or numbers are drawn from an ordinal or linear scale.
>>  
>> The mistake made by many psychologists is to forget that latent 
>> variable theory implies nothing about the measurement properties of 
>> the variable of interest - latent variables are simply constructed 
>> ad-hoc to possess linear properties of measurement. That is not how 
>> normal science proceeds, it is as Michell states a "pathology of 
>> science" (2000).
>>  
>> I propose that a key exemplar which shows how to properly model data 
>> while invoking a latent variable, is the work done by Metametrics. It

>> is no accident that the initial exploratory work was empirical and 
>> based upon much cognitive psychological experimentation, PRIOR to the

>> scaling/modeling exercises. Andrew has already provided excellent 
>> explanations of the history of this work, along with another 
>> exposition recently in his peer response to Michell's target article 
>> in the journal Measurement (references below).
>>  
>> However, if we view edumetrics-psychometrics as largely 
>> pragmatic/technical work, which is concerned with the efficiencies to

>> be gained in standards-based testing/examination/cumulative
risk-scale
>>     
>  
>   
>> environments, then IRT models in general, and the Rasch model make a 
>> great deal of sense. I think it is an illusion that the Rasch or any 
>> IRT/latent variable model magically produces "fundamental
measurement"
>>     
>  
>   
>> in any sense of the word. Michell (2004, and now 2008) has put paid
to
>>     
>  
>   
>> this notion.
>>  
>> I don't think this is a controversial point anymore - from the 
>> standpoint of simple logic, the work by Robert Wood, and from my own 
>> small and almost stupid simulation, the Rasch model cannot possibly 
>> "uncover/discover" the true metric for a "statistically constructed 
>> latent variable". It just does what it does given the data with which

>> it is presented. Whether or not that data is an accurate 
>> representation/set of observations of the phenomenon of interest (my 
>> "bad ruler"), the Rasch scaling will simple create a latent variable 
>> anyway - given sufficient stochastic error in the observations (as 
>> with Wood's coin-tosses). Which is why I think the Metametrics 
>> exemplar is so very important, the scaling is constructed around a 
>> wealth of empirical phenomena and magnitude relationships - and not 
>> just banks of "item responses".
>>  
>> Regards ... Paul
>>  
>> __________________________________________________
>> Paul Barrett 918.749-0632 x 326
>> Chief Research Scientist Skype: pbar088
>> Hogan Assessment Systems Inc.
>> 2622 East 21st St., Tulsa, OK 74114
>>  
>> **References**
>>  
>> Kyngdon, A. (2008) Treating the Pathology of Psychometrics: An
Example
>>     
>  
>   
>> from the Comprehension of Continuous Prose Text. //Measurement: 
>> Interdisciplinary Research & Perspective//, 6, 1 & 2, 108-113.
>>  
>> Michell. J. (2000) Normal science, pathological Science, and 
>> psychometrics. Theory and Psychology, 10, 5, 639-667.
>>  
>> Michell, J. (2004) Item Response Models, pathological science, and
the
>>     
>  
>   
>> shape of error. //Theory and Psychology//, 14, 1, 121-129.
>>  
>> Michell, J. (2008) Is psychometrics pathological science? 
>> //Measurement: Interdisciplinary Research & Perspective//, 6, 1, 7-24
>>  
>> Wood, R. (1978) Fitting the rasch model - a heady tale. //British 
>> Journal of Mathematical and Statistical Psychology//, 31, , 27-32.
>>  
>> **An aside**
>>  
>> The journal "Measurement: Interdisciplinary Research and 
>> Perspective"published issues two issues simultaneously - three target

>> articles and commenatries on the issue:
>>  
>> //The Conceptual Foundations of Psychological Measurement//
>>  
>> The target papers by Denny Borsboom and Keith Markus are also 
>> excellent expositions of their respective positions. Very nice 
>> position pieces.
>>  
>> I've attached the journal link here so you can look at the paper 
>> titles etc.
>>  
>> http://www.informaworld.com/smpp/title~content=g794512699~db=all
<http://www.informaworld.com/smpp/title%7Econtent=g794512699%7Edb=all> 
>>
<http://www.informaworld.com/smpp/title%7Econtent=g794512699%7Edb=all>
>>  
>>  
>> _______________________________________________
>> Rasch mailing list
>> Rasch at acer.edu.au <mailto:Rasch at acer.edu.au>
<mailto:Rasch at acer.edu.au>
>> http://mailinglist.acer.edu.au/mailman/listinfo/rasch
>>  
>>  
>>     
>
------------------------------------------------------------------------
>   
>> _______________________________________________
>> Rasch mailing list
>> Rasch at acer.edu.au <mailto:Rasch at acer.edu.au>
>> http://mailinglist.acer.edu.au/mailman/listinfo/rasch
>>   
>>     
>  
>  
>  
>   
>
>  
>




More information about the Rasch mailing list