[Rasch] Rasch analysis of interval data

Michael Lamport Commons commons at tiac.net
Wed Jul 23 07:43:45 EST 2008


  absolute scale


    A Dictionary of Psychology | Date: 2001

*absolute scale n.* In statistics and measurement theory, a ratio scale 
<http://www.encyclopedia.com/doc/1O87-ratioscale.html> in which the unit 
of measurement is fixed. In practice, values on an absolute scale are 
usually if not always obtained by counting.

MLC:  What is counted is the number of times in a hierarchy that a 
higher order action coordinates two or more lower order actions in a 
non-arbitrary way.  It has a tree structure so one counts how high up in 
the tree one is.

MLC

Andrew Kyngdon wrote:
> "It is absolute in the sense that it does not require norms, it does not
>
> need to be tested on people to know what the hierarchical complexity of 
> a task is".
>
> It does not logically follow from these premises that you can measure an
> ordinal structure with an "absolute" scale.
>  
>
>  
> -----Original Message-----
> From: Michael Lamport Commons [mailto:commons at tiac.net] 
> Sent: Tuesday, July 22, 2008 4:47 PM
> To: Andrew Kyngdon
> Cc: rasch at acer.edu.au
> Subject: Re: [Rasch] Rasch analysis of interval data
>
> Andrew Kyngdon wrote:
>   
>> How can an ordinal structure be measured with an absolute scale?
>>
>>     
>
> It is absolute in the sense that it does not require norms, it does not 
> need to be tested on people to know what the hierarchical complexity of 
> a task is. 
>
> MLC
>   
>>  
>>
>>  
>>
>> Andrew Kyngdon, PhD
>>
>> Senior Research Scientist
>>
>> MetaMetrics, Inc.
>>
>> 1000 Park Forty Plaza Drive
>>
>> Durham NC 27713 USA
>>
>> Tel. 1 919 354 3473
>>
>> Fax. 1 919 547 3401
>>
>>  
>>
>>
>>     
> ------------------------------------------------------------------------
>   
>> *From:* Michael Lamport Commons [mailto:commons at tiac.net]
>> *Sent:* Tuesday, July 22, 2008 2:04 PM
>> *To:* Andrew Kyngdon
>> *Cc:* rasch at acer.edu.au
>> *Subject:* Re: [Rasch] Rasch analysis of interval data
>>
>>  
>>
>> The orders of hierarchical complexity are ordinal, universal, context,
>>     
>
>   
>> content, and participant free.  The are an analytic measure of the 
>> hierarchical complexity of tasks.  We know of 15 orders.  You can see 
>> a description on Wikipedia.  There is a non-arbitrary zero.  Rasch 
>> measures performance.  Hierarchical complexity measures tasks 
>> properties.  This is psychophysics.  The y-axis is Rasch Score, the x 
>> axis is order of hierarhical complexity.  The r's are mostly in the .9
>>     
>
>   
>> -.99 range.
>>
>> MLC
>>
>> Andrew Kyngdon wrote:
>>
>> Michael,
>>  
>> I'm not familiar with your work at all, but I take it by "absolute"
>> scale that you mean you have a continuous, quantitative attribute that
>> you can measure with a scale possessing a non-arbitrary zero point? If
>> so, that is quite a feat in the behavioural sciences, given the lack
>>     
> of
>   
>> natural concatenation operations.
>>  
>> But you state that you can transform Rasch scores into this supposedly
>> "absolute" scale of "Stage scores". Now, correct me if I am wrong, but
>> Rasch logits are usually advanced as interval scale measurements
>> (leaving aside the obvious problem of a lack of a defined unit,
>> something Steve Humphry has been at pains to point out). Interval
>>     
> scale
>   
>> measurements cannot be meaningfully transformed into ratio or absolute
>> scales, unless your substantive theory is sufficiently understood to
>> enable this, such as in temperature with converting Celsius and
>> Fahrenheit measurements into the "absolute" Kelvin scale. But if you
>>     
> can
>   
>> measure something with an absolute scale in the first place, why would
>> you bother with an interval scale, unless there are historical reasons
>> (as in temperature) for so doing?
>>  
>>  
>>  
>> Andrew Kyngdon, PhD
>> Senior Research Scientist
>> MetaMetrics, Inc.
>> 1000 Park Forty Plaza Drive
>> Durham NC 27713 USA
>> Tel. 1 919 354 3473
>> Fax. 1 919 547 3401
>>  
>>  
>>  
>>  
>> -----Original Message-----
>> From: Michael Lamport Commons [mailto:commons at tiac.net] 
>> Sent: Tuesday, July 22, 2008 12:38 PM
>> To: Andrew Kyngdon
>> Cc: Mark Moulton; Paul Barrett; rasch at acer.edu.au
>>     
> <mailto:rasch at acer.edu.au>
>   
>> Subject: Re: [Rasch] Rasch analysis of interval data
>>  
>> We have an absolute scale in order of hierarchical complexity. We 
>> transform the Rasch scores into Stage scores which are based on the 
>> absolute scale.
>>  
>> Michael Lamport Commons
>>  
>> Andrew Kyngdon wrote:
>>   
>>     
>>> This is still not a guarantee that the lexile unit captures the
>>>       
> "true"
>   
>>>     
>>>       
>>  
>>   
>>     
>>> spacing, but fortunately it does not seem to matter.
>>>  
>>> The Lexile unit is best defined unit I have come across in the 
>>> behavioural sciences, so by what criteria one judges to be a "true 
>>> spacing" (whatever that is) seems to be a mystery...
>>>  
>>> Andrew Kyngdon, PhD
>>>  
>>> Senior Research Scientist
>>>  
>>> MetaMetrics, Inc.
>>>  
>>> 1000 Park Forty Plaza Drive
>>>  
>>> Durham NC 27713 USA
>>>  
>>> Tel. 1 919 354 3473
>>>  
>>> Fax. 1 919 547 3401
>>>  
>>>  
>>>     
>>>       
> ------------------------------------------------------------------------
>   
>>   
>>     
>>> *From:* rasch-bounces at acer.edu.au <mailto:rasch-bounces at acer.edu.au>
>>>       
> [mailto:rasch-bounces at acer.edu.au] 
>   
>>> *On Behalf Of *Mark Moulton
>>> *Sent:* Monday, July 21, 2008 6:09 PM
>>> *To:* Paul Barrett
>>> *Cc:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
>>> *Subject:* Re: [Rasch] Rasch analysis of interval data
>>>  
>>> Paul,
>>>  
>>> Thank you for your explanations and for your presentation 10 years 
>>> ago, which are very helpful to me. You raise a fundamental issue, 
>>> still controversial, still worth visiting in my opinion. You make the
>>>       
>
>   
>>> case that Rasch measures are equal-interval representations of counts
>>>       
>
>   
>>> (ratios of counts), and that is all, and that they do not necessarily
>>>       
>
>   
>>> capture a fundamental unit of measurement in the underlying
>>>       
> construct.
>   
>>>  
>>> I think your point is amplified by a simple desktop experiment. Lay a
>>>       
>
>   
>>> ruler on a sheet of paper. Draw dots (representing persons, say) in 
>>> various distributions on the paper. For each centimeter increment on 
>>> the ruler (representing items), count the dots above and below that 
>>> increment and calculate their log ratio. One finds that the logit 
>>> spacings of the ruler increments may be highly unequal (unlike the 
>>> centimeters), depending on how one distributes the dots. If one 
>>> distributes the dots equally up and down the ruler, the logit lengths
>>>       
>
>   
>>> between increments appear to get fatter at the extremes. If one
>>>       
> clumps
>   
>>>     
>>>       
>>  
>>   
>>     
>>> the dots in multiple modes, the logit lengths can be distorted in all
>>>       
>
>   
>>> sorts of cool ways. Interestingly, as the distribution approaches 
>>> normal, the logit lengths between increments seem to approach a
>>>       
> linear
>   
>>>     
>>>       
>>  
>>   
>>     
>>> relationship with the centimeters, (I don't have a proof for why this
>>>       
>
>   
>>> would be true, but presumably it has something to do with the 
>>> relationship between the logistic and normal distributions and may 
>>> account for the similarity between independently calibrated scales).
>>>  
>>> So, I agree that Rasch logits do not capture fundamental units of 
>>> measurement, and are sample-dependent in this sense (and in several 
>>> other senses, too). My question is: What does this do to Rasch claims
>>>       
>
>   
>>> of "invariance," aka "special objectivity," the notion that the 
>>> relative logit spacings of persons will remain the same regardless of
>>>       
>
>   
>>> how the items are spaced? Strangely, I don't think it has any effect 
>>> at all. The disappearance of the item parameter when calculating the 
>>> person parameter, and vice versa, has the same force and implication 
>>> that it always did. And due to how Rasch conjointly calculates
>>>       
> persons
>   
>>>     
>>>       
>>  
>>   
>>     
>>> and items, whatever distortions may occur affect the persons and
>>>       
> items
>   
>>>     
>>>       
>>  
>>   
>>     
>>> equally.
>>>  
>>> I am left with a relativistic notion of psychometric spaces. Each 
>>> Rasch analysis erects a unique space. That space bears no /necessary/
>>>       
>
>   
>>> relationship to any other Rasch space (except perhaps in some 
>>> topographic one-to-one homeomorphic kind of way). However, objects 
>>> within that space are distributed in a way that is reproducible,
>>>       
> hence
>   
>>>     
>>>       
>>  
>>   
>>     
>>> objective, with respect to other objects in that space. Two Rasch 
>>> spaces can be reconciled only by "anchoring" one space to the other 
>>> via common persons or items. This forces the two spaces to share the 
>>> same "distortions," and thus to become one space, and to preserve 
>>> invariance for all objects residing in that space.
>>>  
>>> Your point about the MetaMetrics lexile scale is well-taken. All
>>>       
> texts
>   
>>>     
>>>       
>>  
>>   
>>     
>>> and readers are forced into a common space anchored on the physical 
>>> properties represented by word frequency and sentence length (or log 
>>> transformations thereof). This was facilitated by the fact that 
>>> MetaMetrics discovered and exploited a linear relationship between 
>>> textual empirical variables and item difficulties. But even without 
>>> that relationship, the two types of variables could have been forced 
>>> (by a method MetaMetrics did not use, or need to use) into a common 
>>> space through an anchoring procedure. This is still not a guarantee 
>>> that the lexile unit captures the "true" spacing, but fortunately it 
>>> does not seem to matter.
>>>  
>>> One space is as good as another, so long as they are internally 
>>> consistent. Am I reading this right?
>>>  
>>> Mark H. Moulton
>>>  
>>> Educational Data Systems
>>>  
>>> 2008/7/21 Paul Barrett <pbarrett at hoganassessments.com
>>>       
> <mailto:pbarrett at hoganassessments.com> 
>   
>>> <mailto:pbarrett at hoganassessments.com>>:
>>>  
>>> *From:* rasch-bounces at acer.edu.au <mailto:rasch-bounces at acer.edu.au>
>>>       
> <mailto:rasch-bounces at acer.edu.au> 
>   
>>> [mailto:rasch-bounces at acer.edu.au <mailto:rasch-bounces at acer.edu.au>]
>>>       
>
>   
>>> *On Behalf Of *Anthony James
>>> *Sent:* Wednesday, July 16, 2008 7:19 AM
>>> *To:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
>>>       
> <mailto:rasch at acer.edu.au>
>   
>>> *Subject:* [Rasch] Rasch analysis of interval data
>>>  
>>> Hi all,
>>>  
>>> Has anyone ever tried to Rasch analyse a variable for which there's 
>>> concatenation-based objective measurement? Suppose we make a height 
>>> scale with 6 points:
>>>  
>>>  
>>>     
>>>       
> ------------------------------------------------------------------------
>   
>>   
>>     
>>> *From:* rasch-bounces at acer.edu.au <mailto:rasch-bounces at acer.edu.au>
>>>       
> <mailto:rasch-bounces at acer.edu.au> 
>   
>>> [mailto:rasch-bounces at acer.edu.au <mailto:rasch-bounces at acer.edu.au>]
>>>       
>
>   
>>> *On Behalf Of *Andrew Kyngdon
>>> *Sent:* Wednesday, July 16, 2008 8:13 AM
>>>  
>>>  
>>> *To:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
>>>       
> <mailto:rasch at acer.edu.au>
>   
>>>  
>>> *Subject:* RE: [Rasch] Rasch analysis of interval data
>>>  
>>> I think Paul Barrett did something like this once...
>>>  
>>>  
>>>     
>>>       
> ------------------------------------------------------------------------
>   
>>   
>>     
>>> Yep - 10 years ago to be exact!
>>>  
>>> Sorry I haven't replied until now ...
>>>  
>>> The presentation about the simulation can be downloaded at:
>>>  
>>> http://www.pbarrett.net/presentations/BPS-rasch_98.pdf
>>>  
>>> From my web-page abstract ...
>>>  
>>> **Beyond Psychometrics: the recovery of a standard unit of length**: 
>>> This 50-slide presentation was given at the British Psychological 
>>> Society's Division of Occupational Psychology conference: Assessment 
>>> in the Millennium: Beyond Psychometrics, November 1998, at Birkbeck 
>>> (University of London). The theme of this presentation was about
>>>       
> Rasch
>   
>>>     
>>>       
>>  
>>   
>>     
>>> scaling, and its capacity to construct a standard unit from 
>>> observational data. This presentation contained a data simulation
>>>       
> that
>   
>>>     
>>>       
>>  
>>   
>>     
>>> attempted to hide a true quantitatively structured latent variable of
>>>       
>
>   
>>> length behind some poor ordinal observations. All the Rasch scaling 
>>> did was to construct an equal-interval latent variable of ordinal 
>>> lengths! This simulation was heavily criticised Ben Wright and
>>>       
> others,
>   
>>>     
>>>       
>>  
>>   
>>     
>>> and I have included these criticisms as an addendum to the 
>>> presentation - along with my reply. However, recent papers seem to 
>>> have vindicated my conclusions in some respects.....The reality is 
>>> that these methods simply construct linear latent variables in 
>>> complete isolation of any empirical evidence that such variables
>>>       
> might
>   
>>>     
>>>       
>>  
>>   
>>     
>>> indeed be quantitatively structured.. In my opinion, from a
>>>       
> scientific
>   
>>>     
>>>       
>>  
>>   
>>     
>>> perspective, these scaling methods are frankly of little utility, but
>>>       
>
>   
>>> they are ingenious from a psychometric perspective and do have great 
>>> utility in a more pragmatic sense. It all comes down to what the 
>>> purpose is for using such scaling, science or number scaling.
>>>  
>>> 10 years on - with some better understanding of things (!) - the goal
>>>       
>
>   
>>> and conclusions of the presentation still make sense - but now I
>>>       
> fully
>   
>>>     
>>>       
>>  
>>   
>>     
>>> understand why. Rasch scaling cannot "uncover" a linear latent 
>>> variable from ordinal measures. It simply scales counts and in
>>>       
> effect,
>   
>>>     
>>>       
>>  
>>   
>>     
>>> the numbers applied to its algorithms, without regard to whether
>>>       
> those
>   
>>>     
>>>       
>>  
>>   
>>     
>>> counts or numbers are drawn from an ordinal or linear scale.
>>>  
>>> The mistake made by many psychologists is to forget that latent 
>>> variable theory implies nothing about the measurement properties of 
>>> the variable of interest - latent variables are simply constructed 
>>> ad-hoc to possess linear properties of measurement. That is not how 
>>> normal science proceeds, it is as Michell states a "pathology of 
>>> science" (2000).
>>>  
>>> I propose that a key exemplar which shows how to properly model data 
>>> while invoking a latent variable, is the work done by Metametrics. It
>>>       
>
>   
>>> is no accident that the initial exploratory work was empirical and 
>>> based upon much cognitive psychological experimentation, PRIOR to the
>>>       
>
>   
>>> scaling/modeling exercises. Andrew has already provided excellent 
>>> explanations of the history of this work, along with another 
>>> exposition recently in his peer response to Michell's target article 
>>> in the journal Measurement (references below).
>>>  
>>> However, if we view edumetrics-psychometrics as largely 
>>> pragmatic/technical work, which is concerned with the efficiencies to
>>>       
>
>   
>>> be gained in standards-based testing/examination/cumulative
>>>       
> risk-scale
>   
>>>     
>>>       
>>  
>>   
>>     
>>> environments, then IRT models in general, and the Rasch model make a 
>>> great deal of sense. I think it is an illusion that the Rasch or any 
>>> IRT/latent variable model magically produces "fundamental
>>>       
> measurement"
>   
>>>     
>>>       
>>  
>>   
>>     
>>> in any sense of the word. Michell (2004, and now 2008) has put paid
>>>       
> to
>   
>>>     
>>>       
>>  
>>   
>>     
>>> this notion.
>>>  
>>> I don't think this is a controversial point anymore - from the 
>>> standpoint of simple logic, the work by Robert Wood, and from my own 
>>> small and almost stupid simulation, the Rasch model cannot possibly 
>>> "uncover/discover" the true metric for a "statistically constructed 
>>> latent variable". It just does what it does given the data with which
>>>       
>
>   
>>> it is presented. Whether or not that data is an accurate 
>>> representation/set of observations of the phenomenon of interest (my 
>>> "bad ruler"), the Rasch scaling will simple create a latent variable 
>>> anyway - given sufficient stochastic error in the observations (as 
>>> with Wood's coin-tosses). Which is why I think the Metametrics 
>>> exemplar is so very important, the scaling is constructed around a 
>>> wealth of empirical phenomena and magnitude relationships - and not 
>>> just banks of "item responses".
>>>  
>>> Regards ... Paul
>>>  
>>> __________________________________________________
>>> Paul Barrett 918.749-0632 x 326
>>> Chief Research Scientist Skype: pbar088
>>> Hogan Assessment Systems Inc.
>>> 2622 East 21st St., Tulsa, OK 74114
>>>  
>>> **References**
>>>  
>>> Kyngdon, A. (2008) Treating the Pathology of Psychometrics: An
>>>       
> Example
>   
>>>     
>>>       
>>  
>>   
>>     
>>> from the Comprehension of Continuous Prose Text. //Measurement: 
>>> Interdisciplinary Research & Perspective//, 6, 1 & 2, 108-113.
>>>  
>>> Michell. J. (2000) Normal science, pathological Science, and 
>>> psychometrics. Theory and Psychology, 10, 5, 639-667.
>>>  
>>> Michell, J. (2004) Item Response Models, pathological science, and
>>>       
> the
>   
>>>     
>>>       
>>  
>>   
>>     
>>> shape of error. //Theory and Psychology//, 14, 1, 121-129.
>>>  
>>> Michell, J. (2008) Is psychometrics pathological science? 
>>> //Measurement: Interdisciplinary Research & Perspective//, 6, 1, 7-24
>>>  
>>> Wood, R. (1978) Fitting the rasch model - a heady tale. //British 
>>> Journal of Mathematical and Statistical Psychology//, 31, , 27-32.
>>>  
>>> **An aside**
>>>  
>>> The journal "Measurement: Interdisciplinary Research and 
>>> Perspective"published issues two issues simultaneously - three target
>>>       
>
>   
>>> articles and commenatries on the issue:
>>>  
>>> //The Conceptual Foundations of Psychological Measurement//
>>>  
>>> The target papers by Denny Borsboom and Keith Markus are also 
>>> excellent expositions of their respective positions. Very nice 
>>> position pieces.
>>>  
>>> I've attached the journal link here so you can look at the paper 
>>> titles etc.
>>>  
>>> http://www.informaworld.com/smpp/title~content=g794512699~db=all
>>>       
> <http://www.informaworld.com/smpp/title%7Econtent=g794512699%7Edb=all> 
>   
> <http://www.informaworld.com/smpp/title%7Econtent=g794512699%7Edb=all>
>   
>>>  
>>>  
>>> _______________________________________________
>>> Rasch mailing list
>>> Rasch at acer.edu.au <mailto:Rasch at acer.edu.au>
>>>       
> <mailto:Rasch at acer.edu.au>
>   
>>> http://mailinglist.acer.edu.au/mailman/listinfo/rasch
>>>  
>>>  
>>>     
>>>       
> ------------------------------------------------------------------------
>   
>>   
>>     
>>> _______________________________________________
>>> Rasch mailing list
>>> Rasch at acer.edu.au <mailto:Rasch at acer.edu.au>
>>> http://mailinglist.acer.edu.au/mailman/listinfo/rasch
>>>   
>>>     
>>>       
>>  
>>  
>>  
>>   
>>
>>  
>>
>>     
>
>
>
>   




More information about the Rasch mailing list