[Rasch] Rasch analysis of interval data

Michael Lamport Commons commons at tiac.net
Thu Jul 24 07:16:16 EST 2008


Yes it is.  Except you are not noticing "not" in:"actions does not 
equal"  Russell's paradox says they are equal

Andrew Kyngdon wrote:
> "Each action is discrete and different from all the other actions.  An
> action that is defined in terms of other actions does not equal any of
> those actions:  Set A does not equal the members of Set A."
>
> Isn't this an example of the Russell Paradox A = {A | A is not in A}?
>  
>
> Andrew Kyngdon, PhD
> Senior Research Scientist
> MetaMetrics, Inc.
> 1000 Park Forty Plaza Drive
> Durham NC 27713 USA
> Tel. 1 919 354 3473
> Fax. 1 919 547 3401
>
>  
>
>
> -----Original Message-----
> From: Michael Lamport Commons [mailto:commons at tiac.net] 
> Sent: Wednesday, July 23, 2008 4:32 PM
> To: Andrew Kyngdon
> Cc: rasch at acer.edu.au
> Subject: Re: [Rasch] Rasch analysis of interval data
>
> By the way, hierarchical complexity goes beyond cognitive development.  
> It is a content and context free Mathematical theory.  For example, 
> molecules are made up of atoms.  An atom is made up of subatomic 
> particles.  Subatomic particles are made up of quarks.  Luce has been my
>
> advisor on this project.  He says that there was no need to assume 
> anything beyond ordinality.  You need to read his reasoning on why with 
> ordinal scales, things converge to the rationals relatively fast.  I you
>
> look at the graphs, you can see they are very very linear.
>
> At each order, a task action is defined in terms of 2 or more different 
> lower order actions.  There can be an infinite number of such actions.  
> But in one sequence of such actions, there is only one such action.  
> This creates an upside down tree like structure.  At order 0, no actions
>
> are organized.  At order 1, there are 2 simple actions.  At order 2, 2 
> order 1 actions are organized.  At order 3, 2 order 2 actions are 
> organized yielding 4 total actions.  At order 4, 2 order 3 actions are 
> organized yielding 8 actions.  As you can see, one can count the 
> actions.  Each action is discrete and different from all the other 
> actions.  An action that is defined in terms of other actions does not 
> equal any of those actions:  Set A does not equal the members of Set A.
>
> Andrew Kyngdon wrote:
>   
>> Michael,
>>
>> >From the information on the wikipage and from the graph you have
>>     
> sent,
>   
>> it strongly appears to me that your model of hierarchical complexity
>>     
> is
>   
>> an ordinal theory of cognitive development. Ordinal things are not
>> measurable, unless one subscribes to S.S. Stevens views and believes
>> that ordinal "scales" produce measurements.
>>
>> I don't consider that you have an absolute scale here at all. You have
>>     
> a
>   
>> theory of cognitive development that is ordinal - you don't have a
>> discrete quantity here (like an aggregate of some kind) which can be
>> measured through counting. Your theory is not a mere collection of
>> objects - it is much more complex than that.
>>
>> Don't get me wrong. Psychology needs explicit theories of cognitive
>> processes of the kind you have developed. It desperately needs more
>> Piagets than it does Fred Lords. The truly disturbing thing about the
>> behavioural sciences is the attitude of whilst it is nice to have
>> "substantive theory", it's not necessary for measurement. On the
>> contrary, if we are genuinely measuring a natural system, then our
>> "substantive" and "measurement" theories are one and the same thing,
>> from a realist perspective. Instead, psychometricians are always
>>     
> hoping
>   
>> the practitioners will provide the substantive theory, whilst the
>> practitioners seem to think that psychometric models actually provide
>> this. This kind of thinking pervades and dominates the SEM and HLM
>> lands, for example.
>>  
>> By the way, the definition of an absolute scale given in the
>>     
> "Dictionary
>   
>> of Psychology" is, true to form, quite poor. In all measurement in
>> physics, regardless of whether one is using the so called ratio,
>> absolute or interval scales, the unit is always "fixed".
>>
>> Cheers,
>>
>> Andrew
>>  
>>
>> -----Original Message-----
>> From: Michael Lamport Commons [mailto:commons at tiac.net] 
>> Sent: Tuesday, July 22, 2008 5:44 PM
>> To: Andrew Kyngdon
>> Cc: rasch at acer.edu.au
>> Subject: Re: [Rasch] Rasch analysis of interval data
>>
>>
>>   absolute scale
>>
>>
>>     A Dictionary of Psychology | Date: 2001
>>
>> *absolute scale n.* In statistics and measurement theory, a ratio
>>     
> scale 
>   
>> <http://www.encyclopedia.com/doc/1O87-ratioscale.html> in which the
>>     
> unit
>   
>> of measurement is fixed. In practice, values on an absolute scale are 
>> usually if not always obtained by counting.
>>
>> MLC:  What is counted is the number of times in a hierarchy that a 
>> higher order action coordinates two or more lower order actions in a 
>> non-arbitrary way.  It has a tree structure so one counts how high up
>>     
> in
>   
>> the tree one is.
>>
>> MLC
>>
>> Andrew Kyngdon wrote:
>>   
>>     
>>> "It is absolute in the sense that it does not require norms, it does
>>>     
>>>       
>> not
>>   
>>     
>>> need to be tested on people to know what the hierarchical complexity
>>>     
>>>       
>> of 
>>   
>>     
>>> a task is".
>>>
>>> It does not logically follow from these premises that you can measure
>>>     
>>>       
>> an
>>   
>>     
>>> ordinal structure with an "absolute" scale.
>>>  
>>>
>>>  
>>> -----Original Message-----
>>> From: Michael Lamport Commons [mailto:commons at tiac.net] 
>>> Sent: Tuesday, July 22, 2008 4:47 PM
>>> To: Andrew Kyngdon
>>> Cc: rasch at acer.edu.au
>>> Subject: Re: [Rasch] Rasch analysis of interval data
>>>
>>> Andrew Kyngdon wrote:
>>>   
>>>     
>>>       
>>>> How can an ordinal structure be measured with an absolute scale?
>>>>
>>>>     
>>>>       
>>>>         
>>> It is absolute in the sense that it does not require norms, it does
>>>     
>>>       
>> not 
>>   
>>     
>>> need to be tested on people to know what the hierarchical complexity
>>>     
>>>       
>> of 
>>   
>>     
>>> a task is. 
>>>
>>> MLC
>>>   
>>>     
>>>       
>>>>  
>>>>
>>>>  
>>>>
>>>> Andrew Kyngdon, PhD
>>>>
>>>> Senior Research Scientist
>>>>
>>>> MetaMetrics, Inc.
>>>>
>>>> 1000 Park Forty Plaza Drive
>>>>
>>>> Durham NC 27713 USA
>>>>
>>>> Tel. 1 919 354 3473
>>>>
>>>> Fax. 1 919 547 3401
>>>>
>>>>  
>>>>
>>>>
>>>>     
>>>>       
>>>>         
> ------------------------------------------------------------------------
>   
>>   
>>     
>>>   
>>>     
>>>       
>>>> *From:* Michael Lamport Commons [mailto:commons at tiac.net]
>>>> *Sent:* Tuesday, July 22, 2008 2:04 PM
>>>> *To:* Andrew Kyngdon
>>>> *Cc:* rasch at acer.edu.au
>>>> *Subject:* Re: [Rasch] Rasch analysis of interval data
>>>>
>>>>  
>>>>
>>>> The orders of hierarchical complexity are ordinal, universal,
>>>>       
>>>>         
>> context,
>>   
>>     
>>>>     
>>>>       
>>>>         
>>>   
>>>     
>>>       
>>>> content, and participant free.  The are an analytic measure of the 
>>>> hierarchical complexity of tasks.  We know of 15 orders.  You can
>>>>         
> see
>   
>>>>       
>>>>         
>>   
>>     
>>>> a description on Wikipedia.  There is a non-arbitrary zero.  Rasch 
>>>> measures performance.  Hierarchical complexity measures tasks 
>>>> properties.  This is psychophysics.  The y-axis is Rasch Score, the
>>>>         
> x
>   
>>>>       
>>>>         
>>   
>>     
>>>> axis is order of hierarhical complexity.  The r's are mostly in the
>>>>       
>>>>         
>> .9
>>   
>>     
>>>>     
>>>>       
>>>>         
>>>   
>>>     
>>>       
>>>> -.99 range.
>>>>
>>>> MLC
>>>>
>>>> Andrew Kyngdon wrote:
>>>>
>>>> Michael,
>>>>  
>>>> I'm not familiar with your work at all, but I take it by "absolute"
>>>> scale that you mean you have a continuous, quantitative attribute
>>>>       
>>>>         
>> that
>>   
>>     
>>>> you can measure with a scale possessing a non-arbitrary zero point?
>>>>       
>>>>         
>> If
>>   
>>     
>>>> so, that is quite a feat in the behavioural sciences, given the lack
>>>>     
>>>>       
>>>>         
>>> of
>>>   
>>>     
>>>       
>>>> natural concatenation operations.
>>>>  
>>>> But you state that you can transform Rasch scores into this
>>>>       
>>>>         
>> supposedly
>>   
>>     
>>>> "absolute" scale of "Stage scores". Now, correct me if I am wrong,
>>>>       
>>>>         
>> but
>>   
>>     
>>>> Rasch logits are usually advanced as interval scale measurements
>>>> (leaving aside the obvious problem of a lack of a defined unit,
>>>> something Steve Humphry has been at pains to point out). Interval
>>>>     
>>>>       
>>>>         
>>> scale
>>>   
>>>     
>>>       
>>>> measurements cannot be meaningfully transformed into ratio or
>>>>       
>>>>         
>> absolute
>>   
>>     
>>>> scales, unless your substantive theory is sufficiently understood to
>>>> enable this, such as in temperature with converting Celsius and
>>>> Fahrenheit measurements into the "absolute" Kelvin scale. But if you
>>>>     
>>>>       
>>>>         
>>> can
>>>   
>>>     
>>>       
>>>> measure something with an absolute scale in the first place, why
>>>>       
>>>>         
>> would
>>   
>>     
>>>> you bother with an interval scale, unless there are historical
>>>>       
>>>>         
>> reasons
>>   
>>     
>>>> (as in temperature) for so doing?
>>>>  
>>>>  
>>>>  
>>>> Andrew Kyngdon, PhD
>>>> Senior Research Scientist
>>>> MetaMetrics, Inc.
>>>> 1000 Park Forty Plaza Drive
>>>> Durham NC 27713 USA
>>>> Tel. 1 919 354 3473
>>>> Fax. 1 919 547 3401
>>>>  
>>>>  
>>>>  
>>>>  
>>>> -----Original Message-----
>>>> From: Michael Lamport Commons [mailto:commons at tiac.net] 
>>>> Sent: Tuesday, July 22, 2008 12:38 PM
>>>> To: Andrew Kyngdon
>>>> Cc: Mark Moulton; Paul Barrett; rasch at acer.edu.au
>>>>     
>>>>       
>>>>         
>>> <mailto:rasch at acer.edu.au>
>>>   
>>>     
>>>       
>>>> Subject: Re: [Rasch] Rasch analysis of interval data
>>>>  
>>>> We have an absolute scale in order of hierarchical complexity. We 
>>>> transform the Rasch scores into Stage scores which are based on the 
>>>> absolute scale.
>>>>  
>>>> Michael Lamport Commons
>>>>  
>>>> Andrew Kyngdon wrote:
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> This is still not a guarantee that the lexile unit captures the
>>>>>       
>>>>>         
>>>>>           
>>> "true"
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> spacing, but fortunately it does not seem to matter.
>>>>>  
>>>>> The Lexile unit is best defined unit I have come across in the 
>>>>> behavioural sciences, so by what criteria one judges to be a "true 
>>>>> spacing" (whatever that is) seems to be a mystery...
>>>>>  
>>>>> Andrew Kyngdon, PhD
>>>>>  
>>>>> Senior Research Scientist
>>>>>  
>>>>> MetaMetrics, Inc.
>>>>>  
>>>>> 1000 Park Forty Plaza Drive
>>>>>  
>>>>> Durham NC 27713 USA
>>>>>  
>>>>> Tel. 1 919 354 3473
>>>>>  
>>>>> Fax. 1 919 547 3401
>>>>>  
>>>>>  
>>>>>     
>>>>>       
>>>>>         
>>>>>           
> ------------------------------------------------------------------------
>   
>>   
>>     
>>>   
>>>     
>>>       
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> *From:* rasch-bounces at acer.edu.au
>>>>>           
> <mailto:rasch-bounces at acer.edu.au>
>   
>>>>>       
>>>>>         
>>>>>           
>>> [mailto:rasch-bounces at acer.edu.au] 
>>>   
>>>     
>>>       
>>>>> *On Behalf Of *Mark Moulton
>>>>> *Sent:* Monday, July 21, 2008 6:09 PM
>>>>> *To:* Paul Barrett
>>>>> *Cc:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
>>>>> *Subject:* Re: [Rasch] Rasch analysis of interval data
>>>>>  
>>>>> Paul,
>>>>>  
>>>>> Thank you for your explanations and for your presentation 10 years 
>>>>> ago, which are very helpful to me. You raise a fundamental issue, 
>>>>> still controversial, still worth visiting in my opinion. You make
>>>>>         
>>>>>           
>> the
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> case that Rasch measures are equal-interval representations of
>>>>>         
>>>>>           
>> counts
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> (ratios of counts), and that is all, and that they do not
>>>>>         
>>>>>           
>> necessarily
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> capture a fundamental unit of measurement in the underlying
>>>>>       
>>>>>         
>>>>>           
>>> construct.
>>>   
>>>     
>>>       
>>>>>  
>>>>> I think your point is amplified by a simple desktop experiment. Lay
>>>>>         
>>>>>           
>> a
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> ruler on a sheet of paper. Draw dots (representing persons, say) in
>>>>>           
>
>   
>>>>> various distributions on the paper. For each centimeter increment
>>>>>           
> on
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> the ruler (representing items), count the dots above and below that
>>>>>           
>
>   
>>>>> increment and calculate their log ratio. One finds that the logit 
>>>>> spacings of the ruler increments may be highly unequal (unlike the 
>>>>> centimeters), depending on how one distributes the dots. If one 
>>>>> distributes the dots equally up and down the ruler, the logit
>>>>>         
>>>>>           
>> lengths
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> between increments appear to get fatter at the extremes. If one
>>>>>       
>>>>>         
>>>>>           
>>> clumps
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> the dots in multiple modes, the logit lengths can be distorted in
>>>>>         
>>>>>           
>> all
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> sorts of cool ways. Interestingly, as the distribution approaches 
>>>>> normal, the logit lengths between increments seem to approach a
>>>>>       
>>>>>         
>>>>>           
>>> linear
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> relationship with the centimeters, (I don't have a proof for why
>>>>>         
>>>>>           
>> this
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> would be true, but presumably it has something to do with the 
>>>>> relationship between the logistic and normal distributions and may 
>>>>> account for the similarity between independently calibrated
>>>>>           
> scales).
>   
>>>>>  
>>>>> So, I agree that Rasch logits do not capture fundamental units of 
>>>>> measurement, and are sample-dependent in this sense (and in several
>>>>>           
>
>   
>>>>> other senses, too). My question is: What does this do to Rasch
>>>>>         
>>>>>           
>> claims
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> of "invariance," aka "special objectivity," the notion that the 
>>>>> relative logit spacings of persons will remain the same regardless
>>>>>         
>>>>>           
>> of
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> how the items are spaced? Strangely, I don't think it has any
>>>>>           
> effect
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> at all. The disappearance of the item parameter when calculating
>>>>>           
> the
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> person parameter, and vice versa, has the same force and
>>>>>           
> implication
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> that it always did. And due to how Rasch conjointly calculates
>>>>>       
>>>>>         
>>>>>           
>>> persons
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> and items, whatever distortions may occur affect the persons and
>>>>>       
>>>>>         
>>>>>           
>>> items
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> equally.
>>>>>  
>>>>> I am left with a relativistic notion of psychometric spaces. Each 
>>>>> Rasch analysis erects a unique space. That space bears no
>>>>>         
>>>>>           
>> /necessary/
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> relationship to any other Rasch space (except perhaps in some 
>>>>> topographic one-to-one homeomorphic kind of way). However, objects 
>>>>> within that space are distributed in a way that is reproducible,
>>>>>       
>>>>>         
>>>>>           
>>> hence
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> objective, with respect to other objects in that space. Two Rasch 
>>>>> spaces can be reconciled only by "anchoring" one space to the other
>>>>>           
>
>   
>>>>> via common persons or items. This forces the two spaces to share
>>>>>           
> the
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> same "distortions," and thus to become one space, and to preserve 
>>>>> invariance for all objects residing in that space.
>>>>>  
>>>>> Your point about the MetaMetrics lexile scale is well-taken. All
>>>>>       
>>>>>         
>>>>>           
>>> texts
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> and readers are forced into a common space anchored on the physical
>>>>>           
>
>   
>>>>> properties represented by word frequency and sentence length (or
>>>>>           
> log
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> transformations thereof). This was facilitated by the fact that 
>>>>> MetaMetrics discovered and exploited a linear relationship between 
>>>>> textual empirical variables and item difficulties. But even without
>>>>>           
>
>   
>>>>> that relationship, the two types of variables could have been
>>>>>           
> forced
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> (by a method MetaMetrics did not use, or need to use) into a common
>>>>>           
>
>   
>>>>> space through an anchoring procedure. This is still not a guarantee
>>>>>           
>
>   
>>>>> that the lexile unit captures the "true" spacing, but fortunately
>>>>>           
> it
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> does not seem to matter.
>>>>>  
>>>>> One space is as good as another, so long as they are internally 
>>>>> consistent. Am I reading this right?
>>>>>  
>>>>> Mark H. Moulton
>>>>>  
>>>>> Educational Data Systems
>>>>>  
>>>>> 2008/7/21 Paul Barrett <pbarrett at hoganassessments.com
>>>>>       
>>>>>         
>>>>>           
>>> <mailto:pbarrett at hoganassessments.com> 
>>>   
>>>     
>>>       
>>>>> <mailto:pbarrett at hoganassessments.com>>:
>>>>>  
>>>>> *From:* rasch-bounces at acer.edu.au
>>>>>           
> <mailto:rasch-bounces at acer.edu.au>
>   
>>>>>       
>>>>>         
>>>>>           
>>> <mailto:rasch-bounces at acer.edu.au> 
>>>   
>>>     
>>>       
>>>>> [mailto:rasch-bounces at acer.edu.au
>>>>>         
>>>>>           
>> <mailto:rasch-bounces at acer.edu.au>]
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> *On Behalf Of *Anthony James
>>>>> *Sent:* Wednesday, July 16, 2008 7:19 AM
>>>>> *To:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
>>>>>       
>>>>>         
>>>>>           
>>> <mailto:rasch at acer.edu.au>
>>>   
>>>     
>>>       
>>>>> *Subject:* [Rasch] Rasch analysis of interval data
>>>>>  
>>>>> Hi all,
>>>>>  
>>>>> Has anyone ever tried to Rasch analyse a variable for which there's
>>>>>           
>
>   
>>>>> concatenation-based objective measurement? Suppose we make a height
>>>>>           
>
>   
>>>>> scale with 6 points:
>>>>>  
>>>>>  
>>>>>     
>>>>>       
>>>>>         
>>>>>           
> ------------------------------------------------------------------------
>   
>>   
>>     
>>>   
>>>     
>>>       
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> *From:* rasch-bounces at acer.edu.au
>>>>>           
> <mailto:rasch-bounces at acer.edu.au>
>   
>>>>>       
>>>>>         
>>>>>           
>>> <mailto:rasch-bounces at acer.edu.au> 
>>>   
>>>     
>>>       
>>>>> [mailto:rasch-bounces at acer.edu.au
>>>>>         
>>>>>           
>> <mailto:rasch-bounces at acer.edu.au>]
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> *On Behalf Of *Andrew Kyngdon
>>>>> *Sent:* Wednesday, July 16, 2008 8:13 AM
>>>>>  
>>>>>  
>>>>> *To:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
>>>>>       
>>>>>         
>>>>>           
>>> <mailto:rasch at acer.edu.au>
>>>   
>>>     
>>>       
>>>>>  
>>>>> *Subject:* RE: [Rasch] Rasch analysis of interval data
>>>>>  
>>>>> I think Paul Barrett did something like this once...
>>>>>  
>>>>>  
>>>>>     
>>>>>       
>>>>>         
>>>>>           
> ------------------------------------------------------------------------
>   
>>   
>>     
>>>   
>>>     
>>>       
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> Yep - 10 years ago to be exact!
>>>>>  
>>>>> Sorry I haven't replied until now ...
>>>>>  
>>>>> The presentation about the simulation can be downloaded at:
>>>>>  
>>>>> http://www.pbarrett.net/presentations/BPS-rasch_98.pdf
>>>>>  
>>>>> From my web-page abstract ...
>>>>>  
>>>>> **Beyond Psychometrics: the recovery of a standard unit of
>>>>>           
> length**:
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> This 50-slide presentation was given at the British Psychological 
>>>>> Society's Division of Occupational Psychology conference:
>>>>>           
> Assessment
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> in the Millennium: Beyond Psychometrics, November 1998, at Birkbeck
>>>>>           
>
>   
>>>>> (University of London). The theme of this presentation was about
>>>>>       
>>>>>         
>>>>>           
>>> Rasch
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> scaling, and its capacity to construct a standard unit from 
>>>>> observational data. This presentation contained a data simulation
>>>>>       
>>>>>         
>>>>>           
>>> that
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> attempted to hide a true quantitatively structured latent variable
>>>>>         
>>>>>           
>> of
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> length behind some poor ordinal observations. All the Rasch scaling
>>>>>           
>
>   
>>>>> did was to construct an equal-interval latent variable of ordinal 
>>>>> lengths! This simulation was heavily criticised Ben Wright and
>>>>>       
>>>>>         
>>>>>           
>>> others,
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> and I have included these criticisms as an addendum to the 
>>>>> presentation - along with my reply. However, recent papers seem to 
>>>>> have vindicated my conclusions in some respects.....The reality is 
>>>>> that these methods simply construct linear latent variables in 
>>>>> complete isolation of any empirical evidence that such variables
>>>>>       
>>>>>         
>>>>>           
>>> might
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> indeed be quantitatively structured.. In my opinion, from a
>>>>>       
>>>>>         
>>>>>           
>>> scientific
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> perspective, these scaling methods are frankly of little utility,
>>>>>         
>>>>>           
>> but
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> they are ingenious from a psychometric perspective and do have
>>>>>           
> great
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> utility in a more pragmatic sense. It all comes down to what the 
>>>>> purpose is for using such scaling, science or number scaling.
>>>>>  
>>>>> 10 years on - with some better understanding of things (!) - the
>>>>>         
>>>>>           
>> goal
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> and conclusions of the presentation still make sense - but now I
>>>>>       
>>>>>         
>>>>>           
>>> fully
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> understand why. Rasch scaling cannot "uncover" a linear latent 
>>>>> variable from ordinal measures. It simply scales counts and in
>>>>>       
>>>>>         
>>>>>           
>>> effect,
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> the numbers applied to its algorithms, without regard to whether
>>>>>       
>>>>>         
>>>>>           
>>> those
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> counts or numbers are drawn from an ordinal or linear scale.
>>>>>  
>>>>> The mistake made by many psychologists is to forget that latent 
>>>>> variable theory implies nothing about the measurement properties of
>>>>>           
>
>   
>>>>> the variable of interest - latent variables are simply constructed 
>>>>> ad-hoc to possess linear properties of measurement. That is not how
>>>>>           
>
>   
>>>>> normal science proceeds, it is as Michell states a "pathology of 
>>>>> science" (2000).
>>>>>  
>>>>> I propose that a key exemplar which shows how to properly model
>>>>>           
> data
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> while invoking a latent variable, is the work done by Metametrics.
>>>>>         
>>>>>           
>> It
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> is no accident that the initial exploratory work was empirical and 
>>>>> based upon much cognitive psychological experimentation, PRIOR to
>>>>>         
>>>>>           
>> the
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> scaling/modeling exercises. Andrew has already provided excellent 
>>>>> explanations of the history of this work, along with another 
>>>>> exposition recently in his peer response to Michell's target
>>>>>           
> article
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> in the journal Measurement (references below).
>>>>>  
>>>>> However, if we view edumetrics-psychometrics as largely 
>>>>> pragmatic/technical work, which is concerned with the efficiencies
>>>>>         
>>>>>           
>> to
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> be gained in standards-based testing/examination/cumulative
>>>>>       
>>>>>         
>>>>>           
>>> risk-scale
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> environments, then IRT models in general, and the Rasch model make
>>>>>           
> a
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> great deal of sense. I think it is an illusion that the Rasch or
>>>>>           
> any
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> IRT/latent variable model magically produces "fundamental
>>>>>       
>>>>>         
>>>>>           
>>> measurement"
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> in any sense of the word. Michell (2004, and now 2008) has put paid
>>>>>       
>>>>>         
>>>>>           
>>> to
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> this notion.
>>>>>  
>>>>> I don't think this is a controversial point anymore - from the 
>>>>> standpoint of simple logic, the work by Robert Wood, and from my
>>>>>           
> own
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> small and almost stupid simulation, the Rasch model cannot possibly
>>>>>           
>
>   
>>>>> "uncover/discover" the true metric for a "statistically constructed
>>>>>           
>
>   
>>>>> latent variable". It just does what it does given the data with
>>>>>         
>>>>>           
>> which
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> it is presented. Whether or not that data is an accurate 
>>>>> representation/set of observations of the phenomenon of interest
>>>>>           
> (my
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> "bad ruler"), the Rasch scaling will simple create a latent
>>>>>           
> variable
>   
>>>>>         
>>>>>           
>>   
>>     
>>>>> anyway - given sufficient stochastic error in the observations (as 
>>>>> with Wood's coin-tosses). Which is why I think the Metametrics 
>>>>> exemplar is so very important, the scaling is constructed around a 
>>>>> wealth of empirical phenomena and magnitude relationships - and not
>>>>>           
>
>   
>>>>> just banks of "item responses".
>>>>>  
>>>>> Regards ... Paul
>>>>>  
>>>>> __________________________________________________
>>>>> Paul Barrett 918.749-0632 x 326
>>>>> Chief Research Scientist Skype: pbar088
>>>>> Hogan Assessment Systems Inc.
>>>>> 2622 East 21st St., Tulsa, OK 74114
>>>>>  
>>>>> **References**
>>>>>  
>>>>> Kyngdon, A. (2008) Treating the Pathology of Psychometrics: An
>>>>>       
>>>>>         
>>>>>           
>>> Example
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> from the Comprehension of Continuous Prose Text. //Measurement: 
>>>>> Interdisciplinary Research & Perspective//, 6, 1 & 2, 108-113.
>>>>>  
>>>>> Michell. J. (2000) Normal science, pathological Science, and 
>>>>> psychometrics. Theory and Psychology, 10, 5, 639-667.
>>>>>  
>>>>> Michell, J. (2004) Item Response Models, pathological science, and
>>>>>       
>>>>>         
>>>>>           
>>> the
>>>   
>>>     
>>>       
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> shape of error. //Theory and Psychology//, 14, 1, 121-129.
>>>>>  
>>>>> Michell, J. (2008) Is psychometrics pathological science? 
>>>>> //Measurement: Interdisciplinary Research & Perspective//, 6, 1,
>>>>>         
>>>>>           
>> 7-24
>>   
>>     
>>>>>  
>>>>> Wood, R. (1978) Fitting the rasch model - a heady tale. //British 
>>>>> Journal of Mathematical and Statistical Psychology//, 31, , 27-32.
>>>>>  
>>>>> **An aside**
>>>>>  
>>>>> The journal "Measurement: Interdisciplinary Research and 
>>>>> Perspective"published issues two issues simultaneously - three
>>>>>         
>>>>>           
>> target
>>   
>>     
>>>>>       
>>>>>         
>>>>>           
>>>   
>>>     
>>>       
>>>>> articles and commenatries on the issue:
>>>>>  
>>>>> //The Conceptual Foundations of Psychological Measurement//
>>>>>  
>>>>> The target papers by Denny Borsboom and Keith Markus are also 
>>>>> excellent expositions of their respective positions. Very nice 
>>>>> position pieces.
>>>>>  
>>>>> I've attached the journal link here so you can look at the paper 
>>>>> titles etc.
>>>>>  
>>>>> http://www.informaworld.com/smpp/title~content=g794512699~db=all
>>>>>       
>>>>>         
>>>>>           
> <http://www.informaworld.com/smpp/title%7Econtent=g794512699%7Edb=all>
>   
>>>     
>>>       
>>   
>>     
>>>   
>>>
>>>       
> <http://www.informaworld.com/smpp/title%7Econtent=g794512699%7Edb=all>
>   
>>>   
>>>     
>>>       
>>>>>  
>>>>>  
>>>>> _______________________________________________
>>>>> Rasch mailing list
>>>>> Rasch at acer.edu.au <mailto:Rasch at acer.edu.au>
>>>>>       
>>>>>         
>>>>>           
>>> <mailto:Rasch at acer.edu.au>
>>>   
>>>     
>>>       
>>>>> http://mailinglist.acer.edu.au/mailman/listinfo/rasch
>>>>>  
>>>>>  
>>>>>     
>>>>>       
>>>>>         
>>>>>           
> ------------------------------------------------------------------------
>   
>>   
>>     
>>>   
>>>     
>>>       
>>>>   
>>>>     
>>>>       
>>>>         
>>>>> _______________________________________________
>>>>> Rasch mailing list
>>>>> Rasch at acer.edu.au <mailto:Rasch at acer.edu.au>
>>>>> http://mailinglist.acer.edu.au/mailman/listinfo/rasch
>>>>>   
>>>>>     
>>>>>       
>>>>>         
>>>>>           
>>>>  
>>>>  
>>>>  
>>>>   
>>>>
>>>>  
>>>>
>>>>     
>>>>       
>>>>         
>>>   
>>>     
>>>       
>>
>>   
>>     
>
>
>
>   




More information about the Rasch mailing list