# [Rasch] Rasch analysis of interval data

Michael Lamport Commons commons at tiac.net
Thu Jul 24 07:18:01 EST 2008

```But he thinks it is a scale and a measure.

Andrew Kyngdon wrote:
> "Luce has been my advisor on this project.  He says that there was no
> need to assume anything beyond ordinality."
>
> That's been my point all along...
>
>
> Andrew Kyngdon, PhD
> Senior Research Scientist
> MetaMetrics, Inc.
> 1000 Park Forty Plaza Drive
> Durham NC 27713 USA
> Tel. 1 919 354 3473
> Fax. 1 919 547 3401
>
>
>
>
> -----Original Message-----
> From: Michael Lamport Commons [mailto:commons at tiac.net]
> Sent: Wednesday, July 23, 2008 4:32 PM
> To: Andrew Kyngdon
> Cc: rasch at acer.edu.au
> Subject: Re: [Rasch] Rasch analysis of interval data
>
> By the way, hierarchical complexity goes beyond cognitive development.
> It is a content and context free Mathematical theory.  For example,
> molecules are made up of atoms.  An atom is made up of subatomic
> particles.  Subatomic particles are made up of quarks.  Luce has been my
>
> advisor on this project.  He says that there was no need to assume
> anything beyond ordinality.  You need to read his reasoning on why with
> ordinal scales, things converge to the rationals relatively fast.  I you
>
> look at the graphs, you can see they are very very linear.
>
> At each order, a task action is defined in terms of 2 or more different
> lower order actions.  There can be an infinite number of such actions.
> But in one sequence of such actions, there is only one such action.
> This creates an upside down tree like structure.  At order 0, no actions
>
> are organized.  At order 1, there are 2 simple actions.  At order 2, 2
> order 1 actions are organized.  At order 3, 2 order 2 actions are
> organized yielding 4 total actions.  At order 4, 2 order 3 actions are
> organized yielding 8 actions.  As you can see, one can count the
> actions.  Each action is discrete and different from all the other
> actions.  An action that is defined in terms of other actions does not
> equal any of those actions:  Set A does not equal the members of Set A.
>
> Andrew Kyngdon wrote:
>
>> Michael,
>>
>> >From the information on the wikipage and from the graph you have
>>
> sent,
>
>> it strongly appears to me that your model of hierarchical complexity
>>
> is
>
>> an ordinal theory of cognitive development. Ordinal things are not
>> measurable, unless one subscribes to S.S. Stevens views and believes
>> that ordinal "scales" produce measurements.
>>
>> I don't consider that you have an absolute scale here at all. You have
>>
> a
>
>> theory of cognitive development that is ordinal - you don't have a
>> discrete quantity here (like an aggregate of some kind) which can be
>> measured through counting. Your theory is not a mere collection of
>> objects - it is much more complex than that.
>>
>> Don't get me wrong. Psychology needs explicit theories of cognitive
>> processes of the kind you have developed. It desperately needs more
>> Piagets than it does Fred Lords. The truly disturbing thing about the
>> behavioural sciences is the attitude of whilst it is nice to have
>> "substantive theory", it's not necessary for measurement. On the
>> contrary, if we are genuinely measuring a natural system, then our
>> "substantive" and "measurement" theories are one and the same thing,
>> from a realist perspective. Instead, psychometricians are always
>>
> hoping
>
>> the practitioners will provide the substantive theory, whilst the
>> practitioners seem to think that psychometric models actually provide
>> this. This kind of thinking pervades and dominates the SEM and HLM
>> lands, for example.
>>
>> By the way, the definition of an absolute scale given in the
>>
> "Dictionary
>
>> of Psychology" is, true to form, quite poor. In all measurement in
>> physics, regardless of whether one is using the so called ratio,
>> absolute or interval scales, the unit is always "fixed".
>>
>> Cheers,
>>
>> Andrew
>>
>>
>> -----Original Message-----
>> From: Michael Lamport Commons [mailto:commons at tiac.net]
>> Sent: Tuesday, July 22, 2008 5:44 PM
>> To: Andrew Kyngdon
>> Cc: rasch at acer.edu.au
>> Subject: Re: [Rasch] Rasch analysis of interval data
>>
>>
>>   absolute scale
>>
>>
>>     A Dictionary of Psychology | Date: 2001
>>
>> *absolute scale n.* In statistics and measurement theory, a ratio
>>
> scale
>
>> <http://www.encyclopedia.com/doc/1O87-ratioscale.html> in which the
>>
> unit
>
>> of measurement is fixed. In practice, values on an absolute scale are
>> usually if not always obtained by counting.
>>
>> MLC:  What is counted is the number of times in a hierarchy that a
>> higher order action coordinates two or more lower order actions in a
>> non-arbitrary way.  It has a tree structure so one counts how high up
>>
> in
>
>> the tree one is.
>>
>> MLC
>>
>> Andrew Kyngdon wrote:
>>
>>
>>> "It is absolute in the sense that it does not require norms, it does
>>>
>>>
>> not
>>
>>
>>> need to be tested on people to know what the hierarchical complexity
>>>
>>>
>> of
>>
>>
>>> a task is".
>>>
>>> It does not logically follow from these premises that you can measure
>>>
>>>
>> an
>>
>>
>>> ordinal structure with an "absolute" scale.
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Michael Lamport Commons [mailto:commons at tiac.net]
>>> Sent: Tuesday, July 22, 2008 4:47 PM
>>> To: Andrew Kyngdon
>>> Cc: rasch at acer.edu.au
>>> Subject: Re: [Rasch] Rasch analysis of interval data
>>>
>>> Andrew Kyngdon wrote:
>>>
>>>
>>>
>>>> How can an ordinal structure be measured with an absolute scale?
>>>>
>>>>
>>>>
>>>>
>>> It is absolute in the sense that it does not require norms, it does
>>>
>>>
>> not
>>
>>
>>> need to be tested on people to know what the hierarchical complexity
>>>
>>>
>> of
>>
>>
>>> a task is.
>>>
>>> MLC
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>
>>>> Andrew Kyngdon, PhD
>>>>
>>>> Senior Research Scientist
>>>>
>>>> MetaMetrics, Inc.
>>>>
>>>> 1000 Park Forty Plaza Drive
>>>>
>>>> Durham NC 27713 USA
>>>>
>>>> Tel. 1 919 354 3473
>>>>
>>>> Fax. 1 919 547 3401
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
> ------------------------------------------------------------------------
>
>>
>>
>>>
>>>
>>>
>>>> *From:* Michael Lamport Commons [mailto:commons at tiac.net]
>>>> *Sent:* Tuesday, July 22, 2008 2:04 PM
>>>> *To:* Andrew Kyngdon
>>>> *Cc:* rasch at acer.edu.au
>>>> *Subject:* Re: [Rasch] Rasch analysis of interval data
>>>>
>>>>
>>>>
>>>> The orders of hierarchical complexity are ordinal, universal,
>>>>
>>>>
>> context,
>>
>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>> content, and participant free.  The are an analytic measure of the
>>>> hierarchical complexity of tasks.  We know of 15 orders.  You can
>>>>
> see
>
>>>>
>>>>
>>
>>
>>>> a description on Wikipedia.  There is a non-arbitrary zero.  Rasch
>>>> measures performance.  Hierarchical complexity measures tasks
>>>> properties.  This is psychophysics.  The y-axis is Rasch Score, the
>>>>
> x
>
>>>>
>>>>
>>
>>
>>>> axis is order of hierarhical complexity.  The r's are mostly in the
>>>>
>>>>
>> .9
>>
>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>> -.99 range.
>>>>
>>>> MLC
>>>>
>>>> Andrew Kyngdon wrote:
>>>>
>>>> Michael,
>>>>
>>>> I'm not familiar with your work at all, but I take it by "absolute"
>>>> scale that you mean you have a continuous, quantitative attribute
>>>>
>>>>
>> that
>>
>>
>>>> you can measure with a scale possessing a non-arbitrary zero point?
>>>>
>>>>
>> If
>>
>>
>>>> so, that is quite a feat in the behavioural sciences, given the lack
>>>>
>>>>
>>>>
>>> of
>>>
>>>
>>>
>>>> natural concatenation operations.
>>>>
>>>> But you state that you can transform Rasch scores into this
>>>>
>>>>
>> supposedly
>>
>>
>>>> "absolute" scale of "Stage scores". Now, correct me if I am wrong,
>>>>
>>>>
>> but
>>
>>
>>>> Rasch logits are usually advanced as interval scale measurements
>>>> (leaving aside the obvious problem of a lack of a defined unit,
>>>> something Steve Humphry has been at pains to point out). Interval
>>>>
>>>>
>>>>
>>> scale
>>>
>>>
>>>
>>>> measurements cannot be meaningfully transformed into ratio or
>>>>
>>>>
>> absolute
>>
>>
>>>> scales, unless your substantive theory is sufficiently understood to
>>>> enable this, such as in temperature with converting Celsius and
>>>> Fahrenheit measurements into the "absolute" Kelvin scale. But if you
>>>>
>>>>
>>>>
>>> can
>>>
>>>
>>>
>>>> measure something with an absolute scale in the first place, why
>>>>
>>>>
>> would
>>
>>
>>>> you bother with an interval scale, unless there are historical
>>>>
>>>>
>> reasons
>>
>>
>>>> (as in temperature) for so doing?
>>>>
>>>>
>>>>
>>>> Andrew Kyngdon, PhD
>>>> Senior Research Scientist
>>>> MetaMetrics, Inc.
>>>> 1000 Park Forty Plaza Drive
>>>> Durham NC 27713 USA
>>>> Tel. 1 919 354 3473
>>>> Fax. 1 919 547 3401
>>>>
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Michael Lamport Commons [mailto:commons at tiac.net]
>>>> Sent: Tuesday, July 22, 2008 12:38 PM
>>>> To: Andrew Kyngdon
>>>> Cc: Mark Moulton; Paul Barrett; rasch at acer.edu.au
>>>>
>>>>
>>>>
>>> <mailto:rasch at acer.edu.au>
>>>
>>>
>>>
>>>> Subject: Re: [Rasch] Rasch analysis of interval data
>>>>
>>>> We have an absolute scale in order of hierarchical complexity. We
>>>> transform the Rasch scores into Stage scores which are based on the
>>>> absolute scale.
>>>>
>>>> Michael Lamport Commons
>>>>
>>>> Andrew Kyngdon wrote:
>>>>
>>>>
>>>>
>>>>
>>>>> This is still not a guarantee that the lexile unit captures the
>>>>>
>>>>>
>>>>>
>>> "true"
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> spacing, but fortunately it does not seem to matter.
>>>>>
>>>>> The Lexile unit is best defined unit I have come across in the
>>>>> behavioural sciences, so by what criteria one judges to be a "true
>>>>> spacing" (whatever that is) seems to be a mystery...
>>>>>
>>>>> Andrew Kyngdon, PhD
>>>>>
>>>>> Senior Research Scientist
>>>>>
>>>>> MetaMetrics, Inc.
>>>>>
>>>>> 1000 Park Forty Plaza Drive
>>>>>
>>>>> Durham NC 27713 USA
>>>>>
>>>>> Tel. 1 919 354 3473
>>>>>
>>>>> Fax. 1 919 547 3401
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
> ------------------------------------------------------------------------
>
>>
>>
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>
>>>>> *From:* rasch-bounces at acer.edu.au
>>>>>
> <mailto:rasch-bounces at acer.edu.au>
>
>>>>>
>>>>>
>>>>>
>>> [mailto:rasch-bounces at acer.edu.au]
>>>
>>>
>>>
>>>>> *On Behalf Of *Mark Moulton
>>>>> *Sent:* Monday, July 21, 2008 6:09 PM
>>>>> *To:* Paul Barrett
>>>>> *Cc:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
>>>>> *Subject:* Re: [Rasch] Rasch analysis of interval data
>>>>>
>>>>> Paul,
>>>>>
>>>>> Thank you for your explanations and for your presentation 10 years
>>>>> ago, which are very helpful to me. You raise a fundamental issue,
>>>>> still controversial, still worth visiting in my opinion. You make
>>>>>
>>>>>
>> the
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> case that Rasch measures are equal-interval representations of
>>>>>
>>>>>
>> counts
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> (ratios of counts), and that is all, and that they do not
>>>>>
>>>>>
>> necessarily
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> capture a fundamental unit of measurement in the underlying
>>>>>
>>>>>
>>>>>
>>> construct.
>>>
>>>
>>>
>>>>>
>>>>> I think your point is amplified by a simple desktop experiment. Lay
>>>>>
>>>>>
>> a
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> ruler on a sheet of paper. Draw dots (representing persons, say) in
>>>>>
>
>
>>>>> various distributions on the paper. For each centimeter increment
>>>>>
> on
>
>>>>>
>>>>>
>>
>>
>>>>> the ruler (representing items), count the dots above and below that
>>>>>
>
>
>>>>> increment and calculate their log ratio. One finds that the logit
>>>>> spacings of the ruler increments may be highly unequal (unlike the
>>>>> centimeters), depending on how one distributes the dots. If one
>>>>> distributes the dots equally up and down the ruler, the logit
>>>>>
>>>>>
>> lengths
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> between increments appear to get fatter at the extremes. If one
>>>>>
>>>>>
>>>>>
>>> clumps
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> the dots in multiple modes, the logit lengths can be distorted in
>>>>>
>>>>>
>> all
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> sorts of cool ways. Interestingly, as the distribution approaches
>>>>> normal, the logit lengths between increments seem to approach a
>>>>>
>>>>>
>>>>>
>>> linear
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> relationship with the centimeters, (I don't have a proof for why
>>>>>
>>>>>
>> this
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> would be true, but presumably it has something to do with the
>>>>> relationship between the logistic and normal distributions and may
>>>>> account for the similarity between independently calibrated
>>>>>
> scales).
>
>>>>>
>>>>> So, I agree that Rasch logits do not capture fundamental units of
>>>>> measurement, and are sample-dependent in this sense (and in several
>>>>>
>
>
>>>>> other senses, too). My question is: What does this do to Rasch
>>>>>
>>>>>
>> claims
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> of "invariance," aka "special objectivity," the notion that the
>>>>> relative logit spacings of persons will remain the same regardless
>>>>>
>>>>>
>> of
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> how the items are spaced? Strangely, I don't think it has any
>>>>>
> effect
>
>>>>>
>>>>>
>>
>>
>>>>> at all. The disappearance of the item parameter when calculating
>>>>>
> the
>
>>>>>
>>>>>
>>
>>
>>>>> person parameter, and vice versa, has the same force and
>>>>>
> implication
>
>>>>>
>>>>>
>>
>>
>>>>> that it always did. And due to how Rasch conjointly calculates
>>>>>
>>>>>
>>>>>
>>> persons
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> and items, whatever distortions may occur affect the persons and
>>>>>
>>>>>
>>>>>
>>> items
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> equally.
>>>>>
>>>>> I am left with a relativistic notion of psychometric spaces. Each
>>>>> Rasch analysis erects a unique space. That space bears no
>>>>>
>>>>>
>> /necessary/
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> relationship to any other Rasch space (except perhaps in some
>>>>> topographic one-to-one homeomorphic kind of way). However, objects
>>>>> within that space are distributed in a way that is reproducible,
>>>>>
>>>>>
>>>>>
>>> hence
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> objective, with respect to other objects in that space. Two Rasch
>>>>> spaces can be reconciled only by "anchoring" one space to the other
>>>>>
>
>
>>>>> via common persons or items. This forces the two spaces to share
>>>>>
> the
>
>>>>>
>>>>>
>>
>>
>>>>> same "distortions," and thus to become one space, and to preserve
>>>>> invariance for all objects residing in that space.
>>>>>
>>>>> Your point about the MetaMetrics lexile scale is well-taken. All
>>>>>
>>>>>
>>>>>
>>> texts
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> and readers are forced into a common space anchored on the physical
>>>>>
>
>
>>>>> properties represented by word frequency and sentence length (or
>>>>>
> log
>
>>>>>
>>>>>
>>
>>
>>>>> transformations thereof). This was facilitated by the fact that
>>>>> MetaMetrics discovered and exploited a linear relationship between
>>>>> textual empirical variables and item difficulties. But even without
>>>>>
>
>
>>>>> that relationship, the two types of variables could have been
>>>>>
> forced
>
>>>>>
>>>>>
>>
>>
>>>>> (by a method MetaMetrics did not use, or need to use) into a common
>>>>>
>
>
>>>>> space through an anchoring procedure. This is still not a guarantee
>>>>>
>
>
>>>>> that the lexile unit captures the "true" spacing, but fortunately
>>>>>
> it
>
>>>>>
>>>>>
>>
>>
>>>>> does not seem to matter.
>>>>>
>>>>> One space is as good as another, so long as they are internally
>>>>> consistent. Am I reading this right?
>>>>>
>>>>> Mark H. Moulton
>>>>>
>>>>> Educational Data Systems
>>>>>
>>>>> 2008/7/21 Paul Barrett <pbarrett at hoganassessments.com
>>>>>
>>>>>
>>>>>
>>> <mailto:pbarrett at hoganassessments.com>
>>>
>>>
>>>
>>>>> <mailto:pbarrett at hoganassessments.com>>:
>>>>>
>>>>> *From:* rasch-bounces at acer.edu.au
>>>>>
> <mailto:rasch-bounces at acer.edu.au>
>
>>>>>
>>>>>
>>>>>
>>> <mailto:rasch-bounces at acer.edu.au>
>>>
>>>
>>>
>>>>> [mailto:rasch-bounces at acer.edu.au
>>>>>
>>>>>
>> <mailto:rasch-bounces at acer.edu.au>]
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> *On Behalf Of *Anthony James
>>>>> *Sent:* Wednesday, July 16, 2008 7:19 AM
>>>>> *To:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
>>>>>
>>>>>
>>>>>
>>> <mailto:rasch at acer.edu.au>
>>>
>>>
>>>
>>>>> *Subject:* [Rasch] Rasch analysis of interval data
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Has anyone ever tried to Rasch analyse a variable for which there's
>>>>>
>
>
>>>>> concatenation-based objective measurement? Suppose we make a height
>>>>>
>
>
>>>>> scale with 6 points:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
> ------------------------------------------------------------------------
>
>>
>>
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>
>>>>> *From:* rasch-bounces at acer.edu.au
>>>>>
> <mailto:rasch-bounces at acer.edu.au>
>
>>>>>
>>>>>
>>>>>
>>> <mailto:rasch-bounces at acer.edu.au>
>>>
>>>
>>>
>>>>> [mailto:rasch-bounces at acer.edu.au
>>>>>
>>>>>
>> <mailto:rasch-bounces at acer.edu.au>]
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> *On Behalf Of *Andrew Kyngdon
>>>>> *Sent:* Wednesday, July 16, 2008 8:13 AM
>>>>>
>>>>>
>>>>> *To:* rasch at acer.edu.au <mailto:rasch at acer.edu.au>
>>>>>
>>>>>
>>>>>
>>> <mailto:rasch at acer.edu.au>
>>>
>>>
>>>
>>>>>
>>>>> *Subject:* RE: [Rasch] Rasch analysis of interval data
>>>>>
>>>>> I think Paul Barrett did something like this once...
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
> ------------------------------------------------------------------------
>
>>
>>
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Yep - 10 years ago to be exact!
>>>>>
>>>>> Sorry I haven't replied until now ...
>>>>>
>>>>> The presentation about the simulation can be downloaded at:
>>>>>
>>>>> http://www.pbarrett.net/presentations/BPS-rasch_98.pdf
>>>>>
>>>>> From my web-page abstract ...
>>>>>
>>>>> **Beyond Psychometrics: the recovery of a standard unit of
>>>>>
> length**:
>
>>>>>
>>>>>
>>
>>
>>>>> This 50-slide presentation was given at the British Psychological
>>>>> Society's Division of Occupational Psychology conference:
>>>>>
> Assessment
>
>>>>>
>>>>>
>>
>>
>>>>> in the Millennium: Beyond Psychometrics, November 1998, at Birkbeck
>>>>>
>
>
>>>>> (University of London). The theme of this presentation was about
>>>>>
>>>>>
>>>>>
>>> Rasch
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> scaling, and its capacity to construct a standard unit from
>>>>> observational data. This presentation contained a data simulation
>>>>>
>>>>>
>>>>>
>>> that
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> attempted to hide a true quantitatively structured latent variable
>>>>>
>>>>>
>> of
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> length behind some poor ordinal observations. All the Rasch scaling
>>>>>
>
>
>>>>> did was to construct an equal-interval latent variable of ordinal
>>>>> lengths! This simulation was heavily criticised Ben Wright and
>>>>>
>>>>>
>>>>>
>>> others,
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> and I have included these criticisms as an addendum to the
>>>>> presentation - along with my reply. However, recent papers seem to
>>>>> have vindicated my conclusions in some respects.....The reality is
>>>>> that these methods simply construct linear latent variables in
>>>>> complete isolation of any empirical evidence that such variables
>>>>>
>>>>>
>>>>>
>>> might
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> indeed be quantitatively structured.. In my opinion, from a
>>>>>
>>>>>
>>>>>
>>> scientific
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> perspective, these scaling methods are frankly of little utility,
>>>>>
>>>>>
>> but
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> they are ingenious from a psychometric perspective and do have
>>>>>
> great
>
>>>>>
>>>>>
>>
>>
>>>>> utility in a more pragmatic sense. It all comes down to what the
>>>>> purpose is for using such scaling, science or number scaling.
>>>>>
>>>>> 10 years on - with some better understanding of things (!) - the
>>>>>
>>>>>
>> goal
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> and conclusions of the presentation still make sense - but now I
>>>>>
>>>>>
>>>>>
>>> fully
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> understand why. Rasch scaling cannot "uncover" a linear latent
>>>>> variable from ordinal measures. It simply scales counts and in
>>>>>
>>>>>
>>>>>
>>> effect,
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> the numbers applied to its algorithms, without regard to whether
>>>>>
>>>>>
>>>>>
>>> those
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> counts or numbers are drawn from an ordinal or linear scale.
>>>>>
>>>>> The mistake made by many psychologists is to forget that latent
>>>>> variable theory implies nothing about the measurement properties of
>>>>>
>
>
>>>>> the variable of interest - latent variables are simply constructed
>>>>> ad-hoc to possess linear properties of measurement. That is not how
>>>>>
>
>
>>>>> normal science proceeds, it is as Michell states a "pathology of
>>>>> science" (2000).
>>>>>
>>>>> I propose that a key exemplar which shows how to properly model
>>>>>
> data
>
>>>>>
>>>>>
>>
>>
>>>>> while invoking a latent variable, is the work done by Metametrics.
>>>>>
>>>>>
>> It
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> is no accident that the initial exploratory work was empirical and
>>>>> based upon much cognitive psychological experimentation, PRIOR to
>>>>>
>>>>>
>> the
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> scaling/modeling exercises. Andrew has already provided excellent
>>>>> explanations of the history of this work, along with another
>>>>> exposition recently in his peer response to Michell's target
>>>>>
> article
>
>>>>>
>>>>>
>>
>>
>>>>> in the journal Measurement (references below).
>>>>>
>>>>> However, if we view edumetrics-psychometrics as largely
>>>>> pragmatic/technical work, which is concerned with the efficiencies
>>>>>
>>>>>
>> to
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> be gained in standards-based testing/examination/cumulative
>>>>>
>>>>>
>>>>>
>>> risk-scale
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> environments, then IRT models in general, and the Rasch model make
>>>>>
> a
>
>>>>>
>>>>>
>>
>>
>>>>> great deal of sense. I think it is an illusion that the Rasch or
>>>>>
> any
>
>>>>>
>>>>>
>>
>>
>>>>> IRT/latent variable model magically produces "fundamental
>>>>>
>>>>>
>>>>>
>>> measurement"
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> in any sense of the word. Michell (2004, and now 2008) has put paid
>>>>>
>>>>>
>>>>>
>>> to
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> this notion.
>>>>>
>>>>> I don't think this is a controversial point anymore - from the
>>>>> standpoint of simple logic, the work by Robert Wood, and from my
>>>>>
> own
>
>>>>>
>>>>>
>>
>>
>>>>> small and almost stupid simulation, the Rasch model cannot possibly
>>>>>
>
>
>>>>> "uncover/discover" the true metric for a "statistically constructed
>>>>>
>
>
>>>>> latent variable". It just does what it does given the data with
>>>>>
>>>>>
>> which
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> it is presented. Whether or not that data is an accurate
>>>>> representation/set of observations of the phenomenon of interest
>>>>>
> (my
>
>>>>>
>>>>>
>>
>>
>>>>> "bad ruler"), the Rasch scaling will simple create a latent
>>>>>
> variable
>
>>>>>
>>>>>
>>
>>
>>>>> anyway - given sufficient stochastic error in the observations (as
>>>>> with Wood's coin-tosses). Which is why I think the Metametrics
>>>>> exemplar is so very important, the scaling is constructed around a
>>>>> wealth of empirical phenomena and magnitude relationships - and not
>>>>>
>
>
>>>>> just banks of "item responses".
>>>>>
>>>>> Regards ... Paul
>>>>>
>>>>> __________________________________________________
>>>>> Paul Barrett 918.749-0632 x 326
>>>>> Chief Research Scientist Skype: pbar088
>>>>> Hogan Assessment Systems Inc.
>>>>> 2622 East 21st St., Tulsa, OK 74114
>>>>>
>>>>> **References**
>>>>>
>>>>> Kyngdon, A. (2008) Treating the Pathology of Psychometrics: An
>>>>>
>>>>>
>>>>>
>>> Example
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> from the Comprehension of Continuous Prose Text. //Measurement:
>>>>> Interdisciplinary Research & Perspective//, 6, 1 & 2, 108-113.
>>>>>
>>>>> Michell. J. (2000) Normal science, pathological Science, and
>>>>> psychometrics. Theory and Psychology, 10, 5, 639-667.
>>>>>
>>>>> Michell, J. (2004) Item Response Models, pathological science, and
>>>>>
>>>>>
>>>>>
>>> the
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> shape of error. //Theory and Psychology//, 14, 1, 121-129.
>>>>>
>>>>> Michell, J. (2008) Is psychometrics pathological science?
>>>>> //Measurement: Interdisciplinary Research & Perspective//, 6, 1,
>>>>>
>>>>>
>> 7-24
>>
>>
>>>>>
>>>>> Wood, R. (1978) Fitting the rasch model - a heady tale. //British
>>>>> Journal of Mathematical and Statistical Psychology//, 31, , 27-32.
>>>>>
>>>>> **An aside**
>>>>>
>>>>> The journal "Measurement: Interdisciplinary Research and
>>>>> Perspective"published issues two issues simultaneously - three
>>>>>
>>>>>
>> target
>>
>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>> articles and commenatries on the issue:
>>>>>
>>>>> //The Conceptual Foundations of Psychological Measurement//
>>>>>
>>>>> The target papers by Denny Borsboom and Keith Markus are also
>>>>> excellent expositions of their respective positions. Very nice
>>>>> position pieces.
>>>>>
>>>>> I've attached the journal link here so you can look at the paper
>>>>> titles etc.
>>>>>
>>>>> http://www.informaworld.com/smpp/title~content=g794512699~db=all
>>>>>
>>>>>
>>>>>
> <http://www.informaworld.com/smpp/title%7Econtent=g794512699%7Edb=all>
>
>>>
>>>
>>
>>
>>>
>>>
>>>
> <http://www.informaworld.com/smpp/title%7Econtent=g794512699%7Edb=all>
>
>>>
>>>
>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Rasch mailing list
>>>>> Rasch at acer.edu.au <mailto:Rasch at acer.edu.au>
>>>>>
>>>>>
>>>>>
>>> <mailto:Rasch at acer.edu.au>
>>>
>>>
>>>
>>>>> http://mailinglist.acer.edu.au/mailman/listinfo/rasch
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
> ------------------------------------------------------------------------
>
>>
>>
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>
>>>>> _______________________________________________
>>>>> Rasch mailing list
>>>>> Rasch at acer.edu.au <mailto:Rasch at acer.edu.au>
>>>>> http://mailinglist.acer.edu.au/mailman/listinfo/rasch
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>

```