[Rasch] Models of unidimensional constructs havetheirlimitations: what is a "good thing"?

commons commons at tiac.net
Wed Nov 7 12:03:57 EST 2007


For at least one dimension of ability we do.  The hierarchical complexity of
a task is the major contribution to task difficult.  When we make up a
sequence of tasks  differing in their order of hierarchical complexity, that
order predicts with an r of from about .85 to .99 Rasch scores.  There are
other sources of difficulty.  So in that sense we measure stage as a form of
ability extremely well and understand what the scale is and the absolute
units.

 

My best,


Michael Lamport Commons, Ph.D.
Assistant Clinical Professor
Department of Psychiatry
Harvard Medical School
Beth Israel Deaconess Medical Center

234 Huron Avenue

Cambridge, MA 02138-1328
commons at tiac.net
http://www.dareassociation.org/
617-497-5270 Telephone

617-320-0896  Cellular
617-491-5270  Facsimile

  _____  

From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf
Of Rodney Staples
Sent: Tuesday, November 06, 2007 7:46 PM
To: Lang, William Steve; Rod O'Connor; Trevor Bond; rasch at acer.edu.au
Cc: jwilkers at fgcu.edu
Subject: RE: [Rasch] Models of unidimensional constructs
havetheirlimitations: what is a "good thing"?

 

Hi Folks,

Thank you all for you most interesting and informative discussions over the
past few weeks. Please keep up the kinds of discussion, even if the initial
question seems "dumb", because to relative "newbies" like me it is really
useful.

 

Now its time for me to add my "three-penneth" to the discussions! 

 

In the physical sciences we have no problem talking about the three
dimensions of physical space, or about the four dimensions of space-time, or
the 11? Dimensions currently associated with string theory. I have no
problem imagining a circumstance in complex human behaviour that needs more
than a single dimension to describe. But in the physical sciences we use
measurements in the physical space, one dimension at a time, to create areas
and volumes. or tesserects!

 

It seems to me that Rasch modelling is working towards being able to use
this kind of multi-dimensional thinking, but isn't quite ready for it yet.
We have to be able to reliably build a precision, high-resolution measure
(ruler) in one dimension (length), and be able to independently confirm the
scale, and even perhaps the units, against a commonly agreed "standard"
(like the standard Metre) before we can start constructing multi-dimensional
construct spaces using orthogonal single dimensions. Do we have a "standard
ability unit" locked away in a vault somewhere, against which our measures
of ability can be compared?

 

I'm happily working on trying to build a uni-dimensional construct around
ability in a context, and that is more than enough challenge for me at the
moment. it is the reason why I find your recent discussion so relevant and
timely. But I can see a time, not so far into the future, when I will need
to consider more than one dimension. 

 

For example, we can build a single construct around ability as a
technologist. But when a technologist also must perform as a manager, then
the construct defining the role as a manager will be quite different from
that describing the underlying technology latent trait (although there may
be some generalised abilities that are common. beyond my capacity to deal
with at this stage). Thus to define an engineering manager, we need to have
a two dimensional construct: an engineering dimension; and a management
dimension. In another case more closely related to education, we already
have multi-dimensional assessment when we have a theory component to
assessment (an exam), and a practical component (laboratory work, etc). 

 

Thanks again for the discussion

Regards,

Rod

 

 

___________________________________________________________________________

Dr. Rodney Staples.

e-mail: rodstaples at ozemail.com.au

Telephone: +61 3 9770 2484

Mobile: +61 4 1935 9082

Web: http://members.ozemail.com.au/~rodstaples/

 

 

-----Original Message-----
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au]On Behalf
Of Lang, William Steve
Sent: Tuesday, 6 November 2007 11:42 PM
To: Rod O'Connor; Trevor Bond; rasch at acer.edu.au
Cc: jwilkers at fgcu.edu
Subject: RE: [Rasch] Models of unidimensional constructs have
theirlimitations: what is a "good thing"?

 

I believe this conversation is on track!  Just as "reading comprehension"
was bogged down in a complex search for "variables" when Lou Bashaw and Jack
Stenner envisioned a usable measure that spurred a new wave of research,
Rasch "forces" the scale to understand and map the relevant construct at a
targeted level.

Judy Wilkerson and I also witnessed an endless and unresolvable debate about
the complexities of "teacher dispositions" based on morals, ethics, and
personality that resulted in useless attempts to measure the construct.
(See "Teacher Dispositions: Building a Teacher Education Framework of Moral
Standards", "Teacher Dispositions: Envisioning Their Role in Education", and
"Dispostions in Teacher Education" - all recent books.)  Decades of debate
could not deconstruct the variables.  The solution was to validate
job-related and predictive measures based on a defined and mapped construct
that doesn't attempt to sub-divide all the possible variables. (I hope that
Lou realizes that I was listening in that class!)

Now that the disposition scales are becoming available, appropriate research
and discussion make sense.  Without the measures, you have an endless
"factor analysis nightmare" based on any individual's idea du jour.  Until a
useful measure at some useful dimensional level starts the process, it is
unlikely that a synthesis of "stuff" will magically become useful.  I think
this appears to happen in personality, theories of creativity, theories of
intelligence, and likely some of the health care measures. (Does anyone
remember the Functional Independence Measure discussion on thresholds in
rmt?)

When the "model", such as 3PL, allows the measures to "play around" with the
parameters so that there is a false fit, you are off on the invalidity
slippery slope.  Rasch seems (to me) to force the measures to explain the
construct AT THE LEVEL OF INTEREST.  Once it is working, there are plenty of
opportunities (FACETS, etc.) to break down and aggregate the real
constituent parts!

More importantly to those of us who are looking at "complex human
variables", it is the DPF analysis that seems to inform the construct
interpretation in ways that simply are not envisioned in the original scale
development!  If you look at "Item Response Theory" (Baker & Kim) or "Item
Response Theory for Psychologists" (Embretson & Reise), you hardly see any
mention of keyforms, kidmaps, or construct maps that are familiar to us
Rasch folks.  I don't think you can EVER VALIDATE (or interpret) the
construct or sub-constructs of these complex variables without the person
profiles which are completely confounded in 2PL and 3PL models!

Judy and I will repeat, "Rasch puts the people in assessment".  If you can't
develop the valid unidimensional constructs at some useful level (especially
complex human ones), then you'll likely never have the research tools that
encourage one to break out of the tail-chasing debates and identify the
"real variables" that actually exist in the data.

This is entirely applied and practical, regardless of the mathematical logic
of sufficiency, specific objectivity, and independence even though I'm glad
that others more talented than me have provided that logic.

Steve Lang
University of South Florida St. Petersburg


-----Original Message-----
From: rasch-bounces at acer.edu.au on behalf of Rod O'Connor
Sent: Tue 11/6/2007 12:25 AM
To: Trevor Bond; rasch at acer.edu.au
Subject: Re: [Rasch] Models of unidimensional constructs have their
limitations

Hello Trevor, very nice to hear from you (I recall your earlier
assistance...).

I have no doubt one could fit global HQoL data to a single Rasch scale
(any notion can be/is one construct, and multiple items can generally be
identified), however I fear that people might think this is
automatically a good thing and that having done so the healthcare
researcher has 'done their job'.  Unfortunately in principle such a
scale might not be particularly helpful in 'improving healthcare',
potentially offering little more than a 'do you feel better' question
(except the scale would have more than one item).  My view is that a
HQoL measure useful for healthcare improvement needs to not only provide
an indication of extent but also play a diagnostic role (allow
'formative' as well as 'summative' evaluation), identifying where and
how a change has occurred, and hence where treatment could usefully be
targeted.

Of course it all depends on the criteria by which one would assess
success for the measure, which should always be set-up first. 
Personally I think an overtly multi-dimensional assessment is likely to
be needed, with an accompanying complex scoring/combination rule to
predict values consistent with patient judgement.

Warm regards,
Rod


_Trevor Bond said the following on 6/11/2007 3:34 PM:_
> mea culpa...denied not denied (sorry Dr Freud...it was a slip)
>> Dear Rod
>> HRQoL is one of the areas where Rasch is having a major impact, if I
>> can judge by the articles I am asked to review.
>> Are multidimensional states a composite of unidimensional states?
>> Why would we ever expect that any human condition could be DEFINED by
>> one dimension?
>> thanks for your prompt!
>> TGB
>
_______________________________________________
Rasch mailing list
Rasch at acer.edu.au
http://mailinglist.acer.edu.au/mailman/listinfo/rasch

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20071106/a4923e64/attachment.html 


More information about the Rasch mailing list