[Rasch] Models of unidimensional constructs havetheirlimitations:what is a "good thing"?

commons commons at tiac.net
Fri Nov 9 04:34:03 EST 2007


In my view of measurement, we develop multiple unidimensional scales as
unpacked as possible.  For example, I have worked on unpacking difficulty
into Hierarchical complexity and the associated stage of development on a
task, and simple complexity.  As we build up the number of unpack
unidimensional scales, we find relationships among them and performance.
For example, I have been studying discounting.  It consists of three
dimensions:  Value of the reinforcer delivered without delay, delay and
change in delay.  Together they account well for perceived value of
reinforcement.  Perceived value of reinforcement is a linear scale.

 

My best,


Michael Lamport Commons, Ph.D.
Assistant Clinical Professor
Department of Psychiatry
Harvard Medical School
Beth Israel Deaconess Medical Center

234 Huron Avenue

Cambridge, MA 02138-1328
commons at tiac.net
http://www.dareassociation.org/
617-497-5270 Telephone

617-320-0896  Cellular
617-491-5270  Facsimile

  _____  

From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf
Of Lang, William Steve
Sent: Thursday, November 08, 2007 6:05 AM
To: Velozo,Craig A; Agustin Tristan; -Rasch
Subject: RE: [Rasch] Models of unidimensional constructs
havetheirlimitations:what is a "good thing"?

 

Prediction might occur with each dimension measured in some way even if it
lacks quality in the data.  If we really want units defined at an interval
level and useful scales along all the variables (more like the physics
people would do), we need to use Rasch for each "dimension".  One of the
challenges of Rasch is to continue the work of building each set of
construct measures that are additive (objective, valid).  I believe that
some models (2pl, 3pl) and analyses (SEQ, Value-added) may be useful for
prediction, classification (discriminant analysis, etc.), but short cuts are
ultimately avoiding the parallel of physics that defines every unit scales
according to the constructs so that we all can "agree" to universal
measures.


-----Original Message-----
From: Velozo,Craig A [mailto:cvelozo at PHHP.UFL.EDU]
Sent: Thu 11/8/2007 4:25 AM
To: Agustin Tristan; Lang, William Steve; -Rasch
Subject: RE: [Rasch] Models of unidimensional constructs
havetheirlimitations: what is a "good thing"?

I do not think the solution is to "avoid measurement".  Again, this deserves
a design solution, not a measurement solution.  Using multiple
"unidimensional" measures and using a structural equation model, you not
only should be able to predict the best runner, you should be able to
determine which of the variables are most critical to becoming an Olympic
runner.



Craig



________________________________

From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf
Of Agustin Tristan
Sent: Wednesday, November 07, 2007 10:36 PM
To: Lang, William Steve; -Rasch
Subject: RE: [Rasch] Models of unidimensional constructs
havetheirlimitations: what is a "good thing"?



Hello!

In addition to this problem of one or several dimensions, the discussion
with teachers is that "we" (loving measure people) see things is a very
simplified way, while "they" say that education and learning is frankly
multidimensional; so we are reducing things to a single dimension. I agree
with this (I'm teacher too, as many of us), but evidence is very clear that
we are trying tp measuring something like blood pressure or glucose because
it is a "simple" evidence  of a very complicated situation and functioning
of the body.

Now, let's think on a 100 meter male runner in Olympics. Does he only train
running several hours a day? Of course not! Besides running, he has to go to
the gym, strength his arms, torax and other parts of his body, he also has
to follow instructions of the trainer regarding breath, mental attitude,
motivation and other psychological aspects; he has to check his heart rate,
he has to control his food, to eat this or that according to a plan, to be
at the correct weight, and N other things. BUT the ONLY evidence we finally
have is one 100 meter race! It is probably not fair: many other runners eat
correctly what they need...others are very motivated...others have a very
good attitude...but the race lasts less than 10 seconds and we cannot see
all the effort he has to do to be the fastest man on earth, how many months
he has to prepare this 9.8 seconds race. Olympics are not fair then, as a
race doesn't consider the "process", only the "result" or the "product".

Football, artistic skate, even bull-fight are not fair, we judge a single
dimension during the match (the test). Health checking is not fair either: a
high level of glucose in my blood is the indicator that I'm diabetic?

Is it a limited way of thinking? Certainly it is, but it is very complicated
to define the purpose of our measurement, to arrive to measure one
dimension, so how can we try to measure something that is many times more
complicated? The only solution is to avoid measurement: let's ask all the
runners to come for an interview, they will tell how well they train, how
well motivated they are, the food they eat, etc, except to participate in a
100 meter race, and  I invite judges to use this information to define which
runner will have the gold medal in the next Olympics, for a 100 meter
multidimensional race. Am I simplifying too much?

Hope this helps.

Agustin Tristan

"Lang, William Steve" <WSLang at tempest.coedu.usf.edu> wrote:

        "It seems to me that Rasch modelling is working towards being able
to use this kind of multi-dimensional thinking, but isn't quite ready for it
yet. We have to be able to reliably build a precision, high-resolution
measure (ruler) in one dimension (length), and be able to independently
confirm the scale, and even perhaps the units, against a commonly agreed
"standard" (like the standard Metre) before we can start constructing
multi-dimensional construct spaces using orthogonal single dimensions. Do we
have a "standard ability unit" locked away in a vault somewhere, against
which our measures of ability can be compared?" (quote from Rod)
       
        This part of your comment provides me with some possible insight,
thanks:
       
        Isn't the locked away "meter" or "kilogram" an arbitrary but
constant unit?  Someone else could use a "yard" or "pound" as long as they
used defined units consistently.  That is pretty much what Lexiles are to
reading...you are correct that all disciplines haven't agree on the units,
but that is a philosophical decision as much as a scaling problem.  If the
construct is defined (scaled), the units are a simple transformation.
       
        I'd be at a loss to discuss string theory, but we measure blood
pressure as one dimension even though we know that many variables contribute
to the results.  We then use blood pressure as one measure of "health" when
added to other variables such as blood sugar, weight, etc.  I think that if
we tackle development of the scales that are definable and useful one at a
time, we can search out the underlying variables and additive constructs
next.  Our "well-constructed" measure of blood pressure is an intermediate
starting point.  Any of our measures may be composites of more than one
variable and still be useful as long as they function along a linear
dimension.  The future determination of the separate underlying variables
may remain unknown for now.
       
        If we try to understand all the variables to create a "super
measure" that accounts for everything to start with, we will never visualize
a useful instrument.  That is like a "Healthometer" that gives one value for
all variables.  It is difficult to visualize or diagnose all the elements at
once, yet we often hear social scientists attempt to do that.
       
        It seems most important to me that we avoid creating instruments
based on tools that are not interval level (2PL and 3PL for example), or we
will never define the "locked away units" accurately so that we can create
the scales!  Mixing questionable instrument development (2PL and 3PL) with
attempts to explain "all the variables" at all dimensions really confounds
the problem and we are stuck in quicksand.  This seems to be a common
occurrence.
       
        It seems to me that Rasch provides a tool to scale defined units.
The difficult restriction of Rasch is actually necessary for each of the
measures to ultimately be useful as part of the "system".  It also seems
that there are some efforts to develop multi-dimensional extensions (see
Multivariate and Mixture Distribution Rasch Models, Davier & Carstensen),
but the tendency to define everything at once confuses me.  I'm happy with
one well-defined construct at a time that is useful for a given purpose.
Rod is right though, eventually some complex systems will have to be
explained.
       
        Simplistically for me, fit analysis leads to explaining the
underlying variables.  FACETS might be one way to "add" dimensions across
scales.  Scaling a set of assessments that are linked by a taxonomy or
framework is another non-mathematical way to visualize "dimensions" in
social science.
       
        Steve Lang
        University of South Florida St. Petersburg
       
       
        -----Original Message-----
        From: Rodney Staples [mailto:rodstaples at ozemail.com.au]
        Sent: Tue 11/6/2007 7:46 PM
        To: Lang, William Steve; Rod O'Connor; Trevor Bond;
rasch at acer.edu.au
        Cc: jwilkers at fgcu.edu
        Subject: RE: [Rasch] Models of unidimensional constructs have
theirlimitations: what is a "good thing"?
       
        Hi Folks,
        Thank you all for you most interesting and informative discussions
over the
        past few weeks. Please keep up the kinds of discussion, even if the
initial
        question seems "dumb", because to relative "newbies" like me it is
really
        useful.
       
        Now its time for me to add my "three-penneth" to the discussions!
       
        In the physical sciences we have no problem talking about the three
        dimensions of physical space, or about the four dimensions of
space-time, or
        the 11? Dimensions currently associated with string theory. I have
no
        problem imagining a circumstance in complex human behaviour that
needs more
        than a single dimension to describe. But in the physical sciences we
use
        measurements in the physical space, one dimension at a time, to
create areas
        and volumes. or tesserects!
       
        It seems to me that Rasch modelling is working towards being able to
use
        this kind of multi-dimensional thinking, but isn't quite ready for
it yet.
        We have to be able to reliably build a precision, high-resolution
measure
        (ruler) in one dimension (length), and be able to independently
confirm the
        scale, and even perhaps the units, against a commonly agreed
"standard"
        (like the standard Metre) before we can start constructing
multi-dimensional
        construct spaces using orthogonal single dimensions. Do we have a
"standard
        ability unit" locked away in a vault somewhere, against which our
measures
        of ability can be compared?
       
        I'm happily working on trying to build a uni-dimensional construct
around
        ability in a context, and that is more than enough challenge for me
at the
        moment. it is the reason why I find your recent discussion so
relevant and
        timely. But I can see a time, not so far into the future, when I
will need
        to consider more than one dimension.
       
        For example, we can build a single construct around ability as a
        technologist. But when a technologist also must perform as a
manager, then
        the construct defining the role as a manager will be quite different
from
        that describing the underlying technology latent trait (although
there may
        be some generalised abilities that are common. beyond my capacity to
deal
        with at this stage). Thus to define an engineering manager, we need
to have
        a two dimensional construct: an engineering dimension; and a
management
        dimension. In another case more closely related to education, we
already
        have multi-dimensional assessment when we have a theory component to
        assessment (an exam), and a practical component (laboratory work,
etc).
       
        Thanks again for the discussion
        Regards,
        Rod
       
       
 
___________________________________________________________________________
        Dr. Rodney Staples.
        e-mail: rodstaples at ozemail.com.au
        Telephone: +61 3 9770 2484
        Mobile: +61 4 1935 9082
        Web: http://members.ozemail.com.au/~rodstaples/
       
       
        -----Original Message-----
        From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au]On
Behalf
        Of Lang, William Steve
        Sent: Tuesday, 6 November 2007 11:42 PM
        To: Rod O'Connor; Trevor Bond; rasch at acer.edu.au
        Cc: jwilkers at fgcu.edu
        Subject: RE: [Rasch] Models of unidimensional constructs have
        theirlimitations: what is a "good thing"?
       
        I believe this conversation is on track!  Just as "reading
comprehension"
        was bogged down in a complex search for "variables" when Lou Bashaw
and Jack
        Stenner envisioned a usable measure that spurred a new wave of
research,
        Rasch "forces" the scale to understand and map the relevant
construct at a
        targeted level.
       
        Judy Wilkerson and I also witnessed an endless and unresolvable
debate about
        the complexities of "teacher dispositions" based on morals, ethics,
and
        personality that resulted in useless attempts to measure the
construct.
        (See "Teacher Dispositions: Building a Teacher Education Framework
of Moral
        Standards", "Teacher Dispositions: Envisioning Their Role in
Education", and
        "Dispostions in Teacher Education" - all recent books.)  Decades of
debate
        could not deconstruct the variables.  The solution was to validate
        job-related and predictive measures based on a defined and mapped
construct
        that doesn't attempt to sub-divide all the possible variables. (I
hope that
        Lou realizes that I was listening in that class!)
       
        Now that the disposition scales are becoming available, appropriate
research
        and discussion make sense.  Without the measures, you have an
endless
        "factor analysis nightmare" based on any individual's idea du jour.
Until a
        useful measure at some useful dimensional level starts the process,
it is
        unlikely that a synthesis of "stuff" will magically become useful.
I think
        this appears to happen in personality, theories of creativity,
theories of
        intelligence, and likely some of the health care measures. (Does
anyone
        remember the Functional Independence Measure discussion on
thresholds in
        rmt?)
       
        When the "model", such as 3PL, allows the measures to "play around"
with the
        parameters so that there is a false fit, you are off on the
invalidity
        slippery slope.  Rasch seems (to me) to force the measures to
explain the
        construct AT THE LEVEL OF INTEREST.  Once it is working, there are
plenty of
        opportunities (FACETS, etc.) to break down and aggregate the real
        constituent parts!
       
        More importantly to those of us who are looking at "complex human
        variables", it is the DPF analysis that seems to inform the
construct
        interpretation in ways that simply are not envisioned in the
original scale
        development!  If you look at "Item Response Theory" (Baker & Kim) or
"Item
        Response Theory for Psychologists" (Embretson & Reise), you hardly
see any
        mention of keyforms, kidmaps, or construct maps that are familiar to
us
        Rasch folks.  I don't think you can EVER VALIDATE (or interpret) the
        construct or sub-constructs of these complex variables without the
person
        profiles which are completely confounded in 2PL and 3PL models!
       
        Judy and I will repeat, "Rasch puts the people in assessment".  If
you can't
        develop the valid unidimensional constructs at some useful level
(especially
        complex human ones), then you'll likely never have the research
tools that
        encourage one to break out of the tail-chasing debates and identify
the
        "real variables" that actually exist in the data.
       
        This is entirely applied and practical, regardless of the
mathematical logic
        of sufficiency, specific objectivity, and independence even though
I'm glad
        that others more talented than me have provided that logic.
       
        Steve Lang
        University of South Florida St. Petersburg
       
       
        -----Original Message-----
        From: rasch-bounces at acer.edu.au on behalf of Rod O'Connor
        Sent: Tue 11/6/2007 12:25 AM
        To: Trevor Bond; rasch at acer.edu.au
        Subject: Re: [Rasch] Models of unidimensional constructs have their
        limitations
       
        Hello Trevor, very nice to hear from you (I recall your earlier
        assistance...).
       
        I have no doubt one could fit global HQoL data to a single Rasch
scale
        (any notion can be/is one construct, and multiple items can
generally be
        identified), however I fear that people might think this is
        automatically a good thing and that having done so the healthcare
        researcher has 'done their job'.  Unfortunately in principle such a
        scale might not be particularly helpful in 'improving healthcare',
        potentially offering little more than a 'do you feel better'
question
        (except the scale would have more than one item).  My view is that a
        HQoL measure useful for healthcare improvement needs to not only
provide
        an indication of extent but also play a diagnostic role (allow
        'formative' as well as 'summative' evaluation), identifying where
and
        how a change has occurred, and hence where treatment could usefully
be
        targeted.
       
        Of course it all depends on the criteria by which one would assess
        success for the measure, which should always be set-up first.
        Personally I think an overtly multi-dimensional assessment is likely
to
        be needed, with an accompanying complex scoring/combination rule to
        predict values consistent with patient judgement.
       
        Warm regards,
        Rod
       
       
        _Trevor Bond said the following on 6/11/2007 3:34 PM:_
        > mea culpa...denied not denied (sorry Dr Freud...it was a slip)
        >> Dear Rod
        >> HRQoL is one of the areas where Rasch is having a major impact,
if I
        >> can judge by the articles I am asked to review.
        >> Are multidimensional states a composite of unidimensional states?
        >> Why would we ever expect that any human condition could be
DEFINED by
        >> one dimension?
        >> thanks for your prompt!
        >> TGB
        >
        _______________________________________________
        Rasch mailing list
        Rasch at acer.edu.au
        http://mailinglist.acer.edu.au/mailman/listinfo/rasch
       
        No virus found in this outgoing message.
        Checked by AVG Free Edition.
        Version: 7.5.503 / Virus Database: 269.15.22/1112 - Release Date:
5/11/2007
        7:11 PM

        _______________________________________________
        Rasch mailing list
        Rasch at acer.edu.au
        http://mailinglist.acer.edu.au/mailman/listinfo/rasch





FAMILIA DE PROGRAMAS KALT.

Mariano Jiménez 1830 A

Col. Balcones del Valle

78280, San Luis Potosí, S.L.P. México

TEL (52) 44-4820 37 88, 44-4820 04 31

FAX (52) 44-4815 48 48

web page (in Spanish AND ENGLISH): http://www.ieesa-kalt.com

 __________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20071108/15874e20/attachment.html 


More information about the Rasch mailing list