[Rasch] Unidimensionality
Mark Moulton
markm at eddata.com
Sat May 17 04:34:10 EST 2008
Michael,
Thank you for the powerpoints relating hierarchical complexity to Rasch
difficulty. They are interesting and have very practical implications,
e.g., automatic machine calibration of test difficulty in the style of
MetaMetrics. One issue I am trying to settle is the degree to which content
is intrinsically hierarchical versus hierarchical by convention. For
instance, calculus is often placed near the top of high school math
hierarchies, yet the most common applications of calculus (e.g., derivatives
of polynomials) are quite simple and could be done by kids in lower grades
-- if they were taught. To this degree, it is a hierarchy of convention.
This distinction between intrinsic and conventional hierarchy would seem to
have real measurement implications for kids exposed to unusual or
alternative curricula. (I remember a posting last year that presented
real-life examples of this situation.) Can we build content hierarchies
that are (reasonably) impervious to differences in curricula?
Mark Moulton
Educational Data Systems
On Wed, May 14, 2008 at 7:02 PM, commons <commons at tiac.net> wrote:
> My best,
>
> Michael Lamport Commons, Ph.D.
> Assistant Clinical Professor
> Department of Psychiatry
> Harvard Medical School
> Beth Israel Deaconess Medical Center
> 234 Huron Avenue
> Cambridge, MA 02138-1328
> commons at tiac.net
> http://www.dareassociation.org/
> 617-497-5270 Telephone
> 617-320-0896 Cellular
> 617-491-5270 Facsimile
>
> ------------------------------
> *From:* rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] *On
> Behalf Of *Mark Moulton
> *Sent:* Wednesday, May 14, 2008 7:50 PM
> *To:* Stephen Humphry
> *Cc:* rasch at acer.edu.au
> *Subject:* Re: [Rasch] Unidimensionality
>
> Dear Steve,
>
> Thanks very much for your reply, and pardon my overlong response. By the
> way, would you be willing to refer me to some of your work on
> psychophysics? Your claim is very important.
>
> My claim is that, as an example, respondents taking a questionnaire getting
> at multiple psychological traits can be modelled in a multidimensional
> space, and that this space has (or can have) all the geometrical properties
> of the three physical dimensions, without being limited to three
> dimensions. A true multidimensional space is only possible if all the
> dimensions are in the same metric, and this is accomplished when the
> responses in the data matrix are all of the same type or metric, such as a
> dichotomous "has the trait" or "doesn't have the trait." Given these
> conditions, I say that any set of persons and items can be modelled in a
> common space, and thus that the data set is reducible to vector quantities,
> though the coordinate system used to describe those vectors is indeed
> arbitrary. If the items are in different metrics, as in many data sets they
> are, they must somehow be reduced to a common metric before they can be
> placed in a true vector space.
>
> The distinction between vector and scalar quantities is not an
> issue, because after reduction by the appropriate model all entities are
> vectors; there are no scalars. My reference to spatial dimensions as
> "different types of quantities" was probably not right; I meant that
> location along a given dimension bears no necessary relationship to location
> along another dimension, and that in this sense they are different
> quantities. I also meant that scalar quantities can, given the right
> normalization methodology (no trivial problem), be reduced to a common
> vector space.
>
> My belief in an objective psychological space that has the properties of
> physical space goes back to Rasch experiments I did in the 1990's wherein I
> used psychological impressions of distance (Very Near, Near, Far, Very Far)
> to map successfully the relative positions of the buildings on the
> University of Chicago campus. I did a variety of experiments in this vein,
> including estimating the weights of cups of water. I reasoned that if Rasch
> measures of mental impressions successfully reproduced actual distances and
> weights and other physical quantities with a reasonable degree of precision,
> then Rasch measures have the same geometrical properties as physical
> measures and are likely to be reproducible in general, i.e., objective, so
> long as the fit requirements are met. I surmised that the "specific
> objectivity" property of Rasch is another way of saying that it locates
> objects in a probabilistic 1-dimensional geometrical space. I found that
> this property can be extended to n-dimensional space. I also found that
> objects do not automatically fit in such a space, though they often do, and
> to the degree they misfit in that space they are not properly "objects" at
> all and their properties are not reproducible. This is exactly what we do
> in Rasch.
>
> While current multidimensional methods (neural networks, Bayes, SVD, MDS,
> FA, Regression) are prone to overfit and other problems due in part to
> ambiguities in setting the number of dimensions and to sample dependencies,
> unlike sample-free Rasch where 1-D is specified, they are sufficiently
> successful to justify the claim that they "work." It seems to me, for
> example, that if a 4-D model optimizes the ability to predict the values of
> missing cells in a given data matrix, and that this is generally true, then
> this constitutes real evidence that the persons and items do indeed reside
> in a 4-dimensional space, and not a 1-dimensional space. The data mining
> industry, e.g., Google, routinely finds that high-dimensional solutions
> optimize the accuracy of such predictions.
>
> It is true that prediction is not the same as explanation, a serious
> limitation from a scientific point of view. But if you are a company trying
> to decide what movie to recommend to a customer based on prior viewing
> habits, you don't need an explanation. Just an accurate psychometric
> prediction, however inscrutable.
>
> Mark
>
>
>
>
>
>
> 2008/5/13 Stephen Humphry <stephen.humphry at uwa.edu.au>:
>
>> Mark.
>>
>> I don't follow why you believe the distinction between spatial dimensions
>> and others is artificial: it gives rise to the distinction between scalar
>> and vector quantities.
>>
>> If the three spatial dimensions are "different types of quantities", why
>> is it that the choice of coordinate system is arbitrary?
>>
>> You say: ''We may or may not succeed in accurately modeling
>> psychological multidimensionality, but genuine psychological prediction is
>> nearly impossible without it. Yet it does occur. Google would collapse
>> without it.''
>>
>> Citing cases in which predictions are made is by no means evidence that
>> prediction is nearly impossible without 'multidimensional models', in the
>> sense of models that posit an actual mental or psychological space analogous
>> to real physical space. A set of experiments in psychophysics I've
>> analysed refutes this claim. In addition, prediction does not imply
>> explanation, and without explanation it is not possible to understand the
>> circumstances under which future prediction will or won't hold.
>>
>> I suspect we'll have to agree to disagree.
>>
>> Steve
>>
>> ------------------------------
>> *From:* rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] *On
>> Behalf Of *Mark Moulton
>> *Sent:* Wednesday, 14 May 2008 6:49 AM
>> *To:* 'Stephen Humphry'; rasch at acer.edu.au
>> *Subject:* RE: [Rasch] Unidimensionality
>>
>> Stephen et. al.,
>>
>>
>>
>> "Physical pace is actually three-dimensional, but nothing else has been
>> shown to be actually multidimensional in this sense..."
>>
>>
>>
>> I take a VERY different position. I claim that:
>>
>>
>>
>> a) There exists an objective, verifiable psychological or mental
>> space that has the same geometric attributes as physical space, except that
>> it is not limited to three dimensions;
>>
>> b) Items and persons can be located in this space as vectors, and
>> data can be modeled as the multiplication of these vectors;
>>
>> c) While MDS and other well-known multidimensional models are not
>> yet "objective" in the Rasch sense, this is a limitation of the
>> methodologies, not of the geometrical paradigm they attempt to realize.
>>
>> d) Psychological data and entities are not automatically
>> multidimensional. They need to be subjected to a measurement model that
>> forces a common multidimensional metric and flags departures from geometric
>> assumptions, in a way precisely analogous to Rasch.
>>
>> e) The distinction between "spatial dimensions" and "dimensions
>> characterized by different types of quantities or units" is artificial. All
>> dimensions are different types of quantities (even the three spatial
>> dimensions) and every different type of quantity that is not a linear
>> combination of other types of quantities can legitimately be called a
>> dimension. A common metric is enforced by requiring all data points, in a
>> multidimensional data set, to be of the same type. Thus, the
>> correct/incorrect distinction can be used to model educational data sets
>> that comprise both Math and Language items that erect a demonstrable
>> two-dimensional space.
>>
>>
>>
>> The reason for my position is theoretic and pragmatic. I developed
>> several successful multidimensional algorithms (e.g., see
>> www.eddata.com/resources ), and they rely completely on the ability to
>> apply the axioms and spatial assumptions of classical geometry to
>> non-physical spaces. Other multidimensional models, e.g., Singular Value
>> Decomposition, do as well. And the key point is they work. They make
>> successful predictions about the world. They are the top algorithms, for
>> instance, in the Netflix contest for predicting movie ratings.
>>
>>
>>
>> Such observations make it impossible for me to credit such statements as,
>> "there is no firm basis for claiming traits are 'multidimensional' in either
>> sense in psychology, education and so on..." We may or may not succeed in
>> accurately modeling psychological multidimensionality, but genuine
>> psychological prediction is nearly impossible without it. Yet it does
>> occur. Google would collapse without it.
>>
>>
>>
>> Mark Moulton
>>
>> Educational Data Systems
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>> *From:* Stephen Humphry [mailto:stephen.humphry at uwa.edu.au]
>> *Sent:* Tuesday, May 13, 2008 12:29 AM
>> *To:* rasch at acer.edu.au
>> *Subject:* RE: [Rasch] Unidimensionality
>>
>>
>>
>>
>>
>> Andrew, thanks for the information about that work on multidimensional
>> scaling, that is interesting.
>>
>>
>>
>> On the second sense of 'multidimensional', which as you rightly say
>> relates to a kind of decomposition of dimensions into others, you mention
>> force and electrical resistance. I would recommend to anyone who is dubious
>> about the relevance of the anlogy in physics to take even a cursory look at
>> the expression of electrical resistance, conductance, pressure and so forth
>> in terms of base SI units. See the last column in Table 3 at the following
>> link http://physics.nist.gov/cuu/Units/units.html
>>
>>
>>
>> Each base unit is a unit of a different dimension; i.e. kind of quantity.
>> Even from a cursoy look it should be evident that dimensions like electrical
>> resistance can be expressed in terms of a number of physical dimensions
>> (length, mass, time etc), each of which has a SI base unit.
>>
>>
>>
>> The expressions of dimensions in terms of others are not literally
>> algebraic, as aptly pointed out by Emerson (2008). It's possible to express
>> a large number (approx 100) of derived units in common use, in terms of just
>> seven base units, because "some physical measures are related to others
>> by laws" (Krantz et al, 1971, p. 455). Somtimes one dimension is related to
>> two others simply by definition, as in the case of density in terms of
>> volume and mass.
>>
>>
>>
>> The nature of the relationships is important, however, whether the
>> relationships are commonly called laws or definitions.
>>
>>
>>
>> As Andrew says (or at least I take him to be saying), without the
>> substantive theories regarding dimensions and their relationships, there is
>> no firm basis for claiming traits are 'multidimensional' in either sense in
>> psychology, education and so on. And as I said, in my view it is not even
>> clear in what sense the term is being used.
>>
>>
>>
>> Steve
>>
>>
>>
>>
>>
>> Dr Stephen Humphry
>>
>> Graduate School of Education
>>
>> University of Western Australia
>>
>> 35 Stirling Highway
>>
>> CRAWLEY WA 6009
>>
>> Mailbox M428
>>
>> P: (08) 6488 7008
>>
>> F: (08) 6488 1052
>>
>>
>>
>> CRICOS Provider Code: 00126G
>>
>>
>> ------------------------------
>>
>> *From:* rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] *On
>> Behalf Of *Andrew Kyngdon
>> *Sent:* Tuesday, 13 May 2008 12:05 AM
>> *To:* rasch at acer.edu.au
>> *Subject:* [Rasch] Unidimensionality
>>
>>
>>
>> To claim that ability or mental attributes are 'multidimensional'
>> potentially confuses the geometric representation of correlations with the
>> idea of an actual mental or psychological "space" in which things and
>> phenomena genuinely exist statically and/or dynamically. Physical pace is
>> actually three-dimensional, but nothing else has been shown to be actually
>> multidimensional in this sense...Otherwise, multidimensional might simply
>> mean test results depend on more than one attribute, each of which could in
>> principle be measured independently, with relationships contingent on other
>> factors. In the latter sense, most dimensions in physics could be called
>> 'multidimensional'. The mass of an object depends on its volume and density,
>> but that doesn't mean it can't be measured.
>>
>>
>>
>> Well done Steve, you hit the nail right on the head here. There exists
>> endemic confusion in the behavioural sciences as to what
>> "multidimensionality" is, and you correctly state that the term can be used
>> to describe different kinds of structures.
>>
>>
>>
>> In the case of multidimensional scaling, psychological attributes are
>> assumed to be multidimensional metric spaces, in that the distance between
>> any two points in multidimensional space is positive, symmetric and
>> satisfies the triangle inequality (Beals, Krantz & Tversky, 1968).
>> Dissimilarities judgments often fail the triangle inequality (Tversky &
>> Gati, 1982) so it is unwise merely to assume that psychological attributes
>> form metric spaces. Also, when the Minkowski R metric (or "power" metric) is
>> used, the properties of interdimensional additivity and intradimensional
>> subtractivity often fail when more than just one dimension is involved
>> (Michell, 1990).
>>
>>
>>
>> In physics and in polynomial conjoint measurement theory more generally,
>> single quantities can be decomposed into many other variables. Examples in
>> physics are density, force, and electrical resistance; and possible examples
>> in psychology are Hull's (1952) and Spence's (1956) theories of response
>> strength. Indeed, most quantities in physics are compositions of other
>> variables, yet they nonetheless are "unidimensional" and measureable.
>> Krantz, Luce, Suppes & Tversky (1971) detail several kinds of composition
>> rules, their attendant cancellation conditions and proofs.
>>
>>
>>
>> However, as you imply, most behavioural scientists aren't this specific in
>> their understanding of "multidimensionality". Most prefer to entertain an
>> informal understanding and for guidance will solely rely on the results of
>> applying a "multidimensional" model to their data. Until they start
>> developing substantive theories of sufficient depth, there will be no firm
>> bases for claiming that this or that psychological system is
>> multidimensional.
>>
>>
>>
>> Cheers,
>>
>>
>>
>> Andrew
>>
>>
>>
>>
>>
>> Andrew Kyngdon, PhD
>>
>> Senior Research Scientist
>>
>> MetaMetrics, Inc.
>>
>> 1000 Park Forty Plaza Drive
>>
>> Durham NC 27713 USA
>>
>> Tel. 1 919 354 3473
>>
>> Fax. 1 919 547 3401
>>
>> *MetaMetrics' 2008 Lexile National Conference & Quantile Symposium**
>> **Successful Teachers, Successful Students**
>> *June 16-19 | San Antonio Marriott Rivercenter
>> www.lexile.com/conference2008
>>
>>
>>
>> _______________________________________________
>> Rasch mailing list
>> Rasch at acer.edu.au
>> http://mailinglist.acer.edu.au/mailman/listinfo/rasch
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20080516/f080c836/attachment.html
More information about the Rasch
mailing list