[Rasch] Rasch and Factor Analysis questions
Stephen Humphry
stephen.humphry at uwa.edu.au
Fri Mar 6 12:43:01 EST 2009
If Rasch / IRT indicates that items with skill code X are more difficult on
average than items of code Y, can I infer that the resulting logit pattern
(ability estimates) arrays the skills (and not just the items) along a
difficulty continuum?
S: Yes, and the difficulty of the skills corresponds with the ability at
which persons have a 0.5 chance of demonstrating the skills. Students with
locations higher than the item location are more likely (>0.5) to
demonstrate the skills and vice versa. This is all provided acceptable fit
to the model.
If the items are supposedly testing discrete skills, does this imply
multi-dimensionality of the entire listening data set taken together? If
so, does this violate unidimensionality assumptions for Rasch / IRT? In
other words, if I can show that the items test different skills (latent
traits), can I still use Rasch / IRT on the aggregate data set?
S: By "multi-dimensionality of the ... data", I take you to mean that item
responses are sensitive to more than one trait or attribute that might, in
principle, be measured separately. (For the sake of clarity, it is worth
pointing out that the question is not whether the actual data are
"multidimensional" if you seek to infer measurements of traits from that
data). If so, the answer to the question is that the items may be sensitive
to more than one trait but still measure a single trait, for two reasons.
First, sensitivity of item responses to more than one trait only becomes a
potential violation if it results in violations of local independence. Even
then, these can be dealt with under some conditions (David Andrich and Svend
Kreiner could tell you more about this). If item responses are sensitive to
other traits, but not in a systematic way, it is not a problem in principle
-- it is is one of the reasons responses are probabilistic rather than
deterministic with respect to the trait you want to measure.
S: I assume you are aware that you can conduct principal components analysis
of residuals after applying the Rasch model?
I guess my questions come down to:
What information can Rasch / IRT provide beyond estimates of the ability of
the items?
Is factor analysis appropriate in this situation?
If factor analysis shows two skills to load similarly on the factors, can
this be taken as evidence that they are likely testing the same thing?
S: We can put that second question a little differently. If factor analysis
"shows" two skills load differently on a factor, does that mean they are
testing different things? The answer is no: it may just be an artefact of
the difficulty of the items, or myriad other things. Also, it is possible to
have a perfect correlation between two sets of measurements yet for those
measurements to be of different kinds of quantities (dimensions). Lastly,
scores are not measurement.
If two sets of measurements (i) are genuinely measurements and (ii) have a
low correlation, then they must be different kinds of quantities
(dimensions). On the other hand, though, the fact two sets of measurements
has a high (even perfect) correlation is not sufficient to demonstrate there
is only one kind of quantity (kind of dimension). It's theoretically
instructive to consider whether it is conceivable that levels of two
properties could be uncorrelated among any group of individuals under any
conditions.
The point is that it's an experimental question whether there are conditions
under which two things are not correlated. In my view, a lot of people try
to use correlations as a substitute for experiments that are required to
understand relationships between quantitative properties. Any attempt to use
Factor Analysis to establish whether one or more traits are measured falls
in this category of approach.
I'll let Thurstone say it in his own worrds: "When a problem is so involved
that no rational formulation is available, then some quantification is stil
possible by the calculation of coefficients of correlation or contingency
and the like. But such statistical procedures consitute an acknowledgement
of failure to rationalize the problem and to establish the functions that
underlie the data" (Thurstone, 1959, p. 267). Lastly, O.D Duncan summarised
it nicely in Notes on Social Measurement: ''...multiple factor analysis is
not really a method of measurement at all; it is a method of obtaining a
minimum estaimate of how many distinct hypothetical variables, operating as
common causes of a set of observed variables, are required to account for
the intercorrelations of the latter in some population. Whether any of of
these hypothetical variables, or an estiamte of scores on it derived from
some selection of items, is a measure of a construct simply cannot be
determined by the method." (Duncan, 1984, p. 210).
What statistic would I use to indicate the strength of the relationship
between two (or more) skills?
If an item is shown to load on a general factor, but also to load heavily on
additional factors, does this imply that the item is multi-dimensional? If
so, what implications does this have under Rasch or IRT ?
S: You can obtain disattentuated correlations between person estimates from
two or more item sets that you think measure the skills.
Regards,
Steve
Dr Stephen Humphry
Graduate School of Education
University of Western Australia
35 Stirling Highway
CRAWLEY WA 6009
Mailbox M428
P: (08) 6488 7008
F: (08) 6488 1052
CRICOS Provider Code: 00126G
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20090306/c294bf7c/attachment.html
More information about the Rasch
mailing list