[Rasch] Unidimensionality (A different follow up)

Theo Dawson theo at devtestservice.com
Tue May 13 06:17:23 EST 2008


I would argue however, that density is a well-understood and useful  
construct that is related to the constructs used to measure it in  
clear ways. When we create an average score or sum scores on different  
assessments, we are not making meaning in a similar way. I would argue  
that we are losing meaning, much in the same way we would lose meaning  
if we summed or averaged the measures used in calculating mass---just  
because they all had something to do with the object, and multiple  
observations told us they tended to be correlated with one another.

The deeper meaning of a measurement may not seem so important if all  
we want to know is if someone has "passed", but it means everything if  
we want the assessment to tell us something about what that student  
can benefit from learning next.

Theo

On May 12, 2008, at 3:02 PM, demott at aol.com wrote:

> This is a great discussion, both from a philosophical view and from  
> a practical view.
> Here is what I would say: Of course the Rasch Model is wrong, in  
> that no data will ever fit it perfectly. So are other models. The  
> point is, is Rasch useful. Does it help us in our understanding. The  
> answer is yes, it helps me!
>
> Now here is an interesting paradox:
> In objectives-referenced testing (or standards-referenced testing)  
> we compute how students do on each standard. We may then make some  
> decision on whether thy have "mastered" the skills described. We  
> also look at the total test score, e.g. Mathematics. We report how  
> well the students do on that score domain. We may then say that a  
> student has or has not "passed" the test. For both these purposes we  
> may use the Rasch model (or 3 param IRT, or Clasical Test Theory, or  
> even some Decision Theory model).
>
> We treat the total test score as something meaningful. We treat the  
> score on each objective as something meaningful.
> Clearly the score on objective 1 means something different from the  
> score on objective 2 -- otherwise why would we report them  
> separately? Yet the scores on objectives 1 thru n add up to the  
> total test score -- which we also report! Recall that each subscore  
> can be shown to measure a more or less independent skill. So we are  
> adding apples, oranges, bananas, grapes, plums, pears, etc. to get a  
> "fruit" score. We do so with aplomb (please excuse the pun).
>
> I guess this is akin to the fact that we may measure the height,  
> length, and width of an object to infer its density, given its mass.  
> In this case, the unifying entity is the object itself. I suppose  
> the underlying entity in the testing scenario is the person. It has  
> been noted that score on one reading test closely predicts the score  
> on most other reading tests, for a given individual. In fact, the  
> score on a reading test even predicts the score on a mathematics  
> test rather well. Yet we don't really believe the skills underlying  
> are all the same.
>
> In spite of clear paradoxes, we proceed.
>
> DM
>
>
> David E. W. Mott, Ph.D.
> Tests for Higher Standards &
>     ROSworks LLC -- Reports Online System (ROS)
> 5310 Markel Road, Suite 104
> Richmond, VA 23230-3030USA
> 804.282.3111
> 804.282.4126 fax
> www.rosworks.com
>
> Provide Focus + Facilitate Achievement
> Plan your next roadtrip with MapQuest.com: America's #1 Mapping Site.
> _______________________________________________
> Rasch mailing list
> Rasch at acer.edu.au
> http://mailinglist.acer.edu.au/mailman/listinfo/rasch

_____________________________________________

Take the Sort Task Challenge at:  http://devtestservice.com/sorttasklead.php

Theo Linda Dawson, Ph.D.
President & CEO, Developmental Testing Service, LLC (DTS)
Northampton, MA 01060

http://www.devtestservice.com
http://discotest.org
http://www.lectica.info
theo at devtestservice.com

Cell  413-320-5919
Office  413-303-1484
FAX 413-303-9041

At the Developmental Testing Service, LLC (DTS), we are dedicated to  
developing assessments that capture the transformations in thinking  
that occur up to 13 times over the course of the lifespan. Each of  
these transformations leads to a more complex, integrated, and  
adequate way of thinking. Whereas traditional tests do a good job  
examining what people know, our tests examine what people know AND how  
they think about what they know.

This message contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended  
solely for the use of the addressee(s) named above. Any disclosure,  
distribution, copying or use of the information by others is strictly  
prohibited. If you have received this message in error, please advise  
the sender by immediate reply and delete the original message and any  
and all attachments. Thank you.











-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20080512/2049f3fa/attachment.html 


More information about the Rasch mailing list