[Rasch] Noise in observations ...

Paul Barrett pbarrett at hoganassessments.com
Mon Nov 5 08:24:39 EST 2007


	From: Denny Borsboom [mailto:d.borsboom at uva.nl] 
	Sent: Monday, November 05, 2007 9:28 AM
	To: Paul Barrett
	Subject: Re: [Rasch] Noise in observations ...
	Hi Paul, 
	Thanks for your interesting post. I'm wondering why you would
use sumscores and not, say, product scores or any other of the millions
of alternative ways of aggregating items? 

Hello Denny - funny you should ask that .. 
The answer has to be that intuitively, a simple sumscore is the most
basic way of representing something as crude as "more of this = more or
less of that" - with no more thought than this applied to such an
approach. It really is a kind of "basic" thinking that I'm sure is so
intuitive as it requires nothing more than simple addition of item
The key here is how you treat that "summation". i.e. whether you
continue to rely upon the additivity assumption and so end up with CTT
and and a whole host of "data-model-presumptive" indices - or keep
remembering that really these magnitudes might just as well be crude
orders and so use them as ordered classes and seek evidence for what
these classes might indicate in terms of some expected set of outcomes.
My own "research" work and thinking nowadays is exactly that, seeking
optimal ways of scoring "assessments" (whether questionanire items,
scales, behavioral indicators/observations, or whatever, in terms of
maximizing cross-validated predictive validity - and establishing a set
of clear empirical relations first between "variables" - before
attempting (if at all) to impose a more formal data model upon them. 
Indeed, I have said in several presentations now that I think the next
big advances in psychology will be in how we combine "variables" to
predict outcomes, rather than concerning ourselves too much with the
task of trying to create ever more precise measurement of the variables
(because I've also adopted a view that psychology is more likely a
non-quantitative science).
I know, this all looks very sloppy and "informal" against the precision
and elegance of Rasch and Latent Variable modeling in general, but I
really do think these methods require assumptions about the proposed
causality of such variables (i.e the biological/cognitive systems which
are proposed as maintaining the desired response precision in terms of
measured "equal-interval" magnitudes) which are not very plausible given
the current knowledge we have about neuroscience and cognitions (such as
Gigerenzer's work etc.) 
Of course, whether such arguments or reasoning apply to the area of
"educational" and "medico-diagnostic" variables to which many would
apply a Rasch model is a moot point.
And, I suffer doubts every day about this ... as to move from a strong
quantitative "imperative" to one that says "it's all a bit fuzzy" but
that's how we humans actually are so deal with it accordingly - is a
huge step back in many onlooker's eyes (althouh I have said this is a
paradox in that one's predictions might well increase in accuracy 
given that the criterion itself is reduced to a real-world degree of
precision). But this is another issue; I only mention it here to provide
some little background as to why I'm less convinced than some by "test
theory", "Rasch, and psychometrics in general. I may be quite wrong.
If anybody is interested in my latest presentation papers surrounding
this issue - whizz over http://www.pbarrett.net/NZ_Psych_2007.htm ...
and take a quick look ...
but I'm not sure if these really are relevant to the Rasch community -
as these do not address edumetrics or educational issues per se.
Regards .. Paul


Paul Barrett, Ph.D.

2622 East 21st Street | Tulsa, OK  74114

Chief Research Scientist

Office | 918.749.0632  Fax | 918.749.0635

pbarrett at hoganassessments.com


hoganassessments.com <http://www.hoganassessments.com/> 





-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20071104/48ffaf78/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/bmp
Size: 29886 bytes
Desc: attfa128.bmp
Url : https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20071104/48ffaf78/attachment.bmp 

More information about the Rasch mailing list