[Rasch] Units of measurement in the social sciences

Paul Barrett pbarrett at hoganassessments.com
Wed Oct 24 05:38:31 EST 2007


 

________________________________

	From: Andrew Kyngdon [mailto:akyngdon at lexile.com] 
	Sent: Wednesday, October 24, 2007 3:21 AM
	To: Paul Barrett; rasch at acer.edu.au
	Subject: RE: [Rasch] Units of measurement in the social sciences
	
	

	Hi Paul,

	 

	You've raised an important point of which much attention has
been paid at MetaMetrics. There are a few issues here.

	 

	...... 

	On a related issue, we recently obtained data using an
instructional software application based on the Lexile Framework for
Reading. We found that forecasted text comprehension rates (as
percentages of correct responses) matched observed comprehension rates
to 3 decimal places.

	 

	I hope this helps. Feel free to ask more questions.

	 

Thanks Andrew - seriously impressive data; I have some reading to do!
 
One point that occurs to me though, if the error around the lexile
estimate for an individual is as you related, how is it possible to
report estimated vs actual comprehension rates correct to 3 decimal
places? 
 
Assuming a lexile score is involved somewhere in the calculation of the
estimated correct response rate, wouldn't this introduce greater than
.0005 error in the rate (assuming a rate is reported between 0 and 1 -
which can be multiplied up to a %, but I'm assuming you don't mean 3
decimal places on an already corrected percentage, as this would imply a
standard or accuracy to 5 decimal places)?
 
Presumably this has something to do with how precisely the initial
lexile score was estimated for each individual?
 
How many individuals were used by the way - and did they vary in initial
lexile score (across a reasonable range of lexile scores?).
 
I'm curious because if we think for one moment about how the brain works
(in terms of networks of dynamical system "wetware" biology), the kinds
of precision you seem to be asserting don't make much sense except for
some sensory transducer systems (auditory spatial frequency resolution
etc.). Likewise any proposal that a standard additive-metric unit could
be invoked for any "psychological" variable ("Comprehension" being just
such a variable).
 
But "seem to make sense" is a phrase which can be simply destroyed by
empirical evidence, which is why I find this issue fascinating.
 
I don't mean to be negative here, and I don't want anyone reading this
thinking I'm "getting at" Andrew or Metametrics. It's just my current
thinking and perusal of neuroscience and evolutionary biology has caused
me to doubt strongly that the "behavioral - cognitive" output of a human
brain can ever be "measured" using linear unit-preserving metrics. The
system responsible for these cognitions/behaviors is complex, in the
technical sense of that word. So, we can approximate as a linear
function overlaid on an ordered metric, but quantitative measurement
within such a system would seem impossible as the "causal-biology" of
the system would be unable to sustain a standard unit to the precision
required by the "additivity" property.
 
Hence, I am fascinated by data which seems to demonstrate the
quantitative, near-deterministic (vs stochastic), measurement of a
psychological variable - which is why the validation experiments are so
critical to me personally!
 
Regards ... Paul
 
 
 

 

Paul Barrett, Ph.D.

2622 East 21st Street | Tulsa, OK  74114

Chief Research Scientist

Office | 918.749.0632  Fax | 918.749.0635

pbarrett at hoganassessments.com

      

hoganassessments.com <http://www.hoganassessments.com/> 

 

 

 

 

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20071023/d9106165/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/bmp
Size: 29886 bytes
Desc: att86552.bmp
Url : https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20071023/d9106165/attachment.bmp 


More information about the Rasch mailing list