[Rasch] Models of unidimensional constructshavetheirlimitations:what is a "good thing"?

Andrew Kyngdon akyngdon at lexile.com
Sat Nov 10 09:16:40 EST 2007


Or, do you claim that the Lexile system alone now defines (by its
application) the construct "reading ability"? I would guess Andrew
Kyngdon might want to make some statements here also.!

 

Always happy to oblige, Paul!

 

Whilst the Lexile system departs from conventional psychometrics in
several ways, the key strength of the system is the explicit theory of
the difficulty of written prose text passages. The proxy variables used
in the Lexile construct specification equation are unambiguously
quantitative and as such the theory has good face validity as a
quantitative theory. Of course, it is arbitrary to assume anything more
than monotone relationships between the proxies and the relevant
psychological attribute (viz., text difficulty). This can be addressed
using the theory of conjoint measurement. I have done this and presented
the results as part of talk given to Denny Borsboom's group in Amsterdam
back in June. The results suggested the difficulty of written prose text
passages, as conceptualised by the Lexile theory, possesses additive
structure.

 

Moreover, there is a strong substantive basis for the theory, firmly
rooted in the cognitive psychology of reading. Sentence length has for
some time been considered a good proxy for the demands prose text places
upon working (short term) memory (Klare, 1963; Crain & Shankweiler,
1988). The longer a sentence, the more thought units / words are in it
and the harder it is to keep the material in that sentence in working
memory (consistent with Miller's (1956) famous "7 plus or minus 2
chunks" study). Thus text passages with longer sentences are more
difficult to read.

 

One of the most robust findings in the cognitive psychology of reading
has been the capacity of word frequency (or log word frequency) to
predict eye fixation time and total gaze duration during reading (see
Rayner, 1998 for a complete review). Eye fixation time and gaze duration
are consistently shorter for common words than for rarer words, even
when word length is accounted for (Rayner, Reichle, Stroud, Williams &
Pollatsek, 2006). Moreover, its predictive capacity is not attenuated by
other variables, such a number of letters, subjective word familiarity
or age of word acquisition (AoA) (Juhaz & Rayner, 2003). Rarer words
take longer to cognitively process and recognise. Thus passages with
rarer words are harder to read.

 

To illustrate this, consider Just & Carpenter's (1980) gaze duration
reading study. A sentence from that study appears below. Total gaze
duration (in milliseconds) appears above each word that subjects gazed
(repeatedly fixated their eyes) at. Underneath are the log frequencies
of the words I obtained from the 500 million word Carroll corpus
(Carroll, Davies & Richman, 1977):

 

 

1566           267   400              267         617         767
450         450

Flywheels   are    one   of the  oldest mechanical devices known  to
man.

.3              4.55  4.31              2.26        2.15       2.14
3.15         3.79

 

Ignoring the difference in variable ranges and the fact it's just a
single sentence, the Pearson product moment correlation is -.82. The
rarer the word the longer it takes to recognise it. Little wonder then
that word frequency is a variable in the Lexile construct specification
equation for the difficulty of written prose text. 

 

Now, where test scores enter into the Lexile theory is in the assessment
of "reading ability". However, the Lexile theory suggests a person's
ability to read written prose text is a functional composition of his or
her working memory capacity and vocabulary. We could devise a construct
specification equation for reading ability from the independent
measurement of working memory capacity and vocabulary, similarly for
what we did with text. However, this would probably mean devising an
equation for each and every person, given that persons exhibit much more
"individual difference" variation than passages of written text. Whilst
theoretically possible, it is too daunting given 26 million school
children in the US alone get a Lexile measure. This is also ignoring
intra-individual growth in reading ability, which would mean constant
recalibration of such equations. When the growth model expert Gary
Williamson worked at MetaMetrics he found that reading ability grew
approximately 100, 75 and 50 Lexiles per year for children in
elementary, middle and high school, respectively. As I say, individual
reading ability specification equations are a daunting proposition. So
the Lexile system uses test scores for the measurement of reading
ability, theoretically weak as these are.

 

Nevertheless, there is a way the structure of reading ability can be
tested using test scores. The Lexile system argues that comprehension of
written prose text (item response probabilities) is a non-interactive,
additive function of "reading ability" and text difficulty (as
calculated by the Lexile construct specification equation). Hence text
comprehension is hypothesised to be an additive conjoint structure; and
therefore application of the axioms of conjoint measurement could tell
us something about the structure of reading ability. We have a dataset
here to do just that at some stage. Note this also means the Lexile
system has an explicit, falsifiable theory of the comprehension of
written prose text, the relevant variables being reading ability and
text difficulty.

 

So now to your questions.

 

1. Is reading ability actually a behavioural proxy for vocabulary
knowledge?

 

I would answer "no". The Lexile system suggests the ability to read
written prose text is a functional composition of vocabulary and working
memory capacity.

 

2. What is comprehension, is it reading ability or another variable
which requires separate measurement?

 

The Lexile system suggests that comprehension is a function of reading
ability and text difficulty, as described above.

 

Cheers,

 

Andrew

 

Refs.

 

Carroll, J.B., Davies, P. & Richman, B. (1971). Word frequency book.
Boston: Houghton Mifflin.

 

Juhasz, B.J. & Rayner, K. (2003). Investigating the effects of a set of
intercorrelated variables on eye fixation durations in reading. Journal
of Experimental Psychology: Learning, Memory and Cognition, 29, 1312 -
1318.

 

Just, M.A. & Carpenter, P.A. (1980). A theory of reading: from eye
fixations to comprehension. Psychological Review, 87, 329 - 354.

 

Klare, G.R. (1963) The measurement of readability. Ames, IA: Iowa State
University Press.

 

Rayner, K. (1998). Eye movements in reading and information processing:
20 years of research. Psychological Bulletin, 124, 372 - 422.

 

Rayner, K.,  Reichle, E.D., Stroud, M.J., Williams, C.C. & Pollatsek, A.
(2006). The effect of word frequency, word predictability and font
difficulty on the eye movements of young and older readers. Psychology
and Aging, 21, 448 - 465.

 

 

 

Andrew Kyngdon, PhD

Senior Research Scientist

MetaMetrics, Inc.

1000 Park Forty Plaza Drive

Durham NC 27713 USA

Tel. 1 919 354 3473

Fax. 1 919 547 3401

3rd International Conference in Rasch measurement,

January 22-24 2008, Perth, Australia

http://www.education.uwa.edu.au/__data/page/115936/Rasch_conference_v12.
pdf

 

 

________________________________

From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On
Behalf Of Paul Barrett
Sent: Thursday, November 08, 2007 2:16 PM
To: rasch at acer.edu.au
Subject: RE: [Rasch] Models of unidimensional
constructshavetheirlimitations:what is a "good thing"?

 

	 

	
________________________________


	From: rasch-bounces at acer.edu.au
[mailto:rasch-bounces at acer.edu.au] On Behalf Of Stephanou, Andrew
	Sent: Thursday, November 08, 2007 4:58 PM
	To: Agustin Tristan
	Cc: rasch at acer.edu.au
	Subject: RE: [Rasch] Models of unidimensional constructs
havetheirlimitations:what is a "good thing"?

	... Social scientists illude themselves if they think that
reading ability is a more complex construct than the construct of
temperature.

	 

Andrew

 

Do social scientists delude themselves?

 

Is there a definition of the construct of "reading ability", which
stands independent of the method of its measurement, or are the rules
for measurement also partially constituent of the meaning of the
construct? I'm assuming the former .. but ... 

 

With regard to reading ability, is it possible to be able to read text
without comprehending what is being read? Is that "reading ability" or
something else? I note in the paper 

 

AMANDA SCHAFER  JOHNSON, LISA J.  FLICKER, and PETER A.  LICHTENBERG
(2006) Reading ability mediates the relationship between education and
executive function tasks. Journal of the International
Neuropsychological Society , 12, 64-71.

 

Reading ability is measured by (p. 65) the 


"Wide Range Achievement Test-3 (WRAT-3): Reading (Wilkinson, 1993)


This is a test of word familiarity and sight reading ability involving
the pronunciation of a series of 15 letters of the alphabet and 42
increasingly difficult words. A maximum score of 57 is possible. As
reported in Spreen and Strauss (1998), it has been useful in estimating
premorbid intelligence, and moderately relates to WAIS-R IQ scores
(Griffin et al., 2002). " 

 

The abstract rom the paper reads ...

Neuropsychological test results are affected by multiple factors, but
usually age and education are the only variables by which norms are
stratified. Some authors have questioned whether these variables alone
are sufficient (e.g., Marcopulos et al., 1997; Manly et al., 2002),
since such norms have lead to problems, such as poor specificity for
African Americans on dementia screening devices (Fillenbaum et al.,
1990). Recent research has shown that reading ability, a measure of
educational quality, attenuated racial differences in test performance
(Manly et al., 2002). We specifically examined whether reading ability
would account for a greater amount of variance than education in
executive function tests in a population traditionally subject to poor
educational quality. Results determined that reading ability accounted
for a significantly greater amount of variance than years of education
for Letter-Number Sequencing, Similarities, COWA, Trail Making Test, and
Coloured Progressive Matrices. Reading ability was found to
significantly mediate the relationship between each of these tests and
education.  

 

Then there is the paper by 

 

van den Bos, K. P; Nakken, H; Nicolay, P. G; & van Houten, E. J.(2007)
Adults with mild intellectual disabilities: Can their reading
comprehension ability be improved? Journal of Intellectual Disability
Research. Vol 51(11) Nov 2007, 835-849 (which doesn't use Lexiles by the
way).

 

Are we talking about an ability or a skill, or are these terms
synonymous? 

 

Is reading ability actually a behavioral proxy for vocabulary knowledge?
What is comprehension, is it reading ability or another variable which
requires separate measurement?

 

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 

 

Look at the corresponding definition of temperature; Giancoli (1988)
defines it as:

 

"how hot or cold an object is" .. (p. 406) and on page 449 "Temperature
is a measure of the average kinetic energy of individual molecules"

 

where on page 449, heat is defined as referring to: "the amount of
energy transferred from a hot body to a cold body" 

 

Howerver, defining "energy" is not so simple!

 

 

Perhaps reading ability is as complex as defining "temperature" - when
"average kinetic energy of molecules" is unpacked for meaning?  Maybe
even more so when that heady mix of "variables" like comprehension,
sight/phonic reading (without comprehension), vocabulary, and reasoning
are all taken to be constitutive somehow of the observed "reading
ability"?

 

 

Regards .. Paul

 

Giancoli, D.C. (1988) Physics for Scientists and Engineers with Modern
Physics. 2nd Edition. Prentice-Hall. 

 

 

 

 

Paul Barrett, Ph.D.

2622 East 21st Street | Tulsa, OK  74114

Chief Research Scientist

Office | 918.749.0632  Fax | 918.749.0635

pbarrett at hoganassessments.com

      

hoganassessments.com <http://www.hoganassessments.com/> 

 

 

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20071109/274901b9/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3176 bytes
Desc: image001.jpg
Url : https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20071109/274901b9/attachment.jpe 


More information about the Rasch mailing list