<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">

<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]--><o:SmartTagType
 namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PostalCode"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
 name="country-region" downloadurl="http://www.5iantlavalamp.com/"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
 name="State" downloadurl="http://www.5iamas-microsoft-com:office:smarttags"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
 name="Street" downloadurl="http://www.5iantlavalampft-com:office:smarttags"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
 name="address" downloadurl="http://www.5iamas-microsoft-com:office:smarttags"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
 name="place" downloadurl="http://www.5iantlavalamp.com/"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
 name="City" downloadurl="http://www.5iamas-microsoft-com:office:smarttags"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
 name="PersonName" downloadurl="http://www.microsoft.com"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
 /* Font Definitions */
 @font-face
        {font-family:Tahoma;
        panose-1:2 11 6 4 3 5 4 4 2 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman";}
a:link, span.MsoHyperlink
        {color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {color:blue;
        text-decoration:underline;}
p
        {mso-margin-top-alt:auto;
        margin-right:0in;
        mso-margin-bottom-alt:auto;
        margin-left:0in;
        font-size:12.0pt;
        font-family:"Times New Roman";}
span.EmailStyle19
        {mso-style-type:personal-reply;
        font-family:Arial;
        color:navy;}
@page Section1
        {size:595.3pt 841.9pt;
        margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
        {page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
 <o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
 <o:shapelayout v:ext="edit">
  <o:idmap v:ext="edit" data="1" />
 </o:shapelayout></xml><![endif]-->
</head>

<body bgcolor=white lang=EN-AU link=blue vlink=blue id=compText onunload="">

<div class=Section1>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Dear Paul,<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>A great firebrand post! One major quibble,
however, with what you said.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>In the Lexile system, measures of the
difficulty of written prose text passages are NOT obtained from either test
scores, ratings, simulations or IRT model analyses of such data.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>The Lexile theory argues that the
difficulty of written prose text is a functional composition of two key
cognitive variables &#8211; <i><span style='font-style:italic'>syntactic complexity
</span></i>and <i><span style='font-style:italic'>semantic rarity</span></i>. &nbsp;Syntactic
complexity is </span></font><font size=2 color=navy face=Arial><span
lang=EN-US style='font-size:10.0pt;font-family:Arial;color:navy'>the complexity
of the sentence / thought unit structure of a written prose text passage.
Semantic rarity is the familiarity of the semantic units (words) which comprise
a prose text passage.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>Each is assessed using a
proxy variable. For syntactic complexity. it is the natural logarithm of the
mean sentence length (LMSL). Why is this proxy used? Research in cognition has
found that it is a good proxy for the demand written prose text places upon
Short Term Memory (e.g. Crain &amp; Skankweiler, 1988). The longer a sentence is,
the harder it is to keep all the information in a sentence in STM. Thus text
passages with longer sentences are more difficult to comprehend.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>For semantic rarity, the
proxy variable is the mean natural logarithm of word frequency (MLWF). Why is
this proxy used? Stenner, Smith &amp; Burdick (1983) investigated over 50
variables, including part of speech, number of letters, number of syllables,
modal grade, word content classification and various transformations of these
variables. They found MLWF was the best proxy for the likelihood a person has a
word in their mental lexicon. Moreover, reaction time and gaze duration studies
have found that people will look at rarer words for several hundred
milliseconds longer than commonly occurring words, suggesting rarer words are
harder to locate and retrieve from the mental lexicon. Thus text passages
containing words which are infrequently used will be more difficult to
comprehend.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>The difficulty of a written
prose text passage <i><span style='font-style:italic'>i</span></i>, <i><span
style='font-style:italic'>D<sub>i</sub></span></i>, is a functional composition
of these proxies such that: <i><span style='font-style:italic'>D<sub>i</sub></span></i>
= (<i><span style='font-style:italic'>a</span></i> x LMSL) &#8211; (<i><span
style='font-style:italic'>b</span></i> x MLFW) &#8211; <i><span
style='font-style:italic'>c</span></i>. This expression, referred to as a <i><span
style='font-style:italic'>construct specification equation</span></i>, is a 5
variable simple polynomial conjoint structure (Krantz, Luce, Suppes &amp;
Tversky, 1971) where <i><span style='font-style:italic'>a</span></i>, <i><span
style='font-style:italic'>b</span></i> and <i><span style='font-style:italic'>c
</span></i>are real valued constants. The actual values of these constants are proprietary;
however, they result in <i><span style='font-style:italic'>D<sub>i</sub> </span></i>being
a scale in logit units.</span></font><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Notice the complete absence of &#8220;run
of the mill&#8217; psychometry - test scores, IRT model estimates, fit
statistics, ratings, DIF, reliability coefficients and so on &#8211; in the
measurement of the difficulty of written prose text passages. <o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>The biggest weakness of the theory is the
assumption of something more than monotone relationships existing between proxy
variables and the relevant psychological attributes (Krantz &amp; Tversky,
1971). The theory of conjoint measurement (Krantz, et al, 1971) can be used to
address this issue. I have done this and found the axioms to be satisfied using
Karabatsos&#8217;s (2005) methods for the probabilistic tests of deterministic
axioms.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>The Lexile theory argues that the probability
of comprehending a written prose text passage is a non-interactive, additive
function of the text&#8217;s difficulty and the persons&#8217; reading ability.
A suitable model for this is the Rasch (1960) model. However, unlike
conventional psychometry, we modify the Rasch model to suit the theory. If a
person&#8217;s ability is equal to the text difficulty, we do not consider the
person has successfully comprehended a text passage when the response
probability is only .5. A constant is therefore introduced into the Rasch model
such that when ability equals difficulty, the response probability is .75. Note
this modification does not destroy the raw score sufficiency property (or any
others) of the Rasch model. <o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>This modified Rasch model is Pr [Person<i><span
style='font-style:italic'> v </span></i>comprehends text passage<i><span
style='font-style:italic'> i</span></i>] = exp(</span></font><i><font size=2
color=navy face=Arial><span lang=EN-US style='font-size:10.0pt;font-family:
Arial;color:navy;font-style:italic'>B<sub>v</sub> &#8211; D<sub>i </sub></span></font></i><font
size=2 color=navy face=Arial><span lang=EN-US style='font-size:10.0pt;
font-family:Arial;color:navy'>+ <i><span style='font-style:italic'>k</span></i>)
/ 1 + exp</span></font><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'>(</span></font><i><font
size=2 color=navy face=Arial><span lang=EN-US style='font-size:10.0pt;
font-family:Arial;color:navy;font-style:italic'>B<sub>v</sub> &#8211; D<sub>i</sub>
</span></font></i><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>+ <i><span
style='font-style:italic'>k</span></i>), where <i><span style='font-style:italic'>Bv</span></i>
is the ability of person <i><span style='font-style:italic'>v</span></i> and <i><span
style='font-style:italic'>k</span></i> is the aforementioned constant. Test
scores enter the Lexile theory in the estimation of person abilities. These
come from tests consisting of Lexile passage native items (text passages with a
sentence &#8220;close&#8221;). Use of the Rasch model puts the abilities on the
same scale as the text difficulty variable. Both then can use the same affine
transformation to become measurements in the Lexile scale.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>Hence I consider your
point:<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>&#8220;&#8230;there is no substantive
difference between probabilistic (latent variable) &nbsp;or deterministic (raw
score) scale scores. Which, by default, means all of the Metametrics Lexile and
Quantile work could have been achieved using
conventional&nbsp;deterministic/actuarial score algorithms (now there's a
sobering thought).. &#8220;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>to be incorrect. One
cannot arrive at the measurement of text difficulty, as theorized in the Lexile
system, following conventional psychometric practice.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>Cheers,<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>Andrew<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>P.S. One serious problem
with connectionist models as theories of human thought processes &#8211; no
biological equivalent to the backpropagation algorithm. Does this mean
connectionism is merely another &#8220;unopened black box&#8221; theory of
psychology?<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>Refs:<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>Crain, S. &amp;
Skankweiler, D. (1988). Syntactic complexity and reading acquisition. In A.
Davidson and G.M. Green (Eds.), <i><span style='font-style:italic'>Linguistic
complexity and text comprehension: readability issues reconsidered</span></i>. <st1:place
w:st="on"><st1:City w:st="on">Hillsdale</st1:City>, <st1:State w:st="on">N.J.</st1:State></st1:place>:
Erlbaum.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Karabatsos, G. (2005). The exchangeable
multinomial model as an approach to testing deterministic axioms of choice and
measurement. <i><span style='font-style:italic'>Journal of Mathematical
Psychology</span></i>, <i><span style='font-style:italic'>49</span></i>(1),
51-69.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Krantz, D.H.; Luce, R.D; Suppes, P. &amp;
Tversky, A. (1971). <i><span style='font-style:italic'>Foundations of
Measurement, Vol. I: Additive and polynomial representations</span></i>. <st1:State
w:st="on"><st1:place w:st="on">New York</st1:place></st1:State>: Academic Press.</span></font><font
size=2 color=navy face=Arial><span lang=EN-US style='font-size:10.0pt;
font-family:Arial;color:navy'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Krantz, D.H. &amp; Tversky, A. (1971).
Conjoint measurement analysis of composition rules in psychology. <i><span
style='font-style:italic'>Psychological Review</span></i>, <i><span
style='font-style:italic'>78</span></i>(2), 151-169.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Rasch, G. (1960).&nbsp; <i><span
style='font-style:italic'>Probabilistic models for some intelligence and
attainment tests</span></i>. <st1:City w:st="on"><st1:place w:st="on">Copenhagen</st1:place></st1:City>:
Danish Institute for Educational Research.</span></font><font size=2
color=navy face=Arial><span lang=EN-US style='font-size:10.0pt;font-family:
Arial;color:navy'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>Stenner, A.J., Smith, M.
&amp; Burdick, D. S. (1983) Toward a theory of construct definition. <i><span
style='font-style:italic'>Journal of Education Measurement</span></i>, <i><span
style='font-style:italic'>20</span></i>, 305-315.<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p>&nbsp;</o:p></span></font></p>

<div>

<p class=MsoNormal><st1:PersonName w:st="on"><font size=4 color=navy
 face=Arial><span lang=EN-US style='font-size:14.0pt;font-family:Arial;
 color:navy'>Andrew Kyngdon</span></font></st1:PersonName><font size=4
color=navy face=Arial><span lang=EN-US style='font-size:14.0pt;font-family:
Arial;color:navy'>, PhD</span></font><font size=4 color=navy><span lang=EN-US
style='font-size:14.0pt;color:navy'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>Senior Research Scientist</span></font><font
size=2 color=navy><span lang=EN-US style='font-size:10.0pt;color:navy'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>MetaMetrics, Inc.</span></font><font
size=2 color=navy><span lang=EN-US style='font-size:10.0pt;color:navy'><o:p></o:p></span></font></p>

<p class=MsoNormal><st1:Street tabIndex="0"
style="BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
w:st="on"><st1:address tabIndex="0"
 style="BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
 w:st="on"><font size=2 color=navy face=Arial><span lang=EN-US
  style='font-size:10.0pt;font-family:Arial;color:navy'>1000 Park Forty Plaza
  Drive</span></font></st1:address></st1:Street><font size=2 color=navy><span
lang=EN-US style='font-size:10.0pt;color:navy'><o:p></o:p></span></font></p>

<p class=MsoNormal><st1:place tabIndex="0"
style="BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
w:st="on"><st1:City tabIndex="0"
 style="BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
 w:st="on"><font size=2 color=navy face=Arial><span lang=EN-US
  style='font-size:10.0pt;font-family:Arial;color:navy'>Durham</span></font></st1:City><font
 size=2 color=navy face=Arial><span lang=EN-US style='font-size:10.0pt;
 font-family:Arial;color:navy'> <st1:State tabIndex="0"
 style="BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
 w:st="on">NC</st1:State> <st1:PostalCode w:st="on">27713</st1:PostalCode> <st1:country-region
 tabIndex="0"
 style="BACKGROUND-POSITION: left bottom; BACKGROUND-IMAGE: url(res://ietag.dll/#34/#1001); BACKGROUND-REPEAT: repeat-x"
 w:st="on">USA</st1:country-region></span></font></st1:place><font size=2
color=navy><span lang=EN-US style='font-size:10.0pt;color:navy'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>Tel. 1 919&nbsp;354 3473</span></font><font
size=2 color=navy><span lang=EN-US style='font-size:10.0pt;color:navy'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>Fax. 1 919 547 3401</span></font><font
color=navy><span lang=EN-US style='color:navy'><o:p></o:p></span></font></p>

<div>

<p class=MsoNormal><font size=2 color=navy face=Arial><span lang=EN-US
style='font-size:10.0pt;font-family:Arial;color:navy'>&nbsp;<o:p></o:p></span></font></p>

</div>

</div>

<div>

<p class=MsoNormal><font size=3 face="Times New Roman"><span lang=EN-US
style='font-size:12.0pt'><o:p>&nbsp;</o:p></span></font></p>

</div>

<blockquote style='border:none;border-left:solid black 1.5pt;padding:0in 0in 0in 4.0pt;
margin-left:3.75pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt'>

<div class=MsoNormal align=center style='text-align:center'><font size=2
color=black face=Arial><span lang=EN-US style='font-size:10.0pt;font-family:
Arial;color:black'>

<hr size=2 width="100%" align=center tabIndex=-1>

</span></font></div>

<p class=MsoNormal style='margin-bottom:12.0pt'><b><font size=2 color=black
face=Tahoma><span lang=EN-US style='font-size:10.0pt;font-family:Tahoma;
color:black;font-weight:bold'>From:</span></font></b><font size=2 color=black
face=Tahoma><span lang=EN-US style='font-size:10.0pt;font-family:Tahoma;
color:black'> rasch-bounces@acer.edu.au [mailto:rasch-bounces@acer.edu.au] <b><span
style='font-weight:bold'>On Behalf Of </span></b>renselange@earthlink.net<br>
<b><span style='font-weight:bold'>Sent:</span></b> Tuesday, October 30, 2007
5:28 AM<br>
<b><span style='font-weight:bold'>To:</span></b> Anthony James;
rasch@acer.edu.au<br>
<b><span style='font-weight:bold'>Subject:</span></b> Re: [Rasch] Fan</span></font><font
size=2 color=black face=Arial><span lang=EN-US style='font-size:10.0pt;
font-family:Arial;color:black'><o:p></o:p></span></font></p>

<p><font size=2 color=blue face=Arial><span style='font-size:10.0pt;font-family:
Arial;color:blue'>My take on this is &quot;so what&quot; as well. Getting
&quot;scores&quot; is hardly the main achievement of Rasch modeling.&nbsp;There
simply&nbsp;is little appreciable difference between raw score sums&nbsp;and
the Rasch model, as well as estimates obtained via one, two, or three-parameter
logistic models.</span></font><font size=2 color=black face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:black'> <o:p></o:p></span></font></p>

</blockquote>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>Hello Reese</span></font><font size=1
color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>There you have it in a nutshell, why many
scientists&nbsp;(vs psychometricians) concerned with explaining substantive
phenomena, sometimes using questionnaire data, are not concerned with
itemmetric models. </span></font><font size=1 color=black face=Arial><span
style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>Edumetrics is frankly &quot;technician
work&quot;;&nbsp;the utility and benefits of probabilistic modeling of
item&nbsp;response data&nbsp;seems&nbsp;to be irrelevant to explaining or
predicting phenomenal outcomes more accurately.&nbsp;&nbsp;Whilst other
benefits might accrue to those primarily concerned with examinations,
licensing, or other such pragmatic ventures, as&nbsp;a scientist, you are
concerned with minimizing substantive error in any&nbsp;predictions you might
make on the basis of theory or&nbsp;even &quot;dustbowl&quot;
empiricism.&nbsp;&nbsp;</span></font><font size=1 color=black face=Arial><span
style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>As you state Reese, there is no
substantive difference between probabilistic (latent variable) &nbsp;or
deterministic (raw score) scale scores. Which, by default, means all of the
Metametrics Lexile and Quantile work could have been achieved using
conventional&nbsp;deterministic/actuarial score algorithms (now there's a
sobering thought).. </span></font><font size=1 color=black face=Arial><span
style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>Whether or not IRT is a more
&quot;efficient&quot; method of going about these activities seems to be the
issue - which is an important one neverthless. </span></font><font size=1
color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>But what use is &quot;efficiency&quot;
when the magnitudes obtained are equivalent to just summing up item responses?
Person reliability can easily be computed without any fuss using conventional
item difficulties and rank order response discrepancy (between item difficulty
ranking and observed item responses). You don't need a &quot;data model&quot;
for this simple heuristic. Cross-validation sorts out replicability.</span></font><font
size=1 color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>I'm not so sure IRT is any more efficient
at all than a sensible and intelligent approach to construct assessment, except
perhaps where edumetrics is concerned - but even here, where are the advantages
to be seen, and by whom?&nbsp;Are educational standards improving because of
IRT methods being applied?</span></font><font size=1 color=black face=Arial><span
style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>As far as I can see, the only reason IRT
came into existence was because Rasch and others were&nbsp;fascinated with test
theory, statistical data models, and fundamental measurement, instead of the
actual phenomena of interest, and capturing these directly without the need
for&nbsp;bucketloads of &nbsp;assumptions or practically incomprehensible
algebra and math.</span></font><font size=1 color=black face=Arial><span
style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>Indeed, what is apparent (to me at
least)&nbsp;is that&nbsp;the major source of
explanatory/predictive&nbsp;error&nbsp;is&nbsp;not in the
&quot;measurement&quot; metrics per se, but in the concepts we invoke to
explain phenomena, and how we actually try to measure or assess
&quot;magnitudes&quot; on them.</span></font><font size=1 color=black
face=Arial><span style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>And, I haven't forgotten my reply to
Andrew or Moritz(!)&nbsp;- but I'm flat out on an interrater
discrepancy/reliability program right now - for the next 5 days or so - but I
couldn't resist replying to this as I've seen the same &quot;even though there
is no difference, this is still wonderful stuff&quot; statements in a recent
paper comparing Ideal Point unfolding models, dominance IRT, and simple raw
score methods ...</span></font><font size=1 color=black face=Arial><span
style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<div>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>Cherneyshenko, O.S., Stark, S., Drasgow,
F., &amp; Roberts, B.W. (2007) Constructing personality scales under the
assumption of an Ideal Point response process: toward increasing the
flexibility of personality measures. <em><i><font face=Arial><span
style='font-family:Arial'>Psychological Assessment</span></font></i></em>, 19,
1, 88-106.&nbsp;</span></font><font size=1 color=black face=Arial><span
style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

</div>

<div>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>See page 103, Col 2, 2nd para beginning
&quot;Note also that examination&nbsp;of correlations .. &quot; ... which
contains the sentence ... &quot;</span></font><font size=2 color=navy
face=Arial><span style='font-size:10.0pt;font-family:Arial;color:navy'>The use
of ideal point methods, however, is unlikely to yield increases in
criterion-related validities</span></font><font size=2 color=black face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:black'>&quot;</span></font><font
size=1 color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

</div>

<div>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>and then read the 1st para of the
discussion .. </span></font><font size=1 color=black face=Arial><span
style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

</div>

<div>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>Their argument, as far as I can see, is
more about the technicalities/differential pragmatic benefits&nbsp;of test
construction ...and the consequences on particular measurement ranges within a
scale. </span></font><font size=1 color=black face=Arial><span
style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

</div>

<div>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>But, if the criterion prediction remains
the same - what is the achieved real-world&nbsp;or scientific benefit? </span></font><font
size=1 color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

</div>

<div>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>I have the feeling they are discovering
that the real problem is not with the technology of assessment, but with the
very nature of the constructs we try to assess.</span></font><font size=1
color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

</div>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>However, this is not a call to undo or
dump IRT - as those like Metametrics, ACER, and others&nbsp;who have generated
impressive measurement and assessment systems around IRT are testament to its
functionality, utility, and profitability.&nbsp;</span></font><font size=1
color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>This is merely a statement which says
that it is just another way of &quot;modeling data&quot; and constructing test
scores which has proved to have no tangible advantages over any other
&quot;intelligent&quot; method of analyzing data to make important decisions.</span></font><font
size=1 color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>If you think think this is unfair,
consider the&nbsp;development of the &quot;perceptron&quot;&nbsp;(The
perceptron is a type of artificial neural network invented in 1957 at the
Cornell Aeronautical Laboratory by Frank Rosenblatt .. Wikipedia). In 50 years
look at what&nbsp;this simple &quot;algorithm&quot; for a neural net has
spawned .. an entire&nbsp;new area&nbsp;of psychological
science&nbsp;investigation &quot;connectionist cognitive psychology&quot;, a
neurophysiological model for developmental brain networks and brain
function,&nbsp;artificial intelligence, computational biology,&nbsp;and
literally physical working models of the development of human intelligence,
stock market trading algorithms, seriously accurate&nbsp;market research/brand
analysis&nbsp;prediction models, security camera person-feature detection,
biometric sensors, handwriting, and speech detection, etc, etc.. </span></font><font
size=1 color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>And what has IRT spawned in the same
period? A&nbsp;few tests which are almost indistinguishable from the old
tests,&nbsp;hundreds of books and thousands of papers on test theory and tiny
one-shot applications for which CTT would have done the same job just as well -
and was probably doing so,&nbsp;and a few major-league companies who sell the
technology and some of its results to others.</span></font><font size=1
color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>If I am to do any work as a psychological
scientist - I want to be the inventor of&nbsp; a &quot;perceptron&quot;, and
not the inventor of yet another &quot;test theory&quot; - which is why methods
which predict or measure no better than simple &quot;back of the matchbox&quot;
algorithms or heuristics&nbsp;are not given the &quot;time of day&quot; by
those like myself who want to make big strides in our science.&nbsp;</span></font><font
size=1 color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>So, in response to:&quot;</span></font><font
size=2 color=blue face=Arial><span style='font-size:10.0pt;font-family:Arial;
color:blue'>The main problem here is that non-IRT folks often do not (or
cannot) even conceive of the possibilities afforded by test / form equating,
item / person fit and bias testing, Hence, the advantages of all this appear
rather &quot;academic&quot; ( read: useless).</span></font><font size=2
color=black face=Arial><span style='font-size:10.0pt;font-family:Arial;
color:black'> &quot; ..</span></font><font size=1 color=black face=Arial><span
style='font-size:9.0pt;font-family:Arial;color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>You now have a better idea why&nbsp;some
like me really don't bother with IRT at all anymore - it's virtually irrelevant
to substantive progress in psychological science. And, if you need to make
decisions using test scores, why bother when a simple sum score interpreted
using representative population norms and other relevant and somewhat more
obvious item and scale statistics does &quot;the business&quot; just as well as
scores&nbsp;and indices associated with the &quot;latent variable&quot; work
which only a &quot;psychometric&nbsp; wizard&quot; can seemingly convey!</span></font><font
size=1 color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>What made the neural net so easy to
&quot;sell&quot; was that it predicted valuable and substantive outcomes for
which no other model came close. That's why it was grabbed with open arms by
those &quot;non-academics and academics&quot; alike who looked at the results
and thought &quot; I've gotta have this &quot;. Sheesh, even Shepard, Kruskal,
Lingoes, and Guttman's non-metric Multidimensional Scaling algorithms had a
bigger impact in 10 years from their inception than IRT.</span></font><font
size=1 color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>Until IRT, unfolding, or any test theory
model can produce that &quot;wow&quot; effect - they will&nbsp;remain
&quot;niche&quot;&nbsp;activities for a minority of individuals in a tiny
sub-domain of the social and human sciences.</span></font><font size=1
color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p class=MsoNormal><font size=2 color=black face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:black'>Regards ... Paul</span></font><font
size=1 color=black face=Arial><span style='font-size:9.0pt;font-family:Arial;
color:black'><o:p></o:p></span></font></p>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<div>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

</div>

<div>

<table class=MsoNormalTable border=0 cellspacing=0 cellpadding=0 align=left
 width=500 style='width:375.0pt;border-collapse:collapse' id=table1>
 <tr height=66 style='height:49.5pt'>
  <td width=223 height=66 style='width:167.25pt;padding:0in 0in 0in 0in;
  height:49.5pt'>
  <p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
  mso-element:frame'><font size=3 color=black face="Times New Roman"><span
  style='font-size:12.0pt;color:black'><img width=150 height=66
  id="_x0000_i1026" src="cid:image001.jpg@01C81B11.0208BC20" border=0><o:p></o:p></span></font></p>
  </td>
  <td width=277 height=66 style='width:207.75pt;padding:0in 0in 0in 0in;
  height:49.5pt'>
  <p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
  mso-element:frame'><font size=3 color=black face="Times New Roman"><span
  style='font-size:12.0pt;color:black'>&nbsp;<o:p></o:p></span></font></p>
  </td>
 </tr>
 <tr height=14 style='height:10.5pt'>
  <td width=223 height=14 style='width:167.25pt;padding:0in 0in 0in 0in;
  height:10.5pt'>
  <p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
  line-height:10.5pt;mso-element:frame'><b><font size=1 color=gray face=Arial><span
  style='font-size:8.0pt;font-family:Arial;color:gray;font-weight:bold'>Paul
  Barrett, Ph.D.</span></font></b><font color=black><span style='color:black'><o:p></o:p></span></font></p>
  </td>
  <td width=277 height=14 style='width:207.75pt;padding:0in 0in 0in 0in;
  height:10.5pt'>
  <p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
  line-height:10.5pt;mso-element:frame'><st1:Street w:st="on"><st1:address
   w:st="on"><b><font size=1 color=gray face=Arial><span style='font-size:8.0pt;
    font-family:Arial;color:gray;font-weight:bold'>2622 East 21st Street</span></font></b></st1:address></st1:Street><b><font
  size=1 color=gray face=Arial><span style='font-size:8.0pt;font-family:Arial;
  color:gray;font-weight:bold'> | <!--StartFragment--><st1:place w:st="on"><st1:City
   w:st="on">Tulsa</st1:City>, <st1:State w:st="on">OK</st1:State>&nbsp; <st1:PostalCode
   w:st="on">74114</st1:PostalCode></st1:place></span></font></b><font
  color=black><span style='color:black'><o:p></o:p></span></font></p>
  </td>
 </tr>
 <tr height=14 style='height:10.5pt'>
  <td width=223 height=14 style='width:167.25pt;padding:0in 0in 0in 0in;
  height:10.5pt'>
  <p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
  line-height:10.5pt;mso-element:frame'><b><font size=1 color=gray face=Arial><span
  style='font-size:8.0pt;font-family:Arial;color:gray;font-weight:bold'>Chief
  Research Scientist</span></font></b><font color=black><span style='color:
  black'><o:p></o:p></span></font></p>
  </td>
  <td width=277 height=14 style='width:207.75pt;padding:0in 0in 0in 0in;
  height:10.5pt'>
  <p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
  line-height:10.5pt;mso-element:frame'><b><font size=1 color=gray face=Arial><span
  style='font-size:8.0pt;font-family:Arial;color:gray;font-weight:bold'>Office
  | 918.749.0632&nbsp; Fax | 918.749.0635</span></font></b><font color=black><span
  style='color:black'><o:p></o:p></span></font></p>
  </td>
 </tr>
 <tr height=14 style='height:10.5pt'>
  <td width=223 height=14 style='width:167.25pt;padding:0in 0in 0in 0in;
  height:10.5pt'>
  <p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
  line-height:10.5pt;mso-element:frame'><b><font size=1 color=gray face=Arial><span
  style='font-size:8.0pt;font-family:Arial;color:gray;font-weight:bold'>pbarrett@hoganassessments.com</span></font></b><font
  color=black><span style='color:black'><o:p></o:p></span></font></p>
  </td>
  <td width=277 height=14 style='width:207.75pt;padding:0in 0in 0in 0in;
  height:10.5pt'>
  <p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
  line-height:10.5pt;mso-element:frame'><b><font size=1 color=gray face=Arial><span
  style='font-size:8.0pt;font-family:Arial;color:gray;font-weight:bold'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
  </span></font></b><font color=black><span style='color:black'><o:p></o:p></span></font></p>
  </td>
 </tr>
 <tr height=19 style='height:14.25pt'>
  <td width=223 height=19 style='width:167.25pt;padding:0in 0in 0in 0in;
  height:14.25pt'>
  <p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
  mso-element:frame'><b><font size=3 color=black face=Arial><span
  style='font-size:12.0pt;font-family:Arial;color:black;font-weight:bold'><a
  href="http://www.hoganassessments.com/"
  title="http://www.hoganassessments.com/"><font size=1 color=gray><span
  style='font-size:8.0pt;color:gray;text-decoration:none'><span
  title="http://www.hoganassessments.com/">hoganassessments.com</span></span></font></a></span></font></b><font
  color=black><span style='color:black'><o:p></o:p></span></font></p>
  </td>
  <td width=277 height=19 style='width:207.75pt;padding:0in 0in 0in 0in;
  height:14.25pt'>
  <p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
  mso-element:frame'><font size=3 color=black face="Times New Roman"><span
  style='font-size:12.0pt;color:black'>&nbsp;<o:p></o:p></span></font></p>
  </td>
 </tr>
</table>

<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
margin-left:-5.25pt'><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt;color:black'>&nbsp;<o:p></o:p></span></font></p>

</div>

<p><font size=2 color=black face=Arial><span style='font-size:10.0pt;
font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<p><font size=1 color=black face=Arial><span style='font-size:9.0pt;font-family:
Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

<div>

<p class=MsoNormal><font size=1 color=black face=Arial><span style='font-size:
9.0pt;font-family:Arial;color:black'>&nbsp;<o:p></o:p></span></font></p>

</div>

</div>

</body>

</html>