[Rasch] Re:Urgent question about high stakes and normality

iasonas lambrianou liasonas at lycos.com
Mon Jan 30 04:59:56 EST 2006

Thank you Mike and Agustín.

Indeed, pretending to be able to predict a student's performance in
History using his score in English and Latin is rediculuous. And I do not
think that this could stand in the court as well. Human nature is too
complex and our models (including our favourate, Rasch model) seem to be
too simplistic to predict it (the human nature). Also, we know that
students sometimes pay more attention on one test and study less for
another test: so its not like height or weight, performance on different
tests does not follow our 'physical rules'.

Having said that, I spent too much time talking to people from different
countries. It seems that, at least in Australia, Europe, Singapour etc
the people demand that we need to leave the students free to choose to be
tested on their favourate subjects, but then we need to report and use
'overall scores'. In many countries people use the t-scores and then they
aggregate them in various ways. I would have hoped that somebody might be
able to suggest a better way.

Would it be a good idea to contact all those people and then write a few
short papers describing our case studies and try to find out a solution
to this problem? Or maybe we should just try to convinvce the people and
the politicians that no statistical models can fix problems caused by
irrational political decisions and bad practice?

 Things become even worse when people run Rasch analysis using 50 cases
when their rating scales range from 1-100. They do not even have 1 person
per scale point. By the way, Mike, how would you comment on that?


  ----- Original Message -----
  From: "Agustin Tristan"
  To: "iasonas lambrianou" , "-Rasch"
  Subject: Re: [Rasch] Re:Urgent question about high stakes and
  Date: Sun, 29 Jan 2006 07:17:23 -0800 (PST)

  Hi Iasonas: I can imagine two scenarios: Scenario 1. Tests for
  different profiles of students. In this case you have a set of tests
  with different attributes according to a profile or speciality:
  Profile 1. Physics and Chemistry. Students have to respond a test
  with thess modules: Math, Organic Chemistry, Dynamics, English.
  Profile 2. Socio-economics. Students have to respond these modules:
  Math, Statistics, History and English.    Profile 3.... and so forth.
  A student has to pass only one profile, as they are candidates to
  different specialities or schools.In a profile you are using "shared"
  modules, like Math and English that are used in two or more profiles.
  They are not anchor tests, but modules or blocks that have to be
  included as part of the profile, but under some conditions you may
  use the blocks as anchors.In this case, you get results from students
  in each module and also you have the total result (when I say result
  I can imagine the measure obtained from the Rasch model, you have
  measures for each module and a measure of the total of the test). If
  this is true, I think you can report your results as they come from
  your analysis, and no need to add the scores or results of the parts
  to get a total. Trying to be clear: Profile 1: Math (20 items),
  Organic Chemistry (40 items), Dynamics (25 items) and English (50
  items), all the Test (20+40+25+50 = 135 items). If you analyse to
  score the students, you will have a Rasch measure in Math (say 0.5) ,
  another measure in O.C. (say 0.8), another in Dynamics (say 1.3)  and
  another in English (say 0.0), and also another for the total (say
  0.9).   As you see the total 0.9 is not the addition of the parts,
  but you don't need to add this.By the way, if you get the normal Z
  scores of each student in each part, the Z value for all the test is
  not the addition of the Z of the blocks, and a mean score may
  probably provide wrong interpretations.In this case you may order
  your students and make decisions per profile as they are
  participating in different schools. You will have problems if you
  wish to get a general ordered listing of all the students, as you're
  using different measurement instruments per profile. Scenario 2.
  Testing in blocks. Let's imagine you're responsible of a new
  educational program in Manchester, including new didactic materials,
  new curriculum, a set of video-games, activities and TV shows. You
  need a pre and post test to have "a picture" of the students with and
  without your educational program. As it is very difficult, expensive
  and time consuming to apply a test with all the items for all the
  educational areas, you are producing a test made with blocks: Math,
  Organic Chemistry, Dynamics, English, Statistics and History. Each
  student will respond a version including 3 of these blocks, in such a
  way that all the blocks will be tested, but there is no students
  answering all the blocks. If this is the case, you will not have a
  problem to get measures of the students in the block they have worked
  and you will not have difficulties to analyze each block and the
  items included in it.When this kind of test is used for a nationwide
  evaluation, you will not need the result per student. The problem is
  if you wish to have a comparative result among students.The problem
  in this case is how to get a total result for a student if you wish
  to say how much does he know in the blocks he didn't answer, just
  using the information provided by the blocks he really answered.We
  have some discussions in Mexico and other countries related to that.
  We are facing two options:a) To produce an equated result between
  students of different versions to get an overall result independently
  of the blocks he/she has worked.b) To provide the measure of the
  version he/she has worked without equating.In both cases you don't
  have to add the Z values of the equated measures. The other
  possibilities regard some extrapolations to imagine what is the
  measure in History of a student, if he doesn't take the History
  block, just knowing his results in the other blocks. We think this is
  quite hard to believe, at least for us. If someone has a solution in
  this listserve please let us know. Hope this helps.Agustín Tristan

  FAMILIA DE PROGRAMAS KALT. Mariano Jiménez 1830 A Col. Balcones del
  Valle 78280, San Luis Potosí, S.L.P. México TEL (52) 44-4820 37 88,
  44-4820 04 31 FAX (52) 44-4815 48 48 web page (in Spanish AND
  ENGLISH): http://www.ieesa-kalt.com


  What are the most popular cars? Find out at Yahoo! Autos
  Rasch mailing list
  Rasch at acer.edu.au

Dr. Iasonas Lamprianou
CFAS, School of Education
The University of Manchester
Oxford Road, Manchester M13 9PL, UK
Tel. 0044  161 275 3485
iasonas.lamprianou at man.ac.uk


Search for businesses by name, location, or phone number.  -Lycos Yellow Pages


-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20060129/475f59cb/attachment.html 

More information about the Rasch mailing list