[Rasch] Re:Urgent question about high stakes and normality
ici_kalt at yahoo.com
Mon Jan 30 02:17:23 EST 2006
Hi Iasonas: I can imagine two scenarios:
Scenario 1. Tests for different profiles of students. In this case you have a set of tests with different attributes according to a profile or speciality:
Profile 1. Physics and Chemistry. Students have to respond a test with thess modules: Math, Organic
Chemistry, Dynamics, English.
Profile 2. Socio-economics. Students have to respond these modules: Math, Statistics, History and English.
Profile 3.... and so forth.
A student has to pass only one profile, as they are candidates to different specialities or schools.
In a profile you are using "shared" modules, like Math and English that are used in two or more profiles. They are not anchor tests, but modules or blocks that have to be included as part of the profile, but under some conditions you may use the blocks as anchors.
In this case, you get results from students in each module and also you have the total result (when I say result I can imagine the measure obtained from the Rasch model, you have measures for each module and a measure of the total of the test). If this is true, I think you can report your results as they come from your analysis, and no need to add the scores or results of the parts to get a total. Trying to be clear:
Profile 1: Math (20 items), Organic Chemistry (40 items), Dynamics (25 items) and English (50 items), all the Test (20+40+25+50 = 135 items). If you analyse to score the students, you will have a Rasch measure in Math (say 0.5) , another measure in O.C. (say 0.8), another in Dynamics (say 1.3) and another in English (say 0.0), and also another for the total (say 0.9). As you see the total 0.9 is not the addition of the parts, but you don't need to add this.
By the way, if you get the normal Z scores of each student in each part, the Z value for all the test is not the addition of the Z of the blocks, and a mean score may probably provide wrong interpretations.
In this case you may order your students and make decisions per profile as they are participating in different schools. You will have problems if you wish to get a general ordered listing of all the students, as you're using different measurement instruments per profile.
Scenario 2. Testing in blocks. Let's imagine you're responsible of a new educational program in Manchester, including new didactic materials, new curriculum, a set of video-games, activities and TV shows. You need a pre and post test to have "a picture" of the students with and without your educational program. As it is very difficult, expensive and time consuming to apply a test with all the items for all the educational areas, you are producing a test made with blocks: Math, Organic Chemistry, Dynamics, English, Statistics and History. Each student will respond a version including 3 of these blocks, in such a way that all the blocks will be tested, but there is no students answering all the blocks. If this is the case, you will not have a problem to get measures of the students in the block they have worked and you will not have difficulties to analyze each block and the items included in it.
When this kind of test is used for a nationwide evaluation, you will not need the result per student. The problem is if you wish to have a comparative result among students.
The problem in this case is how to get a total result for a student if you wish to say how much does he know in
the blocks he didn't answer, just using the information provided by the blocks he really answered.
We have some discussions in Mexico and other countries related to that. We are facing two options:
a) To produce an equated result between students of different versions to get an overall result independently of the blocks he/she has worked.
b) To provide the measure of the version he/she has worked without equating.
In both cases you don't have to add the Z values of the equated measures.
The other possibilities regard some extrapolations to imagine what is the measure in History of a student, if he doesn't take the History block, just knowing his results in the other blocks. We think this is quite hard to believe, at least for us. If someone has a solution in this listserve please let us know.
Hope this helps.
FAMILIA DE PROGRAMAS KALT.
Mariano Jiménez 1830 A
Col. Balcones del Valle
78280, San Luis Potosí, S.L.P. México
TEL (52) 44-4820 37 88, 44-4820 04 31
FAX (52) 44-4815 48 48
web page (in Spanish AND ENGLISH): http://www.ieesa-kalt.com
What are the most popular cars? Find out at Yahoo! Autos
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch