[Rasch] A practical one

Looveer, Juho Juho.Looveer at det.nsw.edu.au
Mon Oct 10 16:20:31 EST 2005


As Trevor has said, what is the point of the test?  

If it is to achieve competence (e.g. flying an aircraft, etc), then the
test can be part of the learning experience.  By having failed on some
aspects of the test previously, the testee has gone away and
learned/studied/practiced that aspect, and is now competent on it.
Objective achieved - that person has more competence than someone who
cannot demonstrate their skill in that area.

In most academic tests, students memorise material.  Even a doctor will
have practiced many medical routines (hopefully on cadavers) before they
are let loose on live patients.
In academic tests, students will usually have undertaken practice tests
or assignments, as part of their preparation.

But, that is also what happens in many other situations - a driving test
will each time check that the candidate can undertake a certain minimum
set of core competencies. For a piano exam, everyone knows beforehand
what pieces they will have to play and what skills they need to
demonstrate.


Is the purpose to assess whether someone has achieved some
knowledge/skill/understanding, or to assess who can achieve this in the
least number of attempts.
In this case, are we really assessing the competence/skill, or aptitude
for the skill (ie a combination of the skill and number of attempts at
the test)?


Is the problem the use of the same question in successive tests? 
This could then be a defect in test design - where some candidates are
given a "step up" or an unfair advantage.  This is why Computer Adaptive
Testing relies on a large pool of items - trying to avoid candidates
getting an unfair advantage of knowing the questions they will see.



Dr Juho Looveer 
Sydney NSW 
work phone: 956 18192 
fax: 956 18055 
Juho.Looveer at det.nsw.edu.au 


-----Original Message-----
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On
Behalf Of John Barnard (EPEC)
Sent: Monday, 10 October 2005 2:59 PM
To: 'Trevor Bond'; 'Rasch list'
Subject: RE: [Rasch] A practical one

Of course Trevor, but let's take it from another angle. If a person has
a copy of a paper or can remember some questions and thus answer some
questions correctly in spite of not having the knowledge (but having
seen the questions) it is a different story. What about a person who
passes a test because of "memorising" some answers? Or in your metaphor,
a person who didn't clear a height but stands on a step in the second
attempt now clears the height - is this fair to those who didn't use the
step?

Hopefully the purpose of education is not merely rote learning and
memorisation.

Kindly
John 

John J Barnard
Executive Director: EPEC Pty Ltd
www.users.bigpond.com/JohnBarnard/

DISCLAIMER:
The contents of this e-mail which may include one or more attachments,
is confidential and is intended for the use of the named recipient(s).
If you have received this e-mail in error, you are not permitted to and
must not disclose, distribute or retain it, and are requested to notify
the sender by return e-mail and delete it thereafter.

It is the responsibility of the recipient(s) to ensure that the e-mail
is virus free. Although EPEC uses the latest antiviral software, we do
not accept responsibility for any problems caused by viruses. 

-----Original Message-----
From: Trevor Bond [mailto:trevor.bond at jcu.edu.au]
Sent: Monday, 10 October 2005 2:28 PM
To: John Barnard (EPEC); 'Rasch list'
Subject: RE: [Rasch] A practical one


Thanks John,

this requires us to reflect on the whole nature of educational (and 
other) testing. High jumpers don't find it easier to clear heights 
just because they have failed them (been exposed to them) in the 
past. If mere exposure to the test improves scores: What are we 
actually testing? and, What is the purpose of education?
best
T


At 2:12 PM +1000 10/10/05, John Barnard \(EPEC\) wrote:
>Thanks for your reply Trevor. Of course we would expect (at least hope
>for) improvement in students' performance over time. However, think of 
>this as a test for say pilots to qualify (just for argument's sake). It

>is thus a type of selection test rather than a "scholastic achievement"

>test where one would expect some growth. (It is thus rather a 
>qualifying test and I know one can reason that those who failed could 
>have put in an extra effort this time.) The hypothesis is that the 
>repeat group was unfairly advantaged (competing for the same places) 
>because they have seen the items before. Through what I have done so 
>far, this seems to be the case. But now I want to account for this, 
>i.e. trying to be fair to those who have not seen the items before.
>
>If this was not the case (and there was no additional effort) one would

>expect the repeat group to have approx the same mean ability (if same 
>item difficulty estimates are used) in the two sessions. But, the mean 
>ability of the repeat group increased significantly.
>
>Hope this clarifies the issue a little.
>
>Kindly
>John
>
>John J Barnard
>Executive Director: EPEC Pty Ltd www.users.bigpond.com/JohnBarnard/
>
>DISCLAIMER:
>The contents of this e-mail which may include one or more attachments, 
>is confidential and is intended for the use of the named recipient(s). 
>If you have received this e-mail in error, you are not permitted to and

>must not disclose, distribute or retain it, and are requested to notify

>the sender by return e-mail and delete it thereafter.
>
>It is the responsibility of the recipient(s) to ensure that the e-mail 
>is virus free. Although EPEC uses the latest antiviral software, we do 
>not accept responsibility for any problems caused by viruses.
>
>-----Original Message-----
>From: Trevor Bond [mailto:trevor.bond at jcu.edu.au]
>Sent: Monday, 10 October 2005 1:45 PM
>To: John Barnard (EPEC); Rasch list
>Subject: Re: [Rasch] A practical one
>
>
>Dear John
>
>Perhaps we (or, I, at least) need some more information to understand 
>your problem. Kids who repeated an exam (after doing make up work or 
>just remembering) score better on a test the second time round. They 
>scored more correct responses at T2 than at T1. This seems to be 
>exactly what I would expect . . .or hope for . . .as an educator. 
>Clearly I have missed the nature of your 'problem'. collegially
>Trevor
>
>At 1:32 PM +1000 10/10/05, John Barnard (EPEC) wrote:
>>Dear all
>>
>>I would appreciate some opinions on the following scenario.
>>
>>The same paper was given on two occasions. Some 20% of students (let's

>>call them the repeaters) who sat the first round (and failed) also sat

>>the second round. In round 1, the repeaters' mean ability is (say) 0.5

>>logits less than the non-repeaters'. After anchoring the item 
>>difficulties in the first round and using them in round 2 resulted in 
>>the repeaters now doing significantly better (say 0.7 logits on 
>>average). Also the repeaters now have approx the same mean ability as 
>>the non-repeaters in round 2, say 0.8 logits.
>>
>>The question is this: If the repeaters' mean ability increased by 0.7 
>>logits, how can one account for this (taking the same paper again) to 
>>not unfairly advantage the repeat group in round 2? (I am aware of 
>>learning and other factors, but let's ignore that for the moment.)
>>
>>Kindly
>>John
>>
>>John J Barnard
>>Executive Director: EPEC Pty Ltd www.users.bigpond.com/JohnBarnard/
>>
>>DISCLAIMER:
>>The contents of this e-mail which may include one or more attachments,

>>is confidential and is intended for the use of the named recipient(s).
>  >If you have received this e-mail in error, you are not permitted to 
> and
>
>>must not disclose, distribute or retain it, and are requested to 
>>notify
>
>>the sender by return e-mail and delete it thereafter.
>>
>>It is the responsibility of the recipient(s) to ensure that the e-mail

>>is virus free. Although EPEC uses the latest antiviral software, we do

>>not accept responsibility for any problems caused by viruses.
>>
>>
>>
>>_______________________________________________
>>Rasch mailing list
>>Rasch at acer.edu.au http://listserv3.acer.edu.au/mailman/listinfo/rasch
>
>
>--
>Trevor G BOND Ph D
>Professor and Head of Dept
>Educational Psychology, Counselling & Learning Needs
>D2-2F-01A EPCL Dept.
>Hong Kong Institute of Education
>10 Lo Ping Rd, Tai Po
>New Territories HONG KONG
>
>Voice: (852) 2948 8473
>Fax:  (852) 2948 7983
>Mob:


-- 
Trevor G BOND Ph D
Professor and Head of Dept
Educational Psychology, Counselling & Learning Needs
D2-2F-01A EPCL Dept.
Hong Kong Institute of Education
10 Lo Ping Rd, Tai Po
New Territories HONG KONG

Voice: (852) 2948 8473
Fax:  (852) 2948 7983
Mob:





_______________________________________________
Rasch mailing list
Rasch at acer.edu.au
http://listserv3.acer.edu.au/mailman/listinfo/rasch
**********************************************************************
This message is intended for the addressee named and may contain
privileged information or confidential information or both. If you
are not the intended recipient please delete it and notify the sender.
**********************************************************************



More information about the Rasch mailing list