[Rasch] development and validation of instrument

Mike Linacre (RMT) rmt at rasch.org
Sun Jun 15 05:45:44 EST 2008


Thank you for more questions, Juanito.

You wrote:
>We develop a 50-item instrument anchored on a theory with 4 constructs. We 
>piloted the instrument to 203 persons, then we subjected to Rasch 
>analysis.  After our Rasch analysis we found that the 50 items are 
>unidimentional and the 5-point likert scale categories are logically 
>functional, but while retaining the 4 constructs, five items misfit the 
>model.  Thus, we decided to remove the 5 items and come up with a 45 item 
>instrument.  Our questions to Rasch expert are:
>1. Shall we gather again another data (say n=200) to validate the instrument?

Reply: In your original analysis of the 50-item instrument, you discovered 
that 5 items were defective or off-dimensional in some way. So you want to 
omit them from the revised 45-item instrument. Fine!
Do you think that the responses to those 5 items have skewed the responses 
of your 203 persons to the other 45 items in any way? For instance, perhaps 
the 5 items were the first 5 items on the instrument and so could have 
caused the sample to misunderstand how they were supposed to respond to the 
remaining 45 items. If they would have distorted the responses to the other 
45 items, then you need to repeat the data collection with a new sample on 
the 45 item instrument. If not, then the current sample's data is valid for 
the 45 items. There is no need to repeat the data collection.

You wrote:
>2. How do Rasch validation is done? Do we just repeat the rasch analysis?

Reply: Yes, you would repeat the Rasch analysis but now with only the 45 
items. But remember that you have already excluded the 5 worst items, so 
the sensitivity of the Rasch analysis to problems in the remaining 45 items 
has increased. You will now find another group of "worst items". Don't 
automatically reject them to make a 40-item instrument, but examine the new 
set of worst items closely. Is their misfit really bad enough to distort 
the person measures?

You wrote:
>3. Is our sample big enough for Rasch analysis?

Reply: This depends on your purpose for this instrument, and how you are 
modeling the items. You have 45 items and each has a 5-point Likert scale. 
So the possible range of raw scores is 0 to 225 (or 45 to 270). So there 
are more possible raw scores (226) than your sample size (203). 
Consequently it is likely that you have few of your sample with very high 
or very low scores. If your instrument is intended to probe the full range 
of person performance, then you may need to collect data from more high or 
low performers to verify the operational range of the instrument. But if 
your sample of 203 persons matches the intended use of the instrument in 
all important respects, then your sample size is probably large enough, see 
http://www.rasch.org/rmt/rmt74m.htm - There are additional concerns if you 
are using the "Partial Credit" model.

OK?

Cordially,
Mike L.

Mike Linacre
Editor, Rasch Measurement Transactions
rmt at rasch.org www.rasch.org/rmt/ Latest RMT:  21:4 Spring 2008
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20080614/3ee09c4f/attachment.html 


More information about the Rasch mailing list