RE: [Rasch] Rating Scale or Partial Credit?
tsnider-lotz at previsor.com
Thu May 18 01:15:23 EST 2006
Thanks for your answer. I am digesting your comments, as well as Mike Linacre's.
My mention of "consistent" thresholds just meant that, like most test developers, the people who wrote these items probably never thought about thresholds, but if they had, they would have assumed that each of the 5 options would have been used by a reasonable number of test takers, and that the thresholds would have been in the proper order. In other words, there was no formal requirement for consistent or orderly thresholds, only an implicit (and maybe never-conscious) assumption that they'd produced a textbook example of a well-constructed test.
In Rasch terms, if they'd thought about it, they would have assumed they'd created a test that would fit neatly within the rating-scale model.
My question-within-a-question was, is it legitimate to use the partial predit model simply because the data don't do well in the rating-scale model? I was concerned that this would constitute "model shopping," or something like researchers who go to a 2PL or 3PL model because they don't like how a one-parameter model fits their data.
From: Stephen.Humphry at det.wa.edu.au [mailto:Stephen.Humphry at det.wa.edu.au]
Sent: Wednesday, May 17, 2006 3:25 AM
To: Snider-Lotz, Tom; rasch at acer.edu.au
Subject: RE: [Rasch] Rating Scale or Partial Credit?
Tom, it depends on the reason you constructed the items with the intention of producing 'consistent' thresholds, by which I assume you mean that each threshold has the same distance from the central (item) location for every item. Unless it is critical to your research objectives that the thresholds are consistent, I would look at the evidence to try to ascertain whether there is better fit to the model when each item is allowed to have different (centralised) thresholds, given the additional parameters. I would look at any evidence available about fit including graphical information such as ICCs. Generally speaking, I would be more concerned about which collection of items measure the latent trait(s) than whether the thresholds are necessarily consistent, although it depends on your objectives.
Disordered thresholds indicate a problem with the data elicited by the relevant items, because the formal structure of the model entails a latent Guttman response subspace, which in turn entails ordered thresholds (Andrich, 1978, 2005). Integer scoring is a classification process in which a score of x implies that: (i) the x lowest thresholds are exceeded and, simultaneously, that; (ii) the m - x highest thresholds are not exceeded (where m is the maximum score for the item). It is always a hypothesis that the thresholds are ordered given the nature of response categories comprising an item, and it is empirically possible for the threshold estimates not to be in their natural order; i.e. the hypothesis is refutable.
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 357-74.
Andrich, D. (2005). The Rasch model explained. In Sivakumar Alagumalai, David D Durtis, and Njora Hungi (Eds.) Applied Rasch Measurement: A book of exemplars. Springer-Kluwer. Chapter 3, 308-328.
Stephen Humphry, PhD
Senior Educational Measurement Officer, Psychometrics
Department of Education & Training
151 Royal Street, East Perth, 6004
Phone: +61 8 92644102
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf Of Snider-Lotz, Tom
Sent: Wednesday, 17 May 2006 8:59 AM
To: rasch at acer.edu.au
Subject: [Rasch] Rating Scale or Partial Credit?
Hello everyone --
I am analyzing data from a personality instrument that employs 5-option Likert-style items. Inspection of the results shows that the items vary greatly in terms of how well-spaced and well-ordered their thresholds are. Should I be using a partial credit model, because empirically I know the thresholds are different for each item? Or should I use a rating scale model, because the items were created with the intention of producing consistent thresholds?
-- Tom Snider-Lotz
Thomas G. Snider-Lotz, Ph.D.
1805 Old Alabama Road
Roswell, GA 30076
Ph: 800-281-9713 x555
tsnider-lotz at previsor.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch