[Rasch] fit analysis / rating scale dilemma (please help)
ama at UWasa.fi
Tue Jun 6 06:08:19 EST 2006
The question of applicable rating scale might also depend on the
characteristics of the modelling approach. Some decision making or
marketing research study methods utilise double ended rating scales. In
these the purpose is often create modelling premise for compromises,
relationships, or priorities between alternatives.
The evaluation of the fittness or statistical quality of the data and
models are respectively based on a different set of methods. For example,
in decision making research where AHP (Analytic hierarchy process) is
often used, the fittness statistics focus on internal consistency of the
ratings, even from small number of respondents or groups. These statistics
are not based on large sample sizes such as in factor analysis.
For my own research I used -7 ..+7 scale, as a way to model balance
between two conflicting outcome categories, where managerial balance in
decision making is theoretically applicable as a modelling approach.
Analysis of the data covered small group interviews from 5-13 people
We do currently some research work in a research group developing new
analysis tools for modelling of managerial interpretation/ratings, where
this kind of double end anchored rating scales are applied. Development
of the fittness statistics and critical evaluation rules are also addressed
in this study.
In our research approach, we also try to find complementary perspectives
between methods. IE. we can utilise a widely tested questionnaire
from educational/leadership research, which we then re-formulate into
another kind of questionnaire, where main model dimensions of the
previous method are used as guidelines for the development of new
questions. Then the idea is to find alternative aspects of a same
phenomena with different kind of questionnairres, but based on similar
core concepts, but modelled in different ways.
In the work carried out so far, the middle point on a rating scale has
been included as an alternative, but the evaluation of the data is based
on a modified calculation of the inconsistency ratio.
I would be interested in discussing related questions and literature,
where described characteristics of rating scales and their fittness
analysis are addressed.
with best regards,
On Sun, 4 Jun 2006, Michael Lamport Commons wrote:
> RE: [Rasch] fit analysis / rating scale dilemma (please help)I always use 4 or 6 point scales for that reason. Once in a while when I am interested in probabilities that are list as probits, I use a 7 pint scale. The middle probability is .5. Maybe I should not even do that.
> ----- Original Message -----
> From: Stone, Gregory
> To: Mary A Coleman/AC/VCU ; rasch at acer.edu.au
> Sent: Sunday, June 04, 2006 7:53 PM
> Subject: RE: [Rasch] fit analysis / rating scale dilemma (please help)
> There is NEVER any justification for "No opinion" as a middle point on a rating scale. If indeed the respondent has no opinion, then they will not, cannot or choose not to participate in the data gathering process. It is ironic that after so much evidence supporting the use of even numbered rating scales and the idea that "no opinion" is not a theoretically justifiable midpoint, this continues to arise. Why would no opinion be the theoretical midpoint of a scale? Even if the scale ran from say "strongly disagree to strongly agree" a middlepoint would still not suggest the lack of an opinion. While some have considered "no opinion" to mean "half and half" that is dubious. Either way, it fails to assist us in understanding their likes, beliefs, etc. as it provides us with no difference.
> The only justification for its use is as a comfortable "out" for those who cannot, will not or choose not to answer the question. For those who wish to give participants this out, it must be done with the understanding that it will be considered as missing data during the analysis.
> The justification you seek is not necessary. What IS necessary is for those who continue to promote this notion to justify, explain and defend the theoretically unjustifiable opinion that the lack of a feeling/belief/opinion is the midpoint along a scale ranging from very negative to very positive, etc. We work so hard to create constructs within Rasch and they are defined so beautifully. Should we not expect the same level of clarity from the rating scale?
> Gregory E. Stone, Ph.D., M.A.
> Assistant Professor of Research and Measurement
> The University of Toledo, Mailstop #923
> Toledo, OH 43606 419-530-7224
> Editorial Board, Journal of Applied Measurement www.jampress.org
> For information about the Research and Measurement Programs at The University of Toledo and careers in psychometrics, statistics and evaluation, email gregory.stone at utoledo.edu.
> -----Original Message-----
> From: rasch-bounces at acer.edu.au on behalf of Mary A Coleman/AC/VCU
> Sent: Sun 6/4/2006 11:15 AM
> To: rasch at acer.edu.au
> Subject: [Rasch] fit analysis / rating scale dilemma (please help)
> I am analyzing a 5 question, 5 point likert-type scale where 3 is labeled no opinion (N= 1054). There is no missing data.
> Mean square fit statistics for items are good, but the standardized stats for three items are bad (outfit standardized MNSQs of 3.8, -2.9, -5.4). I have ordered threshold/structure measures and the category probability curves look nearly perfect. When I treat the 3 category as missing, the fit stats improve ( .3, -2.6, -4.2), and the structure measures/category probability curves don't suffer.
> I've read the literature which justifies treating the "No opinion" category as missing, but generally, it is reported only within the context of disordered structure measures.
> I guess I am seeking some feedback/justification for treating "no opinion" responses as missing data given that the scale seems to be functioning or for keeping the item with a standardized outfit of 3.8.
> Mary Angela Coleman
> Virginia Commonwealth University
> colemanma at vcu.edu
> Rasch mailing list
> Rasch at acer.edu.au
More information about the Rasch