[Rasch] Rasch version to Churchill's Paradigm
david.andrich at uwa.edu.au
Fri Nov 5 12:06:09 EST 2010
Thomas. Thanks for the summary. Yes, very familiar in a range of areas.
David Andrich BSc, MEd (UWA), PhD (Chic) FASSA
david.andrich at uwa.edu.au<mailto:david.andrich at uwa.edu.au>
Graduate School of Education
The University of Western Australia
M428, 35 Stirling Highway,
Western Australia , 6009
Telephone: +61 8 6488 1085
Fax: +61 8 6488 1052
CRICOS Code: 00126G
Pearson Psychometric Laboratory<blocked::http://www.education.uwa.edu.au/ppl>
size=2 width="100%" align=center tabindex=-1>
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf Of Thomas Salzberger
Sent: Friday, 5 November 2010 6:29 AM
Subject: Re: [Rasch] Rasch version to Churchill's Paradigm
At 14:19 30.10.2010, Juanito Talili wrote:
Is there a Rasch version to Churchill's Paradigm for Developing Better Measures
As a marketing researcher I would say yes and no.
But first, I want to point out who Churchill is and what role his seminal paper of 1979 has played in marketing, as I assume only few will know him outside marketing. Up to the late 1970s measurement in marketing was predominantely based on single items or short scales which were taken at face value. Some sort of crude content validity if you like. During the late 1970s it began to dawn on the scientific community in marketing that validity has to be addressed emprically.
Churchill's contribution and some more by others, which were published around that time, have had a tremenduous influence on the way measurement is carried out in marketing.
Of course, the paper is somehow outdated by now but the basic principles arguably haven't changed. I think one can say that Churchill made classical test theory the state-of-the-art in marketing.
In fact, there are some reasonable elements in Churchill's 1979 paradigm, which have been watered down later on unfortunately (even by Churchill himself in subsequent versions of the paradigm published in text books). In 1979 Churchill made demands for multiple data collection during scale development and strongly advised against early item deletion based on factor analysis (in order to avoid capitalization on chance). He also said each item should have an equal amount of the core of the property to be measured. So he favoured parallel items. That could easily tested in factor analysis, but no one ever does it. He also asked for norms to facilite the interpretation of measures.
Today, people routinely apply exploratory factor analysis, delete items, then apply confirmatory factor analysis using the same data set and at the end of the day a bundle of SEM fit statistics demonstrate that everything is okay and measures are valid. And they get away with it.
Admittedly, the procedure has been refined, e.g. "advanced versions of reliability" are estimated, like composite reliability or average variance extracted.
In my mind, this changes very little, as the definition of measurement is still the one suggested by Stevens, manifest response scales are still treated as linear interval-scaled measures, sample-dependent statistics are used to assess precision and validity, etc.
Surprisingly, Rasch was proposed in marketing already in the 1980s (I know of two journal papers and one conference presentation). However, it sparked little interest at that time.
It is understandable given that Churchill's paper had been published only a few years before. These papers were well ahead of the times.
Since about 10 to 12 years ago several Rasch papers have been published. So, you may say there is an alternative paradigm around.
At least no one could actually say there would be no alternative.
However, there is a big qualification on that.
If you try to publish something in marketing using Rasch, you still encounter very strong resistance. I am sure that sounds familiar to researchers from other disciplines.
When you apply Rasch analysis, you always have to explain everything from first principles and you have to justify why you use Rasch instead of factor analysis.
I think it is really bizarre: use confirmatory factor analysis, use 2 to 3 items for a latent variable (I recently read an article featuring a study comprising 10 latent variables and a total of just 32 manifest variables - and some of these were very questionable in terms of content validity) - and you get away with it. Use a much more sophisticated paradigm, and you always have to justify and defend yourself.
Ironically, if you do justify and explain everything in detail, then reviewers come up with things like "not focused", "I don't believe this", "I cannot find the factor loadings", "no contribution" etc.
In a recent submission on gender-based DIF, we applied Rasch measurement. We pointed out that Rasch is superior compared to confirmatory factor analysis based on theoretical as well pragmatic reasons. As space was limited, we couldn't provide a comprehensive comparison of Rasch versus CFA; and I do not see the point as this has been done before. But simply citing other papers is not enough. And on top of that, one reviewer explicitely asked not only for a comparison but for a specific result of that comparison: "don't turn CFA down". So much to marketing SCIENCE.
I also believe most editors of leading journals actually do not want to see critical contributions of this sort.
This becomes obvious by the way reviewers are chosen. Each time, we nominated a handful of suitable reviewers, well respected scholars knwoledgable of Rasch - but not a single one was ever chosen as a reviewer.
Recently we had three reviewers and each of them explicitely said that they do not have a clue about Rasch measurement. This perverts the whole review process as you do not get any valuable feedback. Just poor nonsense.
Anyhow, I still hope that the situation will change in the not-too-distant the future. There are many young researchers who are much more open-minded.
On the MBC website www.matildabayclub.net<http://www.matildabayclub.net/> I posted a list of publications on Rasch in marketing (follow Downloads and links and then choose the item in the menu at the bottom).
Or go directly to (then you do not have the outer frames, though):
If anyone knows of other Rasch papers in the Marketing/Business area, I would highly appreciate any hint.
A Paradigm for Developing Better Measures of Marketing Constructs
Gilbert A. Churchill, Jr.
Journal of Marketing Research, Vol. 16, No. 1. (Feb., 1979), pp. 64-73.
Rasch mailing list
Rasch at acer.edu.au
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch