[Rasch] Rasch & validity
ici_kalt at yahoo.com
Thu Jan 27 00:24:55 EST 2011
In addition to the properties mentioned by Professors Svend and Bond, we´re imposing the need of having items covering the range of expected measures and uniform distribution of difficulties centered on 0. Centering is automatically done by Winsteps for instance, but the other two properties need to produce sufficient items to cover the whole range of difficulties: we do not need much more items, we need well distributed items to measure all the persons.
This need a small explanation:
We have used a depression scale to measure a group of High School students.
All items and persons fitted the Rasch model? YES
The trait was correlated with depression? according to the producer of the test YES
Does we find DIF? NO
nevertheless the majority of the students had measures out of the interval of difficulties of the items. It is clear that the Rasch model could get measures for people, but many share the same measure and a high std error. The question is if the measure we got is valid to say something to the student.
This question is related to the Messick's need to have valid interpretations from data (how can we make good interpretations of the measures not having items for the non-pathological students?) and Borsboom who returns back to basics regarding the purpose of the test (how can we distinguish depression among all those students having the same measurement? it is evident that they should have different depression measures but the test does not permit to measure those differences, i.e. it does not fit its purpose).
In fact the original test could be valid to measure depression for pathological cases, NOT for non-pathological persons. In conclusion that we needed more items to cover the range of persons' measures if we need to get a measure of depression of young students not having a pathology (unless I think that depression is a dichotomy, then I do not need to get measures but a pass-fail testing machine, making useless to have the original scale for the pathological cases).
It is the same situation if you wish to use an industrial thermometer for kilns in order to measure body temperature of persons: it is not a problem of reliability but validity of the scale you're using. The concept of scale validity, and what we called the "test design line" was treated elsewhere, see the link below
Hope this helps.
INSTITUTO DE EVALUACION E INGENIERIA AVANZADA.
Ave. Cordillera Occidental No. 635
Colonia Lomas 4ª San Luis Potosí, San Luis Potosí.
C.P. 78216 MEXICO
(52) (444) 8 25 50 76 / (52) (444) 8 25 50 77 / (52) (444) 8 25 50 78
Página Web (en español): http://www.ieia.com.mx
Web page (in English): http://www.ieesa-kalt.com/English/Frames_sp_pro.html
--- On Wed, 1/26/11, Svend Kreiner <S.Kreiner at biostat.ku.dk> wrote:
From: Svend Kreiner <S.Kreiner at biostat.ku.dk>
Subject: Re: [Rasch] Rasch & validity
To: rasch at acer.edu.au
Date: Wednesday, January 26, 2011, 5:32 AM
It depends on your definition of validity. I subscribe to Paul Rosenbaum's (Psychometrika, 1989)definition of criterion related construct validity requiring unidimensionality, monotonocity, local independence and no DIF. For this reason, measurement by items from Rasch models is valid because the Rasch models meet all these requirements.(plus, of course, a number of other requirements that we think are important - Rasch measurement is more than valid, it is also objective and sufficient).
If you think about criterion validity and/or Cronbach & Meehl's definition of construct validity you would not immediately think about Rasch models, but it follows from Rosenbaum's results that if items follow a Rasch model and if X is known to be positively related to the latent trait, then X will also be positively related to the score. If X is a criterion variable and X is not associated to the total score then it follows that items cannot fit the Rasch model.
Den 26-01-2011 11:37, Anthony James skrev:
Validity is defined as "the degree to which a test measures what it is supposed to measure".
How can fit to the Rasch model be evidence of validity?
Can't we have a say, anxiety scale that perefctly fits the Rasch model but does not actually measure anxiety?
How does Rasch analysis guarantee that a test actually test what it should test?
Rasch mailing list
Rasch at acer.edu.au
Department of Biostatistics
University of Copenhagen
Øster farimagsgade 5, entr. B
P.O. Box 2099
DK-1014 Copenhagen K, Denmark
Email: S.Kreiner at biostat.ku.dk
Phone: (+45) 35 32 75 97
Fax: (+45) 35 32 79 07
-----Inline Attachment Follows-----
Rasch mailing list
Rasch at acer.edu.au
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Rasch