[Rasch] Bunched versus spread out items to measure depression

Paul Barrett paul at pbarrett.net
Mon Jul 25 09:14:21 EST 2011


Agustin

 

You stated “We have used the Beck depression questionnaire (and also the suicide scale and other scales), that are supposed to correctly measure depression (or suicide), but what we have found is that the Beck scale is satisfactory only for "depressed" people. But if we use it with what we could call "normal or non depressed " persons, then test is useless. 

Do I have to wait that my young students become depressed if we want to detect them with the Beck questionnaire?

Do I have to wait that my car tank is empty to decide that I have to go to the pump station?

 

The persons distribution, and the SE may say something but only if I am in a certain portion of the scale, because the questionnaire does not have items covering all the possible states from "normal" to pathologically depression. Ben Wright suggest to have items covering all the scale and not only a portion, unless I want to use the scale on depressed persons. ”

 

The problem with your reasoning is that you are assuming a scale can be constructed to “measure” something which does not exist.

 

That is, you wish the scale to work on people who do not possess a magnitude of  “depression”. As Michael said, they simply are not depressed; there is nothing within them to be assessed as “depression”. You are attempting to ‘measure” the absence of something.

 

Whether or not Ben Wright stated something paraphrased by you as “Ben Wright suggest to have items covering all the scale and not only a portion, unless I want to use the scale on depressed persons”, it is wrong-headed –IF- the attribute in question does not vary quantitatively. 

 

Finding that out is not something Rasch scaling can do for you. Rasch IRT will scale anything put before it, if there is sufficient stochastic noise in the data and some ad-hoc probability relations satisfying the mechanics of the model, without regard to meaning. 

 

So, you have to first establish that whatever attribute you have in mind does actually vary quantitatively, then fit a suitable mathematical function to observations that best charactize the underlying quantitative variation.

 

The only people I’ve seen do this properly are Jack Stenner and colleagues at Metametrics Inc. First they did the hard experimental yards to establish empirically that the attribute under examination did look to be varying quantitatively, then the math function-fit to set up routine near-quantitative measurement.

 

Maybe depression really is best represented as a quasi-categorical fuzzy-order … many people at any particular point in time will not be depressed, some will be a bit depressed, some pretty-depressed, some really depressed, some suicidal. To expect to specify “magnitude of depression to even 1 decimal place  seems ‘adventurous’. Why? Because how will you empirically show the difference in someone who is 1.25 depressed vs 1.35? Sure, we can do this with length, by concatenating objects. We can even do it with derived variables. But psychological attributes? Where is the precision coming from? The manipulation of numbers or the manipulation of “units of depression”? They are not necessarily the same.

 

Regards .. Paul

 

P.S. As an aside  ... As you may imagine, I dumped IRT a long time ago, along with all the latent variable hoopla, and most complex quantitative-assumption-laden analysis methods, except where I am prepared to use numbers in the full knowledge it is for pragmatic convenience. I find it subverts the clarity of thinking required in the scientific and measurement tasks of both basic and applied psychology. However, educational assessment is another matter altogether – and IRT methods may suit the more pragmatic requirements of educational examination item scaling.

 

We now live in a post-Michell world. Within the science of psychology, you can either ignore the facts about measurement and play ‘pretends’, or get on with inventing/making useful assessments which pack the pragmatic punch you may need, but recognizing that none of it is actually quantitative until emprical work with the attribute demonstrates those quantitative relations. 

 

 

Advanced Projects R&D Ltd.

__________________________________________________________________________________

W:  <http://www.pbarrett.net/> www.pbarrett.net 

E:  <mailto:paul at pbarrett.net> paul at pbarrett.net 

M: +64-(0)21-415625

 

 

From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf Of Michael Lamport Commons
Sent: Monday, 25 July 2011 2:33 a.m.
To: Agustin Tristan; Rasch
Subject: Re: [Rasch] Bunched versus spread out items to measure depression

 

But most of the participants were not depressed at all!  So I do not know what to make of your comment---

--Original Message----- 
From: Agustin Tristan 
Sent: Jul 24, 2011 3:05 AM 
To: Michael Lamport Commons , Rasch 
Subject: Re: [Rasch] Bunched versus spread out items to measure depression 


Hello!

Sometimes I would like to have a better way to explain myself in English and try to transmit my ideas regarding the problem you're facing actually, I hope this time I shall be clear enough.

The idea of the design of a test is important for everyone and validity is the main and first property for many of us. Nevertheless I think we are not following a former idea by Ben Wright that is presented in several documents concerning the item distribution and all the implications this simple concept has regarding the other characteristics of measurement.

 

An investigator produces a test with N items for a specific construct (depression in this case), using partial credit, dichotomous or any other kind of items. He uses the questionnaire with M persons and passes the responses through the Rasch model using a good software (independently of the brand: Winsteps, Rumm, Rascal, your own software...) and he gets measures that are intended to be in a linear scale. The distribution of the measures of the persons or the individual measures will be interpreted according to the scale and the item measure will give some interpretation concerning depression in this case. This is ok...wait! is this ok? what is the problem with this? It is related to validity.

 

We have used the Beck depression questionnaire (and also the suicide scale and other scales), that are supposed to correctly measure depression (or suicide), but what we have found is that the Beck scale is satisfactory only for "depressed" people. But if we use it with what we could call "normal or non depressed " persons, then test is useless. 

Do I have to wait that my young students become depressed if we want to detect them with the Beck questionnaire?

Do I have to wait that my car tank is empty to decide that I have to go to the pump station?

 

The persons distribution, and the SE may say something but only if I am in a certain portion of the scale, because the questionnaire does not have items covering all the possible states from "normal" to pathologically depression. Ben Wright suggest to have items covering all the scale and not only a portion, unless I want to use the scale on depressed persons. 

Item distribution is related to the test design line, an idea proposed by Ben but not used everywhere and not conceptualized as a theoretical concept. 

I can notice that we accept the Rasch model as a theoretical expected model of an item and, curiously,  we do not accept that there should be also a theoretical expected model for a test (the test design line is that model). That is whay we can accept tests with a bad distribution of items, with a defficient scale, because we are convinced that the Rasch measures permit to define ALL the scale, even if the set of items is not sufficient to describe the construct.

 

Without the theoretical test model, we try to explain the problems with the Beck's measures (or the suicide, etc.) as a function of the SE, or probably as a consequence of the sample, or perhaps because we may think that it is not Guttmann, or probably because the Cronbach's alpha was 0.3, or probably because they copied the answers, or probably because we have to wait that our students become sufficiently depressed if we wish to measure them using Beck or Hamilton questionnaires... and all those explanations may be false, because the basics of the scale is wrong. 

 

I cannot understand how we can try to explain the planetary system unless we understand that the planets translate around the Sun. Everything about the planets comes from this concept, otherwise we can give explanations that are very complicated and someday will fail.

I cannot understand how we can try to explain the results of a test unless we use the test design line as a necessary coondition for the scale's validity. Everything about test and persons measures comes from this concept, otherwise we can give explanations that are very complicated and someday will fail.

 

I've presented the tests design line and all its implications elsewhere...It works.

 

Regards

Agustin

 

 

 

 

 



 

INSTITUTO DE EVALUACION E INGENIERIA AVANZADA.


Ave. Cordillera Occidental No. 635

Colonia Lomas 4ª San Luis Potosí, San Luis Potosí.

C.P. 78216 MEXICO

(52) (444) 8 25 50 76 / (52) (444) 8 25 50 77 / (52) (444) 8 25 50 78

Página Web (en español): http://www.ieia.com.mx <http://www.ieia.com.mx/> 

Web page (in English): http://www.ieesa-kalt.com/English/Frames_sp_pro.html



--- On Sat, 7/23/11, Michael Lamport Commons <commons at tiac.net> wrote:


From: Michael Lamport Commons <commons at tiac.net>
Subject: Re: [Rasch] Bunched versus spread out items to measure depression
To: "Bond,Trevor" <trevor.bond at jcu.edu.au>, "Bond,Trevor" <trevor.bond at jcu.edu.au>, "rasch" <rasch at acer.edu.au>
Cc: "Jose Alves" <alves at psi.uminho.pt>, "Jose Alves" <alves at iep.uminho.pt>
Date: Saturday, July 23, 2011, 6:25 AM

We had people with a very wide range of depression from none to a lot.  We had a sample from listservs for depressed people and regular depressed people.  It might be that depression is not Guttman like, i.e. one does not have to be a little sad before one cries a lot and skulks, and before one stops leaving ones room.  We found it was just about how many items with about the same Rasch score predicted the amount of depression.  Many of the common depression scales such as the Hamilton and Beck seem to have the same property when we Rasch analyzed them.''

MLC

-----Original Message-----
>From: "Bond, Trevor" <trevor.bond at jcu.edu.au <http://us.mc1115.mail.yahoo.com/mc/compose?to=trevor.bond@jcu.edu.au> >
>Sent: Jul 23, 2011 3:15 AM
>To: "Bond, Trevor" <trevor.bond at jcu.edu.au <http://us.mc1115.mail.yahoo.com/mc/compose?to=trevor.bond@jcu.edu.au> >, Michael Lamport Commons <commons at tiac.net <http://us.mc1115.mail.yahoo.com/mc/compose?to=commons@tiac.net> >, rasch <rasch at acer.edu.au <http://us.mc1115.mail.yahoo.com/mc/compose?to=rasch@acer.edu.au> >
>Cc: Jose Alves <alves at psi.uminho.pt <http://us.mc1115.mail.yahoo.com/mc/compose?to=alves@psi.uminho.pt> >, Jose Alves <alves at iep.uminho.pt <http://us.mc1115.mail.yahoo.com/mc/compose?to=alves@iep.uminho.pt> >
>Subject: Re: [Rasch] Bunched versus spread out items to measure depression
>
>Ooops drinking champagne at breakfast time:
>Large person SE (sic.)= too few items. Or huge targetting problem
>
>
>On 23/07/11 8:39 AM, "Bond, Trevor" <trevor.bond at jcu.edu.au <http://us.mc1115.mail.yahoo.com/mc/compose?to=trevor.bond@jcu.edu.au> > wrote:
>
>> Large person SD = too few items.
>> Bunching items is useful if you have a high stakes cut-point to use.
>> Best
>> T
>> 
>> 
>> On 21/07/11 8:48 PM, "Michael Lamport Commons" <commons at tiac.net <http://us.mc1115.mail.yahoo.com/mc/compose?to=commons@tiac.net> > wrote:
>> 
>>> Dear List Members:
>> 
>> What are the disadvantages and advantages of having the
>>> Rasch item scores bunched up with a small S.D. versus spread out for items to
>>> measure depression?  There person Rasch score have a very large S.D. which
>>> indicates that the items are capturing a wide range of depression
>> 
>> 
>> 
>> My
>>> Best,
>> 
>> Michael Lamport Commons, Ph.D.
>> Assistant Clinical Professor
>> 
>> Department
>>> of Psychiatry
>> Harvard Medical School
>> Beth Israel Deaconess Medical Center
>> 234
>>> Huron Avenue
>> Cambridge, MA 02138-1328
>> 
>> Telephone (617) 497-5270
>> Facsimile
>>> (617) 491-5270
>> Cellular (617)
>>> 320­0896
>> Commons at tiac.net <http://us.mc1115.mail.yahoo.com/mc/compose?to=Commons@tiac.net> 
>> http://dareassociation.org/
>> ________________________
>>> _______________________
>> Rasch mailing list
>> Rasch at acer.edu.au <http://us.mc1115.mail.yahoo.com/mc/compose?to=Rasch@acer.edu.au> 
>> Unsubscribe:
>>> 
>https://mailinglist.acer.edu.au/mailman/options/rasch/trevor.bond%40jcu.edu.a>>
>u
>> 
>> 
>> _______________________________________________
>> Rasch mailing list
>> Rasch at acer.edu.au <http://us.mc1115.mail.yahoo.com/mc/compose?to=Rasch@acer.edu.au> 
>> Unsubscribe: 
>> https://mailinglist.acer.edu.au/mailman/options/rasch/trevor.bond%40jcu.edu.au
>> 
>
>


My Best,

Michael Lamport Commons, Ph.D.
Assistant Clinical Professor

Department of Psychiatry
Harvard Medical School
Beth Israel Deaconess Medical Center
234 Huron Avenue
Cambridge, MA 02138-1328

Telephone (617) 497-5270
Facsimile   (617) 491-5270
Cellular    (617) 320–0896
Commons at tiac.net <http://us.mc1115.mail.yahoo.com/mc/compose?to=Commons@tiac.net> 
http://dareassociation.org/
_______________________________________________
Rasch mailing list
Rasch at acer.edu.au <http://us.mc1115.mail.yahoo.com/mc/compose?to=Rasch@acer.edu.au> 
Unsubscribe: https://mailinglist.acer.edu.au/mailman/options/rasch/ici_kalt%40yahoo.com

 
 
My Best,
 
Michael Lamport Commons, Ph.D.
Assistant Clinical Professor
 
Department of Psychiatry
Harvard Medical School
Beth Israel Deaconess Medical Center
234 Huron Avenue
Cambridge, MA 02138-1328
 
Telephone (617) 497-5270
Facsimile   (617) 491-5270
Cellular       (617) 320–0896
Commons at tiac.net
http://dareassociation.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20110725/d52ee417/attachment.html 


More information about the Rasch mailing list