[Rasch] One model for three constructs and four groups (was RE: PCA and DIF)

rsmith rsmith at jampress.org
Tue Jun 7 04:40:46 EST 2011


Dear Juanito,

I understand that my advice might not necessarily be helpful, but I would ask the simple question.  If you wanted to compare average sales at 4 shops would you run 6 separate t-test or use anova?

If avova is your answer, I would ask why would you run multiple DIF analyses when a single between group fit statistic could do the same job and you would not have to worry about the increase in false-positive and false-negative results and an inflated Type I error rate due to multiple comparisons?

Richard

Richard M. Smith, Editor
Journal of Applied Measurement
P.O. Box 1283
Maple Grove, MN  55311, USA
website:  www.jampress.org
phone: 763-268-2282
fax: 763-268-2782 

-------- Original Message --------
> From: Juanito Talili <talilij at yahoo.com>
> Sent: Sunday, June 05, 2011 9:29 PM
> To: AndrewStephanou <Stephanou at acer.edu.au>, Rasch Forum <rasch at acer.edu.au>
> Subject: [Rasch] One model for three constructs and four groups (was RE: PCA and DIF)
> 
> Sir Andrew and all,
>  
> Let me go back to the canteen services scenario I raised a couple of months ago.
>  
> A 15-item questionnaire has three (theoretical) constructs, 5 items for each construct. The questionnaire aims to measure customer satisfaction using a four point ordinal scale for each item.  The questionnaire was adminsitered to four different shops/canteens. 
>  
> Sir Andrew, I followed your suggestion that all data that I gathered from the four stores on the 15 items should be analyzed together.  What shall I do with my three constructs? How to justify my boss that the three constructs are considered as one dimension?
>  
> Further, I followed your suggestion that I should do a Principal Component Analysis of the Residuals with Winsteps, then do a "shop" DIF analysis for the items by comparing the four shops in pairs.  My question is: What does it mean when DIF test is significant for several items  and nonsignificant for other items?  Can I still use the instrument for the four shops in the future ?
>  
> Using Winsteps, how to construct customer-item map in four columns, one for each shop?
>  
> Thank you and warm regards!
>  
> J
>  
> 
> --- On Wed, 4/6/11, Stephanou, Andrew <Stephanou at acer.edu.au> wrote:
> 
> 
> From: Stephanou, Andrew <Stephanou at acer.edu.au>
> Subject: RE: PCA and DIF
> To: "Juanito Talili" <talilij at yahoo.com>
> Date: Wednesday, 6 April, 2011, 3:44 AM
> 
> 
> 
> Best wishes, Juanito.
>  
> Please let me know how your analysis is going.  I am interested.
>  
> Cheers,
> Andrew
> 
> 
> 
> From: Juanito Talili [mailto:talilij at yahoo.com] 
> Sent: Wednesday, 6 April 2011 1:03 PM
> To: Stephanou, Andrew
> Subject: RE: PCA and DIF
> 
> 
> 
> 
> 
> 
> 
> My hometown is Philippines but I am working now in Thailand. 
>  
> Thanks a lot, sir Andrew.
> 
> 
> --- On Tue, 4/5/11, Stephanou, Andrew <Stephanou at acer.edu.au> wrote:
> 
> 
> From: Stephanou, Andrew <Stephanou at acer.edu.au>
> Subject: RE: PCA and DIF
> To: "Juanito Talili" <talilij at yahoo.com>
> Date: Tuesday, 5 April, 2011, 7:54 AM
> 
> 
> 
> 
> Juanito,
>  
> First of all, please keep in mind that a factor is not a measurement dimension.  Weight and height would belong to the same factor because they are correlated highly but they are definitely different physical variables.
>  
> The answer to your questions will depend on findings.
>  
> If each student rates four shops then your "people" are the students and each item-shop is a separate item.  We probably need to discuss this further.
>  
> This is a satisfaction scale where high on the scale you will find satisfied students and items that are hard to satisfy people.  Low on the scale you have students who are not satisfied and items that can easily satisfy people.
>  
> You should display the items in the people-item map into four columns, one for each shop.
>  
> Cheers,
> Andrew
>  
> P.S.  Where are you from, Juanito?
> 
> 
> 
> From: Juanito Talili [mailto:talilij at yahoo.com]
> Sent: Tue 5/04/2011 4:16 PM
> To: Stephanou, Andrew
> Subject: PCA and DIF
> 
> 
> 
> 
> 
> 
> 
> Thank you so much for your reply, Andrew.  Please let me clarify because I was confused of some of your thoughts.
>  
> You wrote: 
> On what grounds do you assume that you are dealing with four different constructs?
>  
> I think you are referring to the number of constructs in the questionnaire.  There are only three constructs/factors, not four.  Yes, the constructs were grounded theoretically.  We dont have problems about it.
>  
> You wrote: If it can be justified on qualitative grounds, why don't you analyse all of your data together and see how the items fit together in the construction of a single scale of which there are four aspects? I would then do a Principal Component Analysis of the Residuals with Winsteps.  Then I would do a "shop" DIF analysis for the items by comparing the four shops in pairs.
>  
> What is the implication in case the Rasch PCA result is different from what we expect? In addition, what is the implication in case the DIF is significant?
>  
> Thank you.
>  
>  
>  
>  
> 
> 
> --- On Tue, 4/5/11, Stephanou, Andrew <Stephanou at acer.edu.au> wrote:
> 
> 
> From: Stephanou, Andrew <Stephanou at acer.edu.au>
> Subject: RE: [Rasch] (no subject)
> To: "Juanito Talili" <talilij at yahoo.com>
> Date: Tuesday, 5 April, 2011, 4:12 AM
> 
> 
> 
> Juanito,
>  
> On what grounds do you assume that you are dealing with four different constructs?
>  
> If it can be justified on qualitative grounds, why don't you analyse all of your data together and see how the items fit together in the construction of a single scale of which there are four aspects?  I would then do a Principal Component Analysis of the Residuals with Winsteps.  Then I would do a "shop" DIF analysis for the items by comparing the four shops in pairs.
>  
> Cheers,
> Andrew
> ACER
> 
> 
> 
> From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf Of Juanito Talili
> Sent: Tuesday, 5 April 2011 1:33 PM
> To: rasch
> Subject: [Rasch] (no subject)
> 
> 
> 
> 
> 
> 
> 
> A 15-item survey questionnaire that utilized a 4-point ordinal response options(1=strongly disagree, 2=disagree, 3=agree, 4=strongly agree) was developed. The said questionnaire will be used to assess the performance of the four food stores (A,B,C, D) in a school canteen.  It is a one page questionnaire where the students as respondents of the survey will indicate their ratings for each store for each item. The questionnaire was administered to 300 students.  
>  
> I want to analyze the psychometric properties of the questionnaire using Rasch model. The questionnaire has three constructs with five items each.  In the analysis, I would assume that each construct is unidimentional.  Thus, I would do Rasch analysis for each constract.  Here are my questions: (1)  Should I analyze the data (n=300) from one store (say store A) only?  (2) Is it wrong to use the 1400 (A+B+C+D) in the analysis? If so, why?
>  
> Thank you for your help?
> 
> 
> 
> _______________________________________________
> Rasch mailing list
> Rasch at acer.edu.au
> Unsubscribe: https://mailinglist.acer.edu.au/mailman/options/rasch/rsmith%40jampress.org 






More information about the Rasch mailing list