[Rasch] FACETS design

Fabio La Porta fabiolaporta at mail.com
Tue Feb 12 04:18:51 AEDT 2019


Rense,

in the first statement I specified "theoretically". In practice, the
estimation of rater severity is surely possible in the presence of missing
rater data, but in the presence of too many missing data the model may not
converge and/or
the precision of rater severity estimates may be too imprecise.

Maybe this is not the case of the within-country design, although I doubt
very much that a facet design could be implemented across countries, given
that there are no overlapping raters across countries.

Indeed, if I understood well Ricardo's design, it seems that subjects of
Country A are rated by judges of their own Country, but not by raters of
Countries B, C, D; subjects of Country B are rated by raters of their own
Country, but not by raters of Countries A, C, D; and so on and so forth.

Considering the above example, the resulting data matrix would be the
following, with no raters (or subjects) overlapping across countries:

Country Raters

Country subjects     A     B    C    D

A         X

B                 X

C                        X

D                               X


Could such a facet design lead the model to converge? And if so, how much
could we trust the precision of the rater severity estimates across
countries in the presence of so many structural missing data?

Does this make sense?

Fabio



Il giorno lun 11 feb 2019 alle ore 16:29 Rense Lange <rense.lange at gmail.com>
ha scritto:

> I mostly agree, but the first point seems incorrect. Facets was in fact
> designed for the case where raters grade only partly overlapping subsets of
> respondents. See RMT for examples of very minimal rater overlap …
>
> Rense Lange
>
> On Feb 11, 2019, at 7:54 PM, Fabio La Porta <fabiolaporta at mail.com> wrote:
>
> Hi Ricardo,
> In order to conduct a multifacet analysis the theoretical requirement is
> that all the raters examine all the subjects.  <<<<< NO
>
> For sure, you cannot apply this design across countries. Here, you will
> want to test the requirement of invariance of the item difficulties trans
> culturally, i.e. the absence of DIF by Country.
>
> Regarding the within-Country facet design, its feasibility depends on the
> amount of missing rater data.
>
> I Hope this helps
> Fabio
>
> Il giorno lun 11 feb 2019 alle 12:27 Ricardo Primi <rprimi at mac.com> ha
> scritto:
>
>> Dear list
>>
>> I have a question about FACETS. I am working in a project where we have
>> tasks translated into various languages administered to students within
>> several countries. Raters are nested within countries, that is, a group A
>> of raters of country A will score tasks of country A students, a group B of
>> raters of country B will score tasks of students of country B students and
>> so on.
>>
>> I have two questions: 1) Do you know any similar study with these
>> characteristics that employed FACETS? Specifically any idea of a design
>> that can be employed to create connectedness and link measures, that is,
>> create a metric that permits to compare countries? 2) The main interest is
>> to estimate country means as reliably as possible. Do you know any study
>> that address how person reliability (level 1 in a multilevel design) is
>> related to group mean reliability (level 2)? A specific question is how
>> design features (number of tasks, number of raters per task, rater
>> agreement) affects this reliability of means at second level.
>>
>> Thanks a lot  in advance
>>
>>
>> Ricardo Primi
>> USF, Brazil
>>
>> ________________________________________
>> Rasch mailing list
>> email: Rasch at acer.edu.au
>> web:
>> https://mailinglist.acer.edu.au/mailman/options/rasch/fabiolaporta%40mail.com
>
> --
> Fabio La Porta, MD PhD
> ________________________________________
> Rasch mailing list
> email: Rasch at acer.edu.au
> web:
> https://mailinglist.acer.edu.au/mailman/options/rasch/rense.lange%40gmail.com
>
> ________________________________________
> Rasch mailing list
> email: Rasch at acer.edu.au
> web:
> https://mailinglist.acer.edu.au/mailman/options/rasch/fabiolaporta%40mail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20190211/934bf651/attachment-0001.html>


More information about the Rasch mailing list