[Rasch] PISA critique in TES - a competition? - from a distance

Stefan Cano stefanjcano at gmail.com
Wed Oct 1 20:57:49 EST 2014

My own view:

There is critique and then there is criticism. The former is always
important; the latter more often "trivially true", but less helpful (and
more divisive) unless something better is proposed.

Like Thomas I am outside of education. And I share his concerns. 

And so taking an "outsider's" viewpoint, it could be argued that the two
take home messages from this thread are: 1) the Rasch model isn't very
useful; and 2) those who use the Rasch model not only do not agree with each
other, they actually think each other is "wrong".

I would suggest that's probably not very helpful.

-----Original Message-----
From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On Behalf
Of Bond, Trevor
Sent: 01 October 2014 11:07
To: <rasch at acer.edu.au>
Subject: Re: [Rasch] PISA critique in TES - a competition? - from a distance
Importance: High

Are we suggesting that critique should be hidden from view?

Sent from 007's iPad

> On 1 Oct 2014, at 5:50 pm, "Stefan Cano" <stefanjcano at gmail.com> wrote:
> Thank you for posting this Thomas. Very nicely articulated.
> -----Original Message-----
> From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On 
> Behalf Of David Andrich
> Sent: 30 September 2014 12:45
> To: rasch at acer.edu.au
> Subject: Re: [Rasch] PISA critique in TES - a competition? - from a 
> distance
> Importance: High
> Totally agree with Steve. 
> David
> David Andrich, BSc MEd W.Aust., PhD Chic, FASSA Chapple Professor 
> david.andrich at uwa.edu.au  Graduate School of Education The University 
> of Western Australia M428, 35 Stirling Highway, Crawley, WA, 6009
> Telephone: +61 8 6488 1085;   Fax: +61 8 6488 1052
> www.matildabayclub.net
> http://www.education.uwa.edu.au/ppl/courses<SPAN style="COLOR: nav 
> ________________________________________
> From: rasch-bounces at acer.edu.au [rasch-bounces at acer.edu.au] On Behalf 
> Of Stephen Humphry [stephen.humphry at uwa.edu.au]
> Sent: Tuesday, 30 September 2014 7:19 PM
> To: rasch at acer.edu.au
> Subject: Re: [Rasch] PISA critique in TES - a competition? - from a 
> distance
> Nicely said Thomas.
> Steve
> From: rasch-bounces at acer.edu.au [mailto:rasch-bounces at acer.edu.au] On 
> Behalf Of Thomas Salzberger
> Sent: Tuesday, 30 September 2014 6:55 PM
> To: rasch at acer.edu.au
> Subject: Re: [Rasch] PISA critique in TES - a competition? - from a 
> distance
> Dear all,
> I have been watching the discussion from a distance since education is 
> not my field of expertise.
> Consequently, I refrain from making specific comments about the content.
> Anyhow, in my mind, it is unfortunate that the controversy divides 
> people who are all, by and large, building upon the same premises and 
> sharing more or less the same understanding of what measurement is or what
it should be.
> It is an all to frequent phenomenon that the Rasch-community/ies 
> get(s) roped into discussions on issues we actually should not assume 
> responsibility for.
> I, personally, do believe that PISA is first and foremost a political
> Isn't there an obsession with comparing everything, everybody, and 
> every country?
> Admittedly, I do see the point. It is interesting to compare the 
> outcome of educational systems of all sorts across all possible 
> countries and derive conclusions. The ones lagging behind can learn from
the vanguard. All fine.
> The objectives are commendable.
> On the other hand, when does it come done to comparing apples and oranges?
> Aren't there different curricula? Different emphasis from one country 
> to another?
> Different people? Culture?
> Again, I am not an expert in the field, but I would think all of the 
> above matters.
> Or should all be the same all over the world? Globalization, same 
> goals, one system?
> Should we all teach to the PISA-test?
> Rasch is a wonderful tool. Providing us with very useful properties if 
> requirements are met.
> But isn't the frame of reference key? How far can we stretch the frame 
> of reference when it comes to PISA? Apples and oranges, important? All 
> fruits, after all.
> However, what if one country tops the list in terms of apples but 
> performs poorly in terms of oranges?
> Whisk the thing and it is all the same?
> I assume that multiple rankings (or a range of rankings) provide a 
> much more valid representation, if one wants to apply Rasch. And I 
> further assume everybody familiar with Rasch and measurement openly or
secretly agrees.
> (Maybe I am wrong.) The problem is, there is a political agenda. No 
> politician (and the public) would have a clue what these fuzzy 
> rankings would mean. They want an unambiguous answer. Simplicity 
> rather than complexity. Damn thing that we live in a complex world.
> In fact, I do understand politicians. If they want to derive clear 
> conclusions, one equivocal ranking is much more useful. So, I do 
> understand all who are behind PISA when they desperately try to achieve
> The question remains what conclusions politicians actually draw anyway?
> This is not a rhetorical question, I really do not know the answer.
> In my own country (Austria), I observed that
> - politicians refer to the gap between Austria and the top (mostly the
> Finns) whenever they want to emphasise that something needs to be done 
> and the thing that needs to be done is what they have always said 
> needs to be done (Pisa as the justification that action is required, 
> but nobody really thinks about how the apparently successful 
> procedures followed in Finland could be implemented in Austria; when 
> asked about that politicians say "Ah that wouldn't work here" or "That's
too expensive, we cannot afford it").
> - ruling parties acclaim any change to the better (even if it may only 
> be a result of random fluctuation, it's different cohorts, after all), 
> and claim that this is a sign that their policy is the right one
> - whenever there is decline, ruling parties take this is evidence that 
> there policy is still the right one but that we need more of that 
> policy (we are on the right track, but it takes time, etc).
> - the opposition claims the reverse.
> No wonder that, as a citizen, I have doubts as to the usefulness of 
> PISA once the results are in the hand of politicians. I guess experts 
> in education might learn more from PISA results and they would know 
> how to interpret ambiguity. Alas, it is not experts who rule this 
> world (perhabs this is a good thing).
> My fear is that the controversy about PISA will not be understood as a 
> controversy primarily about PISA but more as one about the Rasch model.
> People will say "PISA? Isn't that this test where this obscure Rasch 
> model is applied?"
> Or "No wonder that they are in a mess. Rasch just doesn't work."
> This is not to say that we should not discuss the issue. It is one of 
> the great scientific assets of Rasch measurement that we do not shy 
> away from such debates. But we need to emphasise that problems, if 
> any, are not due to the Rasch model but due to the immense complexity of
the task.
> How this can be done? I don't know.
> In any case, the purpose of Rasch is exposing problems, not 
> necessarily solving them.
> This brings us back to politics. Politicians don't fancy problems, 
> they want solutions ...
> Best
> Thomas
> --
> ___________________________________
> Thomas.Salzberger at gmail.com<mailto:Thomas.Salzberger at gmail.com>
> Thomas.Salzberger at wu.ac.at<mailto:Thomas.Salzberger at wu.ac.at>
> ________________________________________
> Rasch mailing list
> email: Rasch at acer.edu.au
> web:
> https://mailinglist.acer.edu.au/mailman/options/rasch/stefanjcano%40gm
> ail.co
> m
> ________________________________________
> Rasch mailing list
> email: Rasch at acer.edu.au
> web: 
> https://mailinglist.acer.edu.au/mailman/options/rasch/trevor.bond%40jc
> u.edu.au
Rasch mailing list
email: Rasch at acer.edu.au

More information about the Rasch mailing list