[Rasch] PISA critique in TES - a competition?

Mike Linacre mike at winsteps.com
Sun Sep 28 17:46:53 EST 2014

Kreiner says "Rasch model is not suitable for PISA"

Surely the fundamental problem is not the Rasch model. The fundamental 
problem is the missing data.

Why don't we propose a subset of a PISA dataset to Kaggle as a 
data-mining competition? Perhaps their 200,000+ statistical experts can 
discover better ways to impute the missing data - 

The proposed Kaggle competition would work as follows:
a) from an actual, but not yet public, meaningful subset of the PISA 
dataset (with lots of missing data) a little more empirical data is 
removed in an intelligently random way. The names of countries, 
specifics of items, etc., are also removed.
b) the competition is to predict all the missing observations in that 
PISA dataset. The competitors are not told which are the observations 
removed in (a).
c) the winner is the expert whose predicted observations match the 
empirical observations removed in (a) the closest.

Having discovered more effective ways to impute the missing observations 
in PISA datasets, we can advance to choosing the best model for 
analyzing the now quasi-complete PISA datasets (2000, 2003, ... , 2012) 
- perhaps by means of another Kaggle competition.

PISA insiders, when the PISA 2015 data is collected, how can we obtain 
an unreleased, anonymized, subset of the PISA 2015 dataset for (a)?

Mike Linacre
mike at winsteps.com

More information about the Rasch mailing list