[Rasch] qualitatively described, quantitatively responded

Agustin Tristan ici_kalt at yahoo.com
Sat Nov 10 01:16:33 EST 2007


Hi Theo: Of course an essay cannot be scored as a single dimension. When you say complexity, coherence, and many more: sintax, logic, order or the text (introduction/development/conclusion), relevance of the message, etc. EACH one is a single dimension, and we have to try to measure each one separately.
  I think the beginning of this discussion, following your example, is to treat an essay as a multidimensional variable that can be measured with a single number. Many in this listserve say no to this approach, that is the reason why we have items trying to measure one dimension each one: certainly there are several items measuring "coherence" (not a single item to measure coherence), and other set of items to measure "complexity", and so forth. We don't like to see a set of items measuring the multidimensional essay as a hole, with my regards to people who likes holistic scoring.
  Yours
  Agustin
  

Theo Dawson <theo at devtestservice.com> wrote:
  Rating the quality of essays on a single dimension is a common practice in educational assessment---not good. This is because the quality of an essay is dependent upon multiple factors that are related to one another in complex ways. To name two:  

  Complexity (structure and level of abstraction of the argument)
    Coherence

  

  These are not only different dimensions; under certain common conditions, a higher level of performance on one can cause a lower level of performance on another. For example, when a student reaches beyond his or her modal level of reasoning complexity on a particular subject (when she or he is engaged in constructing new knowledge), coherence is likely to suffer---even if the writer is a highly competent in these areas when writing at a lower level of complexity. This creates the unacceptable consequence that a student who is working with more sophisticated ideas can be awarded a lower score than a student working with less sophisticated ideas. Worse, clever students learn that they will not be rewarded for stretching, creating a situation in which deep learning is discouraged by grading practices.
  

  My colleagues and I are currently accumulating a database of essays scored independently for complexity level and coherence. If you have essays (in electronic form) that you can contribute to this database, please contact me.
  

  Theo
  
    On Nov 6, 2007, at 6:28 PM, Agustin Tristan wrote:

    

shirin shirazi <shirin71_shirazi at hotmail.com> wrote:  
Hi 
 
Thanks for your concern.
 
I have thirty essays to rate. There is a rating scale consisting of nine descriptors. The first one describes the writing ability of the writer as a "fully competent". If the writer suits this level, he gets nine. If he adequately responds to the task, it brings him down to eight and the same process continues up until zero where there is no response to the task. I wonder how these numbers have been given to these qualitative words. Fully 9, adequately 8, to some extent 7, partially 6, ...
 
The second question is that why do we have a range of descriptors from nine (at most) to three (at least)?
 
Third why don't we have qualitative words to describe levels of performance since descriptors are describing the proficiency in qualitative manner, for example instead of earning a 9 in IELTS, we allocate a word such as competent or incompetent or effective or ineffective which are in harmony with qualitative words appearing in descriptors. Do we use numbers since they are more tangible (People come to grips with them easily)?
 
Thanks in advance and your help is greatly appreciated.
 
Best
Shirin
    
---------------------------------
  Date: Mon, 5 Nov 2007 15:43:55 -0800
From: ici_kalt at yahoo.com
Subject: Re: [Rasch] qualitatively described, quantitatively responded
To: shirin71_shirazi at hotmail.com

  hello! what is exactly the problem you are trying to solve? describe your test please.
  regards
  agustin tristan

shirin ahmadishirazi <shirin71_shirazi at hotmail.com> wrote:
  
  Hi List members,
   
  I would be extremely grateful if you help solve the following controversies:
   
  1. Qualitative words are not appropriate to describe a quantitative construct. 
  2. Qualitative words are used to differentiate descriptors of levels of proficiency (although it is difficult to make distinctions between "some" and "a few", and "several" and "many").
  3. Productive skills (Speaking and writing) are qualitatively described but quantitatively scored. How can qualitative words (e.g., fully, sufficiently, partially, minimally, largely, adequately, effectively, ...) get quantified?
   
  All the Best,
  Shirin
 

  
---------------------------------
  Connect to the next generation of MSN Messenger  Get it now!_______________________________________________
Rasch mailing list
Rasch at acer.edu.au
http://mailinglist.acer.edu.au/mailman/listinfo/rasch



  FAMILIA DE PROGRAMAS KALT.
  Mariano Jiménez 1830 A
  Col. Balcones del Valle
  78280, San Luis Potosí, S.L.P. México
  TEL (52) 44-4820 37 88, 44-4820 04 31
  FAX (52) 44-4815 48 48
  web page (in Spanish AND ENGLISH): http://www.ieesa-kalt.com
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

  
---------------------------------
  Help yourself to FREE treats served up daily at the Messenger Café. Stop by today!


  FAMILIA DE PROGRAMAS KALT.
  Mariano Jiménez 1830 A
  Col. Balcones del Valle
  78280, San Luis Potosí, S.L.P. México
  TEL (52) 44-4820 37 88, 44-4820 04 31
  FAX (52) 44-4815 48 48
  web page (in Spanish AND ENGLISH): http://www.ieesa-kalt.com
  __________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com _______________________________________________
Rasch mailing list
Rasch at acer.edu.au
http://mailinglist.acer.edu.au/mailman/listinfo/rasch








FAMILIA DE PROGRAMAS KALT. 
Mariano Jiménez 1830 A 
Col. Balcones del Valle 
78280, San Luis Potosí, S.L.P. México 
TEL (52) 44-4820 37 88, 44-4820 04 31 
FAX (52) 44-4815 48 48 
web page (in Spanish AND ENGLISH): http://www.ieesa-kalt.com

 __________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailinglist.acer.edu.au/pipermail/rasch/attachments/20071109/a687ea92/attachment.html 


More information about the Rasch mailing list