چکیده:
The consideration of rater training is very important in construct validation of a writing test because it is through training that raters are adapted to the use of students’ writing ability instead of their own criteria for assessing compositions (Charney, 1984). However, although training has been discussed in the literature of writing assessment, there is little research regarding raters’ perceptions and understandings of the training program. Although a few studies have looked at the differences between trained and untrained raters in writing assessment (Cumming, 1990; Huot, 1990), few studies have used a pre-and post-training design. The purpose of this study is to investigate the effectiveness of the training program on experienced and inexperienced raters with regard to a pre-and post- training design. Twelve EFL raters scored 45 pre-rated benchmark essay compositions by an authorized IELTS trainer. These essay compositions were scored before, during and after the training program. The results regarding the comparison across raters showed that inexperienced raters had wider range of inconsistency before training but they became more consistent than experienced raters after training.
خلاصه ماشینی:
"Raters’ Perception and Expertise in Evaluating Second Language Compositions Houman Bijani 1 Zanjan Branch, Islamic Azad University, Zanjan, Iran The consideration of rater training is very important in construct validation of a writing test because it is through training that raters are adapted to the use of students’ writing ability instead of their own criteria for assessing compositions (Charney, 1984).
Only a few studies have considered the differences between trained and untrained raters in writing assessment (Cumming, 1990; Weigle, 1994a), and few ones have used pre-and post-training design (Elder, Barkhuizen, Knoch&Randow, 2007; Hamilton, Reddel& Spratt, 2001) .
Note that the reason for typing essay composition papers and then giving them to the raters to score was that the raters might be influenced by the students’ handwritings, which would wrongly influence the effectiveness of the training program.
Moreover, the raters were given five additional new writing papers during the norming session to rate in pairs or groups to increase the effectiveness of the training program.
Data Analysis In order to answer both research questions of this study, all the materials including the pre-and post training questionnaires, audio interview tapes before and after training and the video recordings of the norming session were carefully analyzed.
92 A brief look at Table 6 reveals that training was effective for both groups in reducing dispersion among scores given by the raters; however, the result of reliability analysis shows that training was more effective for NEW raters than OLD raters."