چکیده:
The present study reports the processes of development and use of an AnalyticDichotomous Evaluation Checklist (ADEC) which aims at enhancing both inter-and intra-rater reliability of writing evaluation. The ADEC consists of a total of 68items that comprises five subscales of content, organization, grammar, vocabulary,and mechanics. Eight raters assessed the writing performance of 20 Iranian EFLstudents using the ADEC. Also, the raters were asked to rate the same sample ofessays holistically based on Test of Written English (TWE) scale. To examine theinter-rater and intra-rater reliability of the ADEC, multiple approaches wereemployed including correlation coefficient, the dichotomous Rasch Model, andmany-faceted Rasch measurement (MFRM). The findings of the study confirmedthat the ADEC introduces higher reliability into scoring procedure compared withholistic scoring. Future research with greater number of raters and examinees mayprovide robust evidence to use analytic scale rather than holistic one.
خلاصه ماشینی:
"Appendix C Frequencies of Categories and Subcategories of Writing Obtained through Think Aloud Categories Frequency Subcategories 549 Content Clarity of ideas 22 Rhetorical function 3 Task development 1 Relevance 34 Adherence to the main topic 10 Appropriacy of ideas 2 Necessity to reread the essay 27 Inadequate development of ideas 1 Well-stated ideas 16 Control ideas through examples 37 Control ideas through details 19 Incomplete/complete essay 15 Well-developed ideas 46 Adequate addressing of the topic 19 Comprehensibility 58 Fluency 8 Repetition of ideas 4 Adequate amount of information 24 Task response 40 Focus 8 Redundancy 8 Explicit thesis statement 36 Implied thesis statement 1 Copying materials 1 IJAL, Vol. 16, No. 1, March 2013 55 Thesis statement in each paragraph 1 Well-developed paragraphs 6 A good response to the topic 5 Paraphrase and/or rephrase of the title 2 Supporting sentences 17 Unclear arguments 1 Interpretation on the part of the reader 1 Correct interpretation of the prompt/task/topic 1 Clarity of details 1 Communication of the message 5 Message diversion 1 Addressing part of the task 1 Flow 9 Off topic essay 1 747 Organization The length of the essay 85 The length of the paragraph 10 Paragraphing 203 Introduction 152 Body 41 Conclusion 96 Referencing 7 Cohesion 4 Coherence 31 Style 37 Cohesive ties/Conjunctions/Enumerators 38 Blueprint 40 Transitions/Markers 6 Logical relations 8 Logical order of ideas 1 462 Grammar Prepositions 24 SVO order 4 Articles 15 Word form 19 56 Using an Analytic Dichotomous Evaluation Checklist to Increase Inter-… Participles 8 Repetition of the same pattern 12 Verb forms/ tense 20 Conditionals 12 Pronouns 11 Subject-verb agreement 12 Passive/active voice 8 Simple sentences 4 Complex/compound sentences 10 Syntactic variety 9 Appropriacy 7 Accuracy 2 Syntactic complexity 3 Conjunctions 31 Adverbs/adjectives 32 The length of the sentences 30 Structural ambiguity 2 Incomplete sentences 2 Coordinators 4 Comparatives 2 Relative clauses/embedded clauses 4 Possessive adjectives 6 Modals 11 Memorized phrases/sentences 11 L1 in L2 3 Redundancy 1 Correct use of nouns 12 Pronoun agreement 7 Numbering 5 Prepositional phrases 1 Parallelism 7 Case 3 Noun phrases 2 Negation 1 Quantifiers 3 Comprehensibility of sentences 1 461 Vocabulary IJAL, Vol. 16, No. 1, March 2013 57 Appropriacy 139 Accuracy 5 Idioms 9 Wide/restricted range of vocabulary 79 L1 in L2 11 Collocations 25 Complex words 11 Lexical complexity distorts understanding 2 Vocabulary relevant to the topic 1 Figurative speech 18 Repetition 40 Lexical variety 3 Simple common words 3 Synonyms/Antonyms 4 Phrasal verbs 7 Words copied from the topic 1 Redundancy 18 Comprehensibility 5 Missing words 4 564 Mechanics Handwriting 232 Spelling 72 Punctuation 287 Capitalization 72 Paragraph spacing/break 3 Neatness 25 Number of paragraphs 33 Spacing between words 3 Spacing between paragraphs 14 Pagination 14 Spacing between lines 5 L1 in L2 3 Spelling mistakes distorts understanding 1 Indentation 1"