T-Space at The University of Toronto Libraries >
School of Graduate Studies - Theses >
Please use this identifier to cite or link to this item:
|Title: ||An Argument-based Validity Inquiry into the Empirically-derived Descriptor-based Diagnostic (EDD) Assessment in ESL Academic Writing|
|Authors: ||Kim, Youn-Hee|
|Advisor: ||Jang, Eunice|
|Department: ||Curriculum, Teaching and Learning|
|Keywords: ||Language Testing|
|Issue Date: ||13-Aug-2010|
|Abstract: ||This study built and supported arguments for the use of diagnostic assessment in English as a second language (ESL) academic writing. In the two-phase study, a new diagnostic assessment scheme, called the Empirically-derived Descriptor-based Diagnostic (EDD) checklist, was developed and validated for use in small-scale classroom assessment. The checklist assesses ESL academic writing ability using empirically-derived evaluation criteria and estimates skill parameters in a way that overcomes the problems associated with the number of items in diagnostic models. Interpretations of and uses for the EDD checklist were validated using five assumptions: (a) that the empirically-derived diagnostic descriptors that make up the EDD checklist are relevant to the construct of ESL academic writing; (b) that the scores derived from the EDD checklist are generalizable across different teachers and essay prompts; (c) that performance on the EDD checklist is related to performance on other measures of ESL academic writing; (d) that the EDD checklist provides a useful diagnostic skill profile for ESL academic writing; and (e) that the EDD checklist helps teachers make appropriate diagnostic decisions and has the potential to positively impact teaching and learning ESL academic writing.
Using a mixed-methods research design, four ESL writing experts created the EDD checklist from 35 descriptors of ESL academic writing. These descriptors had been elicited from nine ESL teachers’ think-aloud verbal protocols, in which they provided diagnostic feedback on ESL essays. Ten ESL teachers utilized the checklist to assess 480 ESL essays and were interviewed about its usefulness. Content reviews from ESL writing experts and statistical dimensionality analyses determined that the underlying structure of the EDD checklist consists of five distinct writing skills: content fulfillment, organizational effectiveness, grammatical knowledge, vocabulary use, and mechanics. The Reduced Reparameterized Unified Model (Hartz, Roussos, & Stout, 2002) then demonstrated the diagnostic quality of the checklist and produced fine-grained writing skill profiles for individual students. Overall teacher evaluation further justified the validity claims for the use of the checklist. The pedagogical implications of the use of diagnostic assessment in ESL academic writing were discussed, as were the contributions that it would make to the theory and practice of second language writing instruction and assessment.|
|Appears in Collections:||Doctoral|
Department of Curriculum, Teaching and Learning - Doctoral theses
Items in T-Space are protected by copyright, with all rights reserved, unless otherwise indicated.