Implementing automated writing evaluation in different instructional contexts: A mixed-methods study

Abstract

There is increasing evidence that automated writing evaluation (AWE) systems support the teaching and learning of writing in meaningful ways. However, a dearth of research has explored ways that AWE may be integrated within different instructional contexts and examined the associated effects on students’ writing performance. This paper describes the AWE system MI Write and presents results of a mixed-methods study that investigated the integration and implementation of AWE with writing instruction at the middle-school level, examining AWE integration within both a traditional process approach to writing instruction and with strategy instruction based on the Self-Regulated Strategy Development model. Both instructional contexts were evaluated with respect to fostering growth in students’ first-draft writing quality across successive essays as well as students’ and teachers’ experiences and perceptions of teaching and learning with AWE. Multilevel model analyses indicated that during an eight-week intervention students in both instructional contexts exhibited growth in first-draft writing performance and at comparable rates. Qualitative analyses of interview data revealed that AWE’s influence on instruction was similar across contexts; specifically, the introduction of AWE resulted in both instructional contexts taking on characteristics consistent with a framework for deliberate practice.