Fleshing out your text: How elaboration and contextualization moves differentially predict writing quality
DOI:
https://doi.org/10.17239/jowr-2023.15.02.05Keywords:
writing quality, annotation, text length, elaboration, contextualizationAbstract
This study explores the relation between writing quality and contextualization and elaboration moves, two kinds of textual expansion devices crucial for building common ground between writers and readers. We ask whether elaboration and contextualization features differentially predict writing quality and whether their quality contributions differ between genres. We also ask to what extent elaboration and contextualization are tied to individual writers, and can be explained by writer characteristics. To examine these issues, we annotated descriptive and argumentative texts of Dutch adolescents. Text quality was rated holistically, using benchmark scales.
As regards elaboration, depth affects quality more than breadth does. It also contributes across genres, whereas breadth only contributes in argumentations. Depth shows a large individual consistency across tasks, which is substantially related to students’ school type, grade and gender. Breadth shows weaker links to individual writers and their characteristics.
With regard to contextualization, opening and closing moves play a modest role in text quality. Initial support moves contribute to quality across tasks; concluding moves contribute more in argumentations. Concluding moves are most consistent within writers; however, for all contextualization moves, the writer variance is substantially explained by writer characteristics. This study opens up new avenues for explicating writing quality and writing skill.
References
Authors (2018). Handleiding Utrechts systeem voor annotatie van leerdertekst. [Manual Utrecht System for Annotating Learner Text]. Internal publication Utrecht University.
Bae, J., & Bachman, L. F. (2010). An investigation of four writing traits and two tasks across two languages. Language Testing, 27(2), 213-234. doi: 10.1177/0265532209349470
Beers, S. F., & Nagy, W. E. (2009). Syntactic complexity as a predictor of adolescent writing quality: Which measures? Which genre?. Reading and Writing, 22(2), 185-200. doi: 10.1007/s11145-007-9107-5
Berman, R., & Nir-Sagiv, B. (2009). Cognitive and linguistic factors in evaluating text quality: Global versus local. In Vyvyan Evans & Stéphanie Pourcel (eds.), New directions in cognitive linguistics, 421–440. Amsterdam: John Benjamins. doi: 10.1075/hcp.24.26ber
Blok, H. (1985). Estimating the reliability, validity, and invalidity of essay ratings. Journal of Educational Measurement, 22, 41–52. doi: 10.1111/j.1745-3984.1985.tb01048.x
Chen, J., Zhang, M., & Bejar, I. I. (2017). An Investigation of the e‐rater® Automated Scoring Engine's Grammar, Usage, Mechanics, and Style Microfeatures and Their Aggregation Model. ETS Research Report Series, 2017(1), 1-14. doi: 10.1002/ets2.12131
Cho, Y. H., & Cho, K. (2011). Peer reviewers learn from giving comments. Instructional Science, 39(5), 629-643. doi: 10.1016/j.learninstruc.2009.08.006
Clark, I. (2005). A genre approach to writing assignments. Composition Forum, 14(2). https://compositionforum.com/issue/14.2/clark-genre-writing.php
Clark, H.H., & Brennan S.E. (1991) Grounding in Communication. In L. Resnick, J. Levine & S.Teasley (Eds.), Perspectives on Socially Shared Cognition (127-149). Hyattsville, MD: American Psychological Association.
Crammond, J. G. (1998). The uses and complexity of argument structures in expert and student persuasive writing. Written Communication, 15(2), 230-268. doi: 10.1177/0741088398015002004
Crossley, S. A., & McNamara, D. S. (2016). Say More and Be More Coherent: How Text Elaboration and Cohesion Can Increase Writing Quality. Grantee Submission, 7(3), 351-370. doi: 10.17239/jowr-2016.07.3.02
Crossley, S. A., Kyle, K., & McNamara, D. S. (2015). To aggregate or not? Linguistic features in automatic essay scoring and feedback systems. Journal of Writing Assessment, 8. http:// journalofwritingassessment.org/article.php?article=80.
Dennis, A. R., Fuller, R. M., & Valacich, J. S. (2008). Media, tasks, and communication processes: A theory of media synchronicity. MIS quarterly, 32(3), 575-600. doi: 10.2307/25148857
Dobbs, C. L. (2014). Signaling organization and stance: Academic language use in middle grade persuasive writing. Reading and Writing, 27(8), 1327-1352. doi: 10.1007/s11145-013-9489-5
Donovan, C. A., & Smolkin, L. B. (2002). Children's genre knowledge: An examination of K‐5 students' performance on multiple tasks providing differing levels of scaffolding. Reading Research Quarterly, 37(4), 428-465.
Donovan, C.A. (2001). Children's development and control of written story and informational genres: Insights from one elementary school. Research in the Teaching of English, 35(3), 452-497.
Enright, M. K., & Quinlan, T. (2010). Complementing human judgment of essays written by English language learners with e-rater® scoring. Language Testing, 27(3), 317-334. doi: 10.1177/0265532210363144.
Espin, C. A., De La Paz, S., Scierka, B. J., & Roelofs, L. (2005). The relationship between curriculum-based measures in written expression and quality and completeness of expository writing for middle school students. The Journal of Special Education, 38(4), 208-217. doi: 10.1177/00224669050380040201
Ferretti, R. P., Lewis, W. E., & Andrews-Weckerly, S. (2009). Do goals affect the structure of students’ argumentative writing strategies?. Journal of Educational Psychology, 101(3), 577-589. doi: 10.1037/a0014702
Ferretti, R. P., MacArthur, C. A., & Dowdy, N. S. (2000). The effects of an elaborated goal on the persuasive writing of students with learning disabilities and their normally achieving peers. Journal of Educational Psychology, 92(4), 694. doi: 10.1037//0022-0663.92.4.694
Glopper, K. de, & Prenger, J. (2013). Schrijfmeters maken [Making the Schrijfmeters Corpus]. In A. Mottart & S. Vanhooren (Red.), Zevenentwintigste conferentie Onderwijs Nederlands (pp. 86–89). Gent: Academia Press.
Goldstein, M., Crowell, A., & Kuhn, D. (2009). What constitutes skilled argumentation and how does it develop? Informal Logic, 29, 379–395. Doi: 10.22329/il.v29i4.2905
Graham, S., MacArthur, C., & Schwartz, S. (1995). Effects of goal setting and procedural facilitation on the revising behavior and writing performance of students with writing and learning problems. Journal of Educational Psychology, 87(2), 230. doi: 10.1037//0022-0663.87.2.230
Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students. Journal of educational psychology, 99(3), 445-476. doi: 10.1037/0022-0663.99.3.445
Gregg, N., Coleman, C., Davis, M., & Chalk, J. C. (2007). Timed essay writing: Implications for high-stakes tests. Journal of Learning Disabilities, 40(4), 306-318. doi: 10.1177/00222194070400040201
Henry, A., & Roseberry, R. L. (2001). A narrow-angled corpus analysis of moves and strategies of the genre: ‘Letter of Application’. English for Specific Purposes, 20(2), 153-167. doi: 10.1016/s0889-4906(99)00037-x
Holliway, D. R., & McCutchen, D. (2004). Audience perspective in young writers' composing and revision: Reading as the reader. In L. Allal, L. Chanquoy, & P. Largy (), Studies in Writing: Vol. 13. Revision: Cognitive and instructional processes (pp. 87-101). Amsterdam, the Netherlands: Kluwer. doi: 10.1007/978-94-007-1048-1_6
Jeffery, J. V. (2009). Constructs of writing proficiency in US state and national writing assessments: Exploring variability. Assessing Writing, 14(1), 3-24. doi: 10.1016/j.asw.2008.12.002
Kamberelis, G. (1999). Genre development and learning: Children writing stories, science reports, and poems. Research in the Teaching of English, 33, 403-463.
Kellogg, R.T. ( 2008). Training writing skills: A cognitive developmental perspective . Journal of Writing Research, 1, 1-26. doi: 10.17239/jowr-2008.01.01.1
Klein, P.D., Haug, K.N. & Arcon, N. (2017). The effects of rhetorical and content subgoals on writing and learning, The Journal of Experimental Education, 85(2), 291-308. doi: 10.1080/00220973.2016.1143795
Knudson, R. E. (1992). Analysis of argumentative writing at two grade levels. The Journal of Educational Research, 85(3), 169-179. doi: 10.1080/00220671.1992.9944434
Kock, N. (2004). The psychobiological model: Towards a new theory of computer-mediated communication based on Darwinian evolution. Organization science, 15(3), 327-348. doi: 10.1287/orsc.1040.0071
Kock, N. (2007). Media naturalness and compensatory encoding: The burden of electronic media obstacles is on senders. Decision Support Systems, 44(1), 175-187. doi: 10.1016/j.dss.2007.03.011
MacArthur, C. A., Jennings, A., & Philippakos, Z. A. (2019). Which linguistic features predict quality of argumentative writing for college basic writers, and how do those features change with instruction?. Reading and Writing, 32(6), 1553-1574. doi: 10.1016/j.asw.2008.12.002
McCann, T. M. (1989). Student argumentative writing knowledge and ability at three grade levels. Research in the Teaching of English, 23(1), 62-76.
McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010). Linguistic features of writing quality. Written Communication, 27(1), 57-86. doi: 10.1177/0741088309351547
Myhill, D. (2008). Towards a linguistic model of sentence development in writing. Language and Education, 22(5), 271-288. doi: 10.2167/le775.0
Myhill, D. (2009a). Becoming a designer: Trajectories of linguistic development. In Bear, R., Myhill, D., Riley, J., Nystrand, M. The Sage Handbook of Writing Development. (pp.402-13). London: Sage Publications.
Myhill, D. (2009b). Developmental trajectories in mastery of paragraphing: Towards a model of development. Written Language & Literacy, 12(1), 26-51. doi: 10.1075/wll.12.1.02myh
Nussbaum, E. M., Kardash, C. M., & Graham, S. E. (2005). The effects of goal instructions and text on the generation of counterarguments during writing. Journal of Educational Psychology, 97(2), 157. doi: 10.1037/0022-0663.97.2.157
Nimon, K. (2010). Regression commonality analysis: demonstration of an SPSS solution. Multiple Linear Regression Viewpoints, 36(1), 10–17.
Olinghouse, N. G., & Wilson, J. (2013). The relationship between vocabulary and writing quality in three genres. Reading and Writing, 26(1), 45-65. doi: 10.1007/s11145-012-9392-5
Pander Maat, H. (2002). Tekstanalyse. Wat teksten tot teksten maakt [Text analysis. What makes a text a text]. Coutinho, Bussum.
Pollmann, E., Prenger, J., & Glopper, K. de (2012). Het beoordelen van leerlingteksten met behulp van een schaalmodel [Rating learner texts using a scale model]. Levende Talen Tijdschrift, 13(3), 15-24.
Salas, N., Llauradó, A., Castillo, C., Taulé, M., & Martí, M. A. (2016). Linguistic correlates of text quality from childhood to adulthood. In J. Perera, M. Aparici, E. Rosado & N. Salas (eds.), Written and spoken language development across the lifespan. Essays in honour of Liliana Tolchinsky. 307- 326. New York: Springer International Publishing. doi: 10.1007/978-3-319-21136-7_18
Schoonen, R. (2005). Generalizability of writing scores: an application of structural equation modeling. Language Testing, 22(1), 1–30. doi: 10.1191/0265532205lt295oa
Stein, N. L. & Glenn, C. G. (1979). An analysis of story comprehension in elementary school children. In Freedle, R. O. (ed.), New directions in discourse processing, 53–120. Norwood, NJ: Ablex.
Tracy, B., Reid, R., & Graham, S. (2009). Teaching young students strategies for planning and drafting stories: The impact of self-regulated strategy development. The Journal of Educational Research, 102(5), 323-331.
Traxler, M. J., & Gernsbacher, M. A. (1993). Improving written communication through perspective-taking. Language and Cognitive Processes, 8(3), 311-334. doi: 10.1080/01690969308406958
Uccelli, P., Dobbs, C. L., & Scott, J. (2013). Mastering academic language: Organization and stance in the persuasive writing of high school students. Written Communication, 30(1), 36-62. doi: 10.1177/0741088312469013
Upton, T. A. (2002). Understanding direct mail letters as a genre. International Journal of Corpus Linguistics, 7(1), 65-85. doi: 10.1075/ijcl.7.1.04upt
Varner, L. K., Roscoe, R. D., & McNamara, D. S. (2013). Evaluative misalignment of 10th-grade student and teacher criteria for essay quality: An automated textual analysis. Journal of Writing Research, 5(1). doi: 10.17239/jowr-2013.05.01.2
Weston, J. L., Crossley, S. A., McCarthy, P. M., & McNamara, D. S. (2011). Number of words versus number of ideas: Finding a better predictor of writing quality. In R. C. Murray, & P. M. McCarthy (Eds.), Proceedings of the 24th International Florida Artificial Intelligence Research Society (FLAIRS) Conference(pp. 335–340). Menlo Park, CA: AAAI Press.
Yunxia, Z. (2000). Structural moves reflected in English and Chinese sales letters. Discourse Studies, 2(4), 473-496. doi: 10.1177/1461445600002004004
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Henk Pander Maat, Kees De Glopper, Kay Raaijmakers, Joris Veerbeek, Dennis Vermeulen
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 Unported License.
