Training programmes on writing with AI – but for whom? Identifying students’ writer profiles through two-step cluster analysis.
DOI:
https://doi.org/10.17239/Keywords:
Artificial intelligence, writing with AI, student teachers, cluster analysisAbstract
Generative AI has the potential to transform writing in schools and universities. This makes it necessary to develop training programmes for writing with AI, especially for students in teacher training. So far, however, little is known about the students' initial preconditions on which the trainings can be based upon. Evidence so far has come mainly from observational studies and questionnaire studies examining the frequency and type of AI use. However, the students themselves were not considered, nor the extent to which they can be categorised into groups. In other words, the focus has been on the writing rather than on the writers. To address this gap, the present article analyses data from a survey of N=505 students. To identify writer profiles, i.e. groups of students with comparable characteristics, we apply two-step cluster analysis. The students are clustered based on their use of AI for writing, as well as their level of awareness of AI applications, AI literacy, digital media literacy and writing-related self-concept. The results reveal four clusters, the two largest of which are characterised by the fact that students tend not to use AI, sometimes because they apparently have no awareness of AI, sometimes despite having such awareness. Merely one cluster, which describes 20% of the students, is characterised by regular use of AI for writing. The results therefore provide a useful insight for planning training in the context of university teaching.
References
Alier, M., García-Peñalvo, F.-J., & Camba, J. D. (2024). Generative Artificial Intelligence in Education: From Deceptive to Disruptive. International Journal of Interactive Multimedia and Artificial Intelligence, 8(5), 5. https://doi.org/10.9781/ijimai.2024.02.011
Anson, C., & Straume, I. (2022). Amazement and Trepidation: Implications of AI-Based Natural Language Production for the Teaching of Writing. Journal of Academic Writing, 12(1), 1–9. https://doi.org/10.18552/joaw.v12i1.820
Bacher, J., Wenzig, K., & Vogler, M. (2004). SPSS TwoStep Cluster—A first evaluation. Universität Erlangen-Nürnberg. https://www.ssoar.info/ssoar/bitstream/handle/document/32715/ssoar-2004-bacher_et_al-SPSS_TwoStep_Cluster_-_a.pdf?sequence=1
Balabdaoui, F., Dittmann-Domenichini, N., Grosse, H., Schlienger, C., & Kortemeyer, G. (2024). A survey on students’ use of AI at a technical university. Discover Education, 3(1), 51. https://doi.org/10.1007/s44217-024-00136-4
Barrett, A., & Pack, A. (2023). Not quite eye to A.I.: Student and teacher perspectives on the use of generative artificial intelligence in the writing process. International Journal of Educational Technology in Higher Education, 20(1), 59. https://doi.org/10.1186/s41239-023-00427-0
Bhatia, P. (2023). ChatGPT for academic writing: A game changer or a disruptive tool? Journal of Anaesthesiology Clinical Pharmacology, 39(1), 1. https://doi.org/10.4103/joacp.joacp_84_23
Burkhard, M. (2022). Student Perceptions of AI-Powered Writing Tools: Towards Individualized Teaching Strategies. 19th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2022), Lissabon, Portugal.
Chiu, T., Fang, D., Chen, J., Wang, Y., & Jeris, C. (2001). A robust and scalable clustering algorithm for mixed type attributes in large database environment. Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 263–268. https://doi.org/10.1145/502512.502549
Črček, N., & Patekar, J. (2023). Writing with AI: University Students’ Use of ChatGPT. Journal of Language and Education, 9(4), 128–138. https://doi.org/10.17323/jle.2023.17379
Flower, L., & Hayes, J. R. (1981). A Cognitive Process Theory of Writing. College Composition and Communication, 32(4), 365. https://doi.org/10.2307/356600
Fyfe, P. (2023). How to cheat on your final paper: Assigning AI for student writing. AI & Society, 38(4), 1395–1405. https://doi.org/10.1007/s00146-022-01397-z
Gelbard, R., Goldman, O., & Spiegler, I. (2007). Investigating diversity of clustering methods: An empirical comparison. Data & Knowledge Engineering, 63(1), 155–166. https://doi.org/10.1016/j.datak.2007.01.002
Helm, G., & Hesse, F. (2024). Usage and beliefs of student teachers towards artificial intelligence in writing. Research in Subject-matter Teaching and Learning (RISTAL), 7(1), 1–18.
Hoffmann, N., & Schmidt, S. (2023). Vorläufige Kurzauswertung der bundesweiten Studierendenbefragung „Die Zukunft des akademischen Schreibens mit KI gestalten“. Goethe Universität Frankfurt am Main. https://www.starkerstart.uni-frankfurt.de/142467510/kurzbericht-akademisches-schreiben-mit-ki.pdf
Hornberger, M., Bewersdorff, A., & Nerdel, C. (2023). What do university students know about Artificial Intelligence? Development and validation of an AI literacy test. Computers and Education: Artificial Intelligence, 5, 100165. https://doi.org/10.1016/j.caeai.2023.100165
Huberty, C. J., Jordan, E. M., & Brandt, W. C. (2005). Cluster Analysis in Higher Education Research. In J. C. Smart (Hrsg.), Higher Education: Handbook of Theory and Research (Bd. 20, S. 437–457). Springer-Verlag. https://doi.org/10.1007/1-4020-3279-X_8
Jensen, L. X., Buhl, A., Sharma, A., & Bearman, M. (2024). Generative AI and higher education: A review of claims from the first months of ChatGPT. Higher Education, 88(1).
Johnson, F., Schmit, J., & Koßmann, J. (2023). Digitalisierungsbezogene Fortbildung in der Lehrer*innenbildung: Wen interessiert’s? 6MB. https://doi.org/10.18716/OJS/KON/2023.4
Kandlhofer, M., Steinbauer, G., Hirschmugl-Gaisch, S., & Huber, P. (2016). Artificial intelligence and computer science in education: From kindergarten to university. 2016 IEEE Frontiers in Education Conference (FIE), 1–9. https://doi.org/10.1109/FIE.2016.7757570
Kaufman, L., & Rousseeuw, P. J. (1990). Finding groups in data: An introduction to cluster analysis. Wiley.
Kelly, A., Sullivan, M., & Strampel, K. (2023). Generative artificial intelligence: University student awareness, experience, and confidence in use across disciplines. Journal of University Teaching and Learning Practice, 20(6). https://doi.org/10.53761/1.20.6.12
Kim, H. (2020). Profiles of undergraduate student writers: Differences in writing strategy and impacts on text quality. Learning and Individual Differences, 78, 101823. https://doi.org/10.1016/j.lindif.2020.101823
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024). AI literacy and its implications for prompt engineering strategies. Computers and Education: Artificial Intelligence, 6, 100225. https://doi.org/10.1016/j.caeai.2024.100225
Kong, S.-C., Cheung, W. M.-Y., & Zhang, G. (2022). Evaluating artificial intelligence literacy courses for fostering conceptual learning, literacy and empowerment in university students: Refocusing to conceptual building. Computers in Human Behavior Reports, 7, 100223. https://doi.org/10.1016/j.chbr.2022.100223
Korthagen, F. (2016). Inconvenient truths about teacher learning: Towards professional development 3.0. Teachers and Teaching, 1–19. https://doi.org/10.1080/13540602.2016.1211523
Kruse, O., Meyer, H., & Everke Buchanan, S. (2015). Schreiben an der Universität Konstanz eine Befragung von Studierenden und Lehrenden. ZHAW.
Kunter, M., Kleickmann, T., Klusmann, U., & Richter, D. (2011). Die Entwicklung professioneller Kompetenz von Lehrkräften. In M. Kunter, J. Baumert, W. Blum, U. Klusmann, S. Krauss, & M. Neubrand (Hrsg.), Professionelle Kompetenz von Lehrkräften. Ergebnisse des Forschungsprogramms COACTIV (S. 55–68). Waxmann.
Laupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023). Development of the “Scale for the assessment of non-experts’ AI literacy” – An exploratory factor analysis. Computers in Human Behavior Reports, 12, 100338. https://doi.org/10.1016/j.chbr.2023.100338
Lipowsky, F., & Rzejak, D. (2021). Fortbildungen für Lehrpersonen wirksam gestalten: Ein praxisorientierter und forschungsgestützter Leitfaden. https://doi.org/10.11586/2020080
Long, D., & Magerko, B. (2020). What is AI Literacy? Competencies and Design Considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16. https://doi.org/10.1145/3313831.3376727
Malmström, H., Stöhr, C., & Ou, W. (2023). Chatbots and other AI for learning: A survey of use and views among university students in Sweden [Application/pdf]. Chalmers University of Technology. https://doi.org/10.17196/CLS.CSCLHE/2023/01
Nisak, K., & Ishlahiyah, M. (2023). AI Chatbots’ role in assisting English literature students’ writing skill. Presented at The 5th Annual International Conference on Linguistics, Literature and Media.
Pajares, F. (2003). Self-efficacy beliefs, motivation, and achievement in writing: A review of the literature. Reading & Writing Quarterly, 19, 139–158.
Pastor, D. A., Barron, K. E., Miller, B. J., & Davis, S. L. (2007). A latent profile analysis of college students’ achievement goal orientation. Contemporary Educational Psychology, 32(1), 8–47. https://doi.org/10.1016/j.cedpsych.2006.10.003
Penman, R. (2007). Media Literacy—Concepts, Research and Regulatory Issues. Australian Communications and Media Authority.
Rubach, C., & Lazarides, R. (2019). Eine Skala zur Selbsteinschätzung digitaler Kompetenzen bei Lehramtsstudierenden: Entwicklung eines Instrumentes und die Validierung durch Konstrukte zur Mediennutzung und Werteüberzeugungen zur Nutzung digitaler Medien im Unterricht. Zeitschrift für Bildungsforschung, 9(3), 345–374. https://doi.org/10.1007/s35834-019-00248-0
Steinhoff, T. (2007). Wissenschaftliche Textkompetenz: Sprachgebrauch und Schreibentwicklung in wissenschaftlichen Texten von Studenten und Experten. de Gruyter. https://doi.org/10.1515/9783110973389
Tossell, C. C., Tenhundfeld, N. L., Momen, A., Cooley, K., & De Visser, E. J. (2024). Student Perceptions of ChatGPT Use in a College Essay Assignment: Implications for Learning, Grading, and Trust in Artificial Intelligence. IEEE Transactions on Learning Technologies, 17, 1069–1081. https://doi.org/10.1109/TLT.2024.3355015
UNESCO. (2021). AI and education: Guidance for policy-makers. UNESCO. https://doi.org/10.54675/PCSP7350
van Waes, L. (1992). The influence of the computer on writing profiles. In H. P. Maat & M. Steehouder (Hrsg.), Studies of functional text quality (S. 173–186).
van Waes, Luuk, & Schellens, P. J. (2003). Writing profiles: The effect of the writing mode on pausing and revision patterns of experienced writers. Journal of Pragmatics, 35, 829–853.
von Garrel, J., Mayer, J., & Mühlfeld, M. (2023). Künstliche Intelligenz im Studium Eine quantitative Befragung von Studierenden zur Nutzung von ChatGPT & Co. [Application/pdf]. 2616 KB, 84 pages. https://doi.org/10.48444/H_DOCS-PUB-395
Welding, L. (2023). Half of College Students Say Using AI on Schoolwork Is Cheating or Plagiarism. Bestcolleges.com. https://www.bestcolleges.com/research/college-students-ai-tools-survey/
Williamson, P. (2023). Is ChatGPT the end of education as we know it? 3 Takeaways from the Panel Discussion. https://www.tomorrow.university/. https://www.tomorrow.university/post/is-chatgpt-the-end-of-education-as-we-know-it
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023). Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–21. https://doi.org/10.1145/3544548.3581388
Published
Issue
Section
License
Copyright (c) 2025 Gerrit Helm, Florian Hesse
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 Unported License.
