Baseline assessment in writing research: A case study of popularization discourse in first-year undergraduate students

Authors

  • Florentine Marnel Sterk Utrecht University | The Netherlands
  • Merel Margot Van Goch Utrecht University | The Netherlands
  • Michael Burke Utrecht University | The Netherlands
  • Iris Van der Tuin Utrecht University | The Netherlands

DOI:

https://doi.org/10.17239/jowr-2022.14.01.02

Keywords:

baseline assessment, popularization discourse, student writing, science communication, higher education

Abstract

In popularization discourse, insights from academic discourse are recontextualized and reformulated into newsworthy, understandable knowledge for a lay audience. Training in popularization discourse is a relatively new and unexplored research topic. Existing studies in the science communication field suffer from under-utilized baseline assessments and pretests in teaching interventions. This methodological problem leads both to a lack of evidence for claims about student progress and to a gap in knowledge about baseline popularization skills. We draw the topic into the realm of writing research by conducting a baseline assessment of pre-training popularization skills in first-year undergraduate students. Undergraduate science communication texts are analyzed to identify instances of popularization strategies using a coding scheme for text analysis of popularization discourse. The results indicate a lack of genre knowledge in both academic and popularized discourse: textual styles are either too academic or overly popularized; the academic text is misrepresented; and the essential journalistic structure lacking. An educational program in popularization discourse should therefore focus on the genre demands of popularization discourse, awareness of academic writing conventions, the genre change between academic and popularized writing, the role of the student as a writer, and stylistic attributes.

References

Baram-Tsabari, A., & Lewenstein, B. V. (2017). Science communication training: What are we trying to teach? International Journal of Science Education, Part B, 7(3), 285-300. https://doi.org/10.1080/21548455.2017.1303756

Berezow, A. (2017, March 5). Infographic: The best and worst science news sites. American Council on Science and Health. https://www.acsh.org/news/2017/03/05/infographic-best-and-worst-science-news-sites-10948

Besley, J. C., & Tanner, A. H. (2011). What science communication scholars think about training scientists to communicate. Science Communication, 33(2), 239-263.

https://doi.org/10.1177/1075547010386972

Boettger, R. K. (2014). Explicitly teaching five technical genres to English first-language adults in a multi-major technical writing course. Journal of Writing Research, 6(1), 29-59. https://doi.org/10.17239/jowr-2014.06.01.2

Bouwer, R., Koster, M., & Van den Bergh, H. (2018). Effects of a strategy-focused instructional programme on the writing quality of upper elementary students in the Netherlands. Journal of Educational Psychology, 110(1), 58-71. https://doi.org/10.1037/edu0000206

Boynton, S. (2018). The appliance of science: The challenges of undergraduate science writing popular science. The Asian Journal of Applied Linguistics, 5(1), 4-18. Retrieved from: https://caes.hku.hk/ajal/index.php/ajal/article/view/513

Braaksma, M., Rijlaarsdam, G., & Van den Bergh, H. (2018). Effects of hypertext writing and observational learning on content knowledge acquisition, self-efficacy, and text quality: Two experimental studies exploring aptitude treatment interactions. Journal of Writing Research, 9(3), 259-300. https://doi.org/10.17239/jowr-2018.09.03.02

Bray, B., France, B., & Gilbert, J. K. (2012). Identifying the essential elements of effective science communication: What do the experts say? International Journal of Science Education, Part B, 2(1), 23-41. https://doi.org/10.1080/21548455.2011.611627

Bromley, A. P., Boran, J. R., & Myddelton, W. A. (2007). Investigating the baseline skills of research students using a competency-based self-assessment method. Active Learning in Higher Education, 8(2), 117-137. https://doi.org/10.1177/1469787407077992

Brownell, S. E., Price, J. V., & Steinman, L. (2013a). A writing-intensive course improves biology undergraduates’ perception and confidence of their abilities to read scientific literature and communicate science. AJP: Advances in Physiology Education, 37(1), 70-79. https://doi.org/10.1152/advan.00138.2012

Brownell, S. E., Price, J. V., & Steinman, L. (2013b). Science communication to the general public: Why we need to teach undergraduate and graduate students this skill as part of their formal scientific training. Journal of Undergraduate Neuroscience Education, 12(1), E6-E10. Retrieved from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3852879/

Bruno, F., & Vercellesi, L. (2002). Science information in the media: An academic approach to improve its intrinsic quality. Pharmacology Research, 45(1), 51-55.

https://doi.org/10.1006/phrs.2001.0901

Calsamiglia, H., & Van Dijk, T. A. (2004). Popularization discourse and knowledge about the genome. Discourse & Society, 15(4), 369-389. https://doi.org/10.1177/0957926504043705

Ciapuscio, G. E. (2003). Formulation and reformulation procedures in verbal interactions between experts and (semi-)laypersons. Discourse Studies, 5(2), 207-233.

https://doi.org/10.1177/1461445603005002004

Cirino, L. A., Emberts, Z., Joseph, P. N., Allen P. E., Lopatto, D., & Miller, C. W. (2017). Broadening the voice of science: Promoting scientific communication in the undergraduate classroom. Ecology and Evolution, 7, 10124-10130. https://doi.org/10.1002/ece3.3501

Early, J. S., & De Costa, M. (2011). Making a case for college: A genre-based college admission essay intervention for underserved high school students. Journal of Writing Research, 2(3), 299-329. https://doi.org/10.17239/jowr-2011.02.03.2

Gotti, M. (2014). Reformulation and recontextualisation in popularization discourse. Ibérica, 27, 15-34. Retrieved from: https://www.researchgate.net/publication/288159802_ Reformulation_and_recontextualization_in_popularization_discourse

Graham, S. E., & Harris, K. R. (2014). Conducting high quality writing intervention research: Twelve recommendations. Journal of Writing Research, 6(2), 89-123.

https://doi.org/10.17239/jowr-2014.06.02.1

Graham, S. E., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99(3), 445-476. https://doi.org/10.1037/0022-0663.99.3.445

Heath, K. D., Bagley, E., Berkey, A. J. M., Birlenbach, D. M., Carr-Markell, M. K., Crawford, J. W., … Wesseln, C. J. (2014). Amplify the signal: Graduate training in broader impacts of scientific research. BioScience, 64(6), 517-523. https://doi.org/10.1093/biosci/biu051

Hundey, E. J., Olker, J. H., Carreira, C., Daigle, R. M., Elgin, A. K., Finiguerra, M., … Wood-Charlson, E. M. (2016). A shifting tide: Recommendations for incorporating science communication into graduate training. Limnology and Oceanography Bulletin, 25, 109-116. https://doi.org/10.1002/lob.10151

Hyland, K. (2010). Constructing proximity: Relating to readers in popular and professional science. Journal of English for Academic Purposes, 9(2), 116-127.

https://doi.org/10.1016/j.jeap.2010.02.00

Kloepper, L. N. (2017). We need to teach science communication to graduate and undergraduate students. Here’s how. Proceedings of Meetings on Acoustics, 30(1), 025003. https://doi.org/10.1121/2.0000569

Koster, M., Tribushinina, E., de Jong, P. F., & Van den Bergh, H. (2015). Teaching children to write: A meta-analysis of writing intervention research. Journal of Writing Research, 7(2), 249-274. https://doi.org/10.17239/jowr-2015.07.02.2

Kuiper, C., Smit, J., De Wachter, L., & Elen, J. (2017). Scaffolding tertiary students’ writing in a genre-based writing intervention. Journal of Writing Research, 9(1), 27-59. https://doi.org/10.17239/jowr-2017.09.01.02

Kyriakides, L. (1999). Research on baseline assessment in mathematics at school entry. Assessment in Education: Principles, Policy & Practice, 6(3), 357-375. https://doi.org/10.1080/09695949992793

Latimore, J. A., Dreelin, E. A., & Burroughs, J. P. (2014). Outreach and engagement education for graduate students in natural resources: Developing a course to enrich a graduate outreach requirement. Journal of Higher Education Outreach and Engagement, 18(3), 129-154. Retrieved from: https://www.researchgate.net/publication/274022968_Outreach_ and_Engagement_Education_for_Graduate_Students_in_Natural_Resources_Developing_a_Course_to_Enrich_a_Graduate_Outreach_Requirement

Luna, M., Villalón, R., Mateos, M., & Martín, E. (2020). Improving university argumentative writing through online training. Journal of Writing Research, 12(1), 233-262. https://doi.org/10.17239/jowr-2020.12.01.08

Luzón, M. J. (2013). Public communication of science in blogs: Recontextualizing scientific discourse for a diversified audience. Written Communication, 30(4), 428-457. https://doi.org/10.1177/0741088313493610

McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276-282. https://doi.org/10.11613/BM.2012.031

McKinnon, M., & Bryant, C. (2017). Thirty years of a science communication course in Australia. Science Communication, 39(2), 169-194. https://doi.org/10.1177/1075547017696166

Mercer-Mapstone. L. D., & Kuchel, L. J. (2015a). Core skills for effective science communication: A teaching resource for undergraduate science education. International Journal of Science Education Part B, 7(2), 181-201.

https://doi.org/ 10.1080/21548455.2015.1113573

Mercer-Mapstone, L. D., & Kuchel, L. J. (2015b). Teaching scientists to communicate: Evidence-based assessment for undergraduate science education. International Journal of Science Education, 37(10): 1613-1638. https://doi.org/10.1080/09500693.2015.1045959

Mercer-Mapstone, L. D., & Kuchel, L. J. (2016). Integrating communication skills into undergraduate science degrees: A practical and evidence-based approach. Teaching & Learning Inquiry: The ISSOTL Journal, 4(2), 1-14.

https://doi.org/10.20343/teachlearninqu.4.2.11

Mellor, F. (2013). Twenty years of teaching science communication: A case study of Imperial College’s Master’s programme. Public Understanding of Science, 22(8), 916-926. https://doi.org/10.1177/0963662513489386

Miller, R. E., & Spellmeyer, K. (2014). The new humanities reader (5th ed.). Cengage Learning.

Moni, R. W., Hryciw, D. H., Poronnik, P., & Moni, K. B. (2007). Using explicit teaching to improve how bioscience students write to the lay public. Advances in Physiology Education, 31, 167-175. https://doi.org/10.1152/advan.00111.2006

Myers, G. (2003). Discourse studies of scientific popularization: Questioning the boundaries. Discourse Studies, 5(2), 265-279. https://doi.org/10.1177/1461445603005002006

Nutbrown, C. (1999). Baseline assessment of writing: The need for reconsideration. Journal of Research in Reading, 22(1), 37-44. https://doi.org/10.1111/1467-9817.00067

O’Keeffe, K., & Bain, R. (2018). ComSciCon-Triangle: Regional science communication training for graduate students. Journal of Microbiology & Biology Education, 19(1), 1-7. https://doi.org/10.1128/jmbe.v19i1.1420

Poronnik, P., & Moni, R. W. (2006). The opinion editorial: Teaching physiology outside the box. Advances in Psychological Education, 30(2), 73-82. https://doi.org/10.1152/advan.00075.2005

QSR International Pty Ltd. (Version 12, 2018). NVivo qualitative data analysis software. [Computer software].

Rakedzon, T., & Baram-Tsabari, A. (2017). Assessing and improving L2 graduate students’ popular science and academic writing in an academic writing course. Educational Psychology, 37(1), 48-66. https://doi.org/10.1080/01443410.2016.1192108

Rietdijk, S., Janssen, T., Van Weijen, D., Van den Bergh, H., & Rijlaarsdam, G. (2017). Improving writing in primary schools through a comprehensive writing programme. Journal of Writing Research, 9(2), 173-225. https://doi.org/10.17239/jowr-2017.09.02.04

Saldaña, J. (2015). The coding manual for qualitative researchers (3rd ed.). SAGE Publications.

Sammons, P., & Smees, R. (1998). Measuring pupil progress at key stage 1: Using baseline assessment to investigate value added. School Leadership & Management, 18(3), 389-407. https://doi.org/10.1080/13632439869574

Schmidt, C. (2004). The analysis of semi-structured interviews. In U. Flick, E. von Kardoff & I. Steinke (Eds.), A companion to qualitative research (pp. 253-258). SAGE Publications.

Shivni, R., Cline, C., Newport, M., Yuan, S., & Bergan-Roller, H. E. (2021). Establishing a baseline of science communication skills in an undergraduate environmental science course. International Journal of STEM Education, 8, Article 47. https://doi.org/10.1186/s40594-021-00304-0

Sivey, J. D., & Lee, C. M. (2008). Using popular magazine articles to teach the art of writing for nontechnical audiences. Journal of Chemical Education, 85(1), 55-58.

https://doi.org/10.1021/ed085p55

Sterk, F. M., Van Goch, M. M., Burke, M., & Van der Tuin, I. (forthcoming). Analyzing Popularization Discourse: Developing a Coding Scheme for Popularization Strategies.

Trench, B. (2009). Masters (MSc) in science communication. Dublin City University. Journal of Science Communication, 8(1), C05. https://doi.org/10.22323/2.08010305

Tymms, P. (1999). Baseline assessment, value-added and the prediction of reading. Journal of Research in Reading, 22(1), 27-36. https://doi.org/10.1111/1467-9817.00066

Van Drie, J., Braaksma, M., & Van Boxtel, C. (2015). Writing in history: Effects of writing instruction on historical reasoning and text quality. Journal of Writing Research, 7(1), 123-156. https://doi.org/10.17239/jowr-2015.07.01.06

Wall, D., & Horák, T. (2007). Using baseline studies in the investigation of test impact. Assessment in Education, 14(1), 99-116. https://doi.org/10.1080/09695940701272922

Whittington, C. P., Pellock, S. J., Cunningham, R. L., Cox, J. R. (2013). Combining content and elements of communication into an upper-level biochemistry course. Biochemistry and Molecular Biology Education, 42(2), 136-141. https://doi.org/10.1002/bmb.20770

Wilkinson, J. E., Johnson, S., Watt, J., Napuk, A., & Normand, B. (2001). Baseline assessment in Scotland: An analysis of pilot data. Assessment in Education: Principles, Policy & Practice, 8(2), 171-192. https://doi.org/10.1080/09695940124706

Woods, H. C., & Scott, H. (2016). #Sleepyteens: Social media use in adolescence is associated with poor sleep quality, anxiety, depression and low self-esteem. Journal of Adolescence, 51, 41-49. https://doi.org/10.1016/j.adolescence.2016.05.008

Yeoman, K. H., James, H. A., & Bowater, L. (2011). Development and evaluation of an undergraduate science communication module. Bioscience Education, 17(1), 1-16. https://doi.org/10.3108/beej.17.7

Published

2022-04-18

How to Cite

Sterk, F. M., Van Goch, M. M., Burke, M., & Van der Tuin, I. (2022). Baseline assessment in writing research: A case study of popularization discourse in first-year undergraduate students. Journal of Writing Research, 14(1), 35–76. https://doi.org/10.17239/jowr-2022.14.01.02

Issue

Section

Articles