Natural Language Processing techniques for researching and improving peer feedback

Abstract

Peer review has been viewed as a promising solution for improving students' writing, which still remains a great challenge for educators. However, one core problem with peer review of writing is that potentially useful feedback from peers is not always presented in ways that lead to revision. Our prior investigations found that whether students implement feedback is significantly correlated with two feedback features: localization information and concrete solutions. But focusing on feedback features is time-intensive for researchers and instructors. We apply data mining and Natural Language Processing techniques to automatically code reviews for these feedback features. Our results show that it is feasible to provide intelligent support to peer review systems to automatically assess students' reviewing performance with respect to problem localization and solution. We also show that similar research conclusions about helpfulness perceptions of feedback across students and different expert types can be drawn from automatically coded data and from hand-coded data.