Peer Review With Digital Tools

Interactions are a key component of student learning in online and face-to-face settings including student-instructor interactions, student-content interactions, and student-student interactions. Peer review is one type of student-student interaction that can build classroom community and deepen student learning (Nilson 2018). Feedback is a crucial part of learning but doesn’t only have to come from instructors –peer feedback can potentially save some time for instructors to do other important work with students (Brigati & Swann 2015). Guiding students to useful, productive peer feedback strengthens student performance and provides students with more feedback for learning growth.

Workshops

How to Get Started

Step 1:

  • Start the peer-review process with students by explaining the purpose of using peer review in your course. Stay positive and share why this is a worthwhile learning experience. Share that peer feedback should be descriptive (not evaluative) and the goal is for it to be formative, meaning that it will help change and develop future work that students submit.
  • Determine the flow of the peer review. Specifically —
    • Will the peer feedback be graded or earn points in the class?
    • What should the size of peer groups be and how many reviews should each assignment receive?
    • Create a peer review feedback sheet or co-create wone with students –be sure it is aligned with goals/objectives for the assignment and any final evaluations that will occur.
  • Examples of peer review activities from the Center for Teaching Innovation at Cornell:
    • After collection, redistribute papers randomly along with a grading rubric. After students have evaluated the papers ask them to exchange with a neighbor, evaluate the new paper, and then compare notes.
    • After completing an exam, have students compare and discuss answers with a partner. You may offer them the opportunity to submit a new answer, dividing points between the two.
    • In a small class, ask students to bring one copy of their paper with their name on it and one or two copies without a name. Collect the “name” copy and redistribute the others for peer review. Provide feedback on all student papers. Collect the peer reviews and return papers to their authors.
    • For group presentations, require the class to evaluate the group’s performance using a predetermined marking scheme.
    • When working on group projects, have students evaluate each group member’s contribution to the project on a scale of 1-10. Require students to provide rationale for how and why they awarded points.

Step 2:

  • Provide student models/examples of quality feedback and guidance for what students should do and what their feedback should “look like.” Align the feedback examples and expectations with learning objectives for students on the assignment requiring peer feedback. In the examples you share with students, model clear, constructive feedback with positive feedback as a first step. Note: the positive feedback is not praise about the student or person but specific to the task. 
  • Infographic example of “How to Give Awesome Feedback”
  • Teach the difference between specific and non-specific feedback/comments:
    Specific Feedback Non-Specific Feedback
    Your voice in your presentation was loud and the pacing made me able to follow your content but in the 2nd section I was confused by some of the terminology. Can you explain terms more or use terminology from class? I didn’t get it. The presentation is confusing.
    I found 2 piece of strong evidence from journals in your 2nd paragraph. They are connected to your topic sentence/subject but I think we are supposed to have three pieces of evidence You didn’t do the assignment right.
    Your infographic visuals are engaging to look at and the color-contrast makes them standout. The infographic looked great!
    The FAQ section of your website has two useful questions and answers but you should have at least five to ten. Maybe think about your website from the audience perspective to find more questions to answer or look at other FAQ examples. There isn’t enough information or questions/answers.
    The point you made during your speech about….made it clear that…… It was a good speech.
    I still have questions about …. But the section where you explained….is very clear. This section was unclear, you should write more.

Step 3:

Step 4:

  • Choose a digital tool to support the peer review process —see the digital tool section (tab) for details. Recommended tools include PlayPosit Peer Review, Moodle Workshop Activity, Turnitin PeerMark, Google Workspace Tools

Step 5:

  • Ideally allow students to keep revising and working with the peer feedback to make improvements in their work. This may mean fewer assignments get done in a class but students may show more growth and improvement in skills/deeper understanding of course concepts.

Best Practices

  • Provide training to students and guided worksheets or reference materials to help them give better quality feedback. As part of this process, include examples and models of what effective feedback looks like.
  • Explain the purpose of peer review to students, helping students realize the role of a peer-reviewer is to read and describe and not to evaluate or grade (Washington University Center for Teaching and Learning 2022). Consider asking students to analyze, react to, and describe their peers’ work rather than grade or evaluate.
  • In guides, worksheets, or expectations ask students to give positive feedback first when describing the work of their peers (Nilson & Goodson 2018).
  • In an online course, give a time estimate for how long student should expect to spend on a peer review. This could be part of a partnership contract for peer review (you can also consider having students design contracts together).
  • Consider these guides and practitioner suggestions:

Digital Tools: Feature Comparison

Moodle Workshop Activity: Peer Review Activity for Multiple Submission Types

Moodle’s workshop activity enables the collection, review and peer assessment of students’ work. Students can submit digital files (such as documents, slides, spreadsheets, or other attachments) or type text directly into a field using the text editor. Then, depending on instructor settings and guidance, students assess each other and give comments/feedback to their peers and/or give grades to their peers. After the peer-assessment window closes, instructors review the grades given to students, make any adjustments, and then post the grades to the Moodle gradebook.

Moodle Workshop has a complicated set up procedure and a bit of a learning curve for instructors and students. Therefore it is best if you plan on using it multiple times in a course. Workshop is particularly useful for large classes in which an instructor wants to get make grading more efficient by using students to grade their peers (note: there is research to show that peer grading can be valid and reliable).

If you only want students to provide feedback to each other and have an instructor provide the grade, be sure to use the “Comments” grading strategy when setting up the Workshop activity. Then have students turn in a revised copy of an assignment in an separate Moodle assignment activity so the instructor can provide the final grade.

Student submissions can be assessed by their peers in multiple ways depending on how an instructor sets up the activity. Peers can use a “checklist” of criteria or a rubric with specific quality indicators – both must be defined by the instructor. Students are given the opportunity to assess one or more of their peers’ submissions. Submissions and reviewers may be anonymous. Students obtain two grades in a workshop activity — a grade for their submission and a grade for their assessment of their peers’ submissions. Both grades are recorded in the Moodle Gradebook. 

Turnitin PeerMark: Peer Review Tool for Writing Assignments/Papers

PeerMark is a peer review assignment tool for essays, papers, and other writing assignments. Instructors can create and manage PeerMark assignments that allow students to read, review, and evaluate one or many papers submitted by their classmates. Reviews can be anonymous.

Instructors will create a Turnitin Assignment 2 within Moodle and then add a PeerMark assignment to the original assignment. After students have submitted writing assignments they will be given peer’s submissions to provide feedback. An instructor can determine pairings/groups for peer review or allow them to be assigned by the Turnitin system. An instructor can provide questions to guide the review – these questions can be open-ended or associated with points. Inline and pop-up comments can also be added by peers. Note: if instructors want to assign grades to students based on their work doing peer reviews, those grades will have to be added to Moodle manually.

NOTE: Turnitin does not work on a mobile device

Google Workspace Tools: Peer Reviews not Synced with Moodle

Each of the primary Google Workspace Tools: Docs, Slides, Sheets allow for comments and assigning tasks to students. These features can help students with peer review as can annotating text by highlighting and underlining with different colors. In Google Docs, additional peer review tools include the ability to add a watermark i.e. “Draft” and use the “Suggesting” mode to make changes directly to a document that can be accepted or declined by the author/owner of the document.

For workflow, students can also create a tickable checkbox/schedule for peer review at the top of the document. Instructors can also create templates for students to use when drafting that have built in headings, question, etc. for peers to consider when reviewing — ask students to make a copy of a document or you can create a “forced copy” version of a document. Instructors can also use shared folders to organize peer reviews. Using Google Workspace tools for peer review will not automatically add grades to Moodle –instructors would need to manually add grades to Moodle. 

Features within Google Tools:

Guidance Using Google Tools in Conjunction with Moodle:

  • Google Groups: Create a class folder in Drive and then share it or group folders with students –groups can be managed through Wolfware
  • Use a Discussion Forum/Board: Students submit work and use the forum to give each other feedback —note: feedback will be viewable to peers. Instructors can also create forums for small peer groups. A rubric can be attached to the Discussion Forum via “Advanced Grading”
  • Moodle Assignments: Students independently use Google commenting, suggestion, annotation tools and upload a marked copy to a Moodle Assignment. A rubric can be attached to the assignment via “Advanced Grading”
  • Google Assignments: Allows students to turn in work directly from Google Drive and it controls the sharing/access to documents but does not have a built in peer/collaboration feature. Use the template option and ask students to turn in the work with the peer comments. For more on Google Assignments in general use this instructor guide.
  • Manual Grading in Moodle: If students are using a Google Form or shared documents in Drive folders and you want to add a grade to Moodle, add a grade item and enter the grades manually. Instructors can add multiple grade items for the creator/writer’s work and for the peers who are reviewing.

Video Project Student Submission and Peer Review

PlayPosit is an interactive video creator allowing instructors to overlay questions, discussion, web content, and other interactions on top of a video creating opportunities for students to interact with course content.

The Peer Review feature allows students to upload videos and then provide feedback on peer-submitted videos and get peer feedback for their own work. This tool is particularly useful for performance-based assignments.

Students can submit a recording of their own performance in a speech, presentation, micro-teaching session, model job interview, artistic performance, etc. and receive peer feedback on that work. Instructors can provide different levels of guidance in how students should provide feedback and feedback can be anonymous. PlayPosit communicates with the Moodle Gradebook through a “completion” grade just for uploading a video and giving comments OR through a graded assignment in which the instructor gives students comments and/or feedback via an embedded rubric.

Panopto is the NC State enterprise tool for recording, storing, and sharing videos and audio. Both students and instructors can record in Panopto and then share their work with others within and outside of NC State.

Instructors can add a “Panopto Student Submission” activity in a Moodle course. This will allow students to add a video from Panopto (they can record within Panopto or upload a video to Panopto). However, in the Panopto Student Submission activity type, student-submitted videos are only viewable to the instructor.

If instructors want students to view each others work and give feedback, they can create and use the “Assignment Folder” feature built into Panopto.

Research: Books & Articles

Boyd, J. P., Dominelli, A., & Polimeni, J. (2022, January 19). Teach One, Save One: The Unlimited Power of Peer Education. Faculty Focus. Retrieved 7 February 2022, from https://www.facultyfocus.com/articles/course-design-ideas/teach-one-save-one-the-unlimited-power-of-peer-education/?st=FFWeekly%3Bsc%3DFFWeekly220121%3Butm_term%3DFFWeekly220121 

Brigati, J. R., & Swann, J. M. (2015). Facilitating improvements in laboratory report writing skills with less grading: a laboratory report peer-review process. Journal of microbiology & biology education, 16(1), 61–68. https://journals.asm.org/doi/full/10.1128/jmbe.v16i1.884  

Hattie, J., & Clarke, S. (2018). Visible Learning: Feedback (1st ed.). Routledge. https://doi-org.prox.lib.ncsu.edu/10.4324/9780429485480 

Madland, C. & Richards, G. (2016). Enhancing Student-Student Online Interaction: Exploring the Study Buddy Peer Review Activity. International Review of Research in Open and Distributed Learning, 17(3), 157–175. https://doi.org/10.19173/irrodl.v17i3.2179 

Mcleod, M., Grabill, J., & Hart-Davidson, B. (2022) Feedback and Revision: The Key Components of Powerful Writing Pedagogy. ELI Review. Retrieved 7 February 2022, from https://elireview.com/content/td/feedback/ 

Nilson, L. B., & Goodson, L. A. (2018). Online teaching at its best: Merging instructional design with teaching and learning research. https://catalog.lib.ncsu.edu/catalog/NCSU4049218 

Back to Top