Using Peer Review

Peer Review

Peer review is a natural fit for science communication settings, because it is such a vital component of the scientific publishing process. Thus, involving students in peer review may not only improve students’ writing, but also their understanding of science. Peer review can also help to make students more responsible and reflective when it comes to their written work¹.

When incorporating writing tasks into science classes, one of the main concerns educators have is the time required to grade and provide effective feedback. This is a greater concern with large classes. If your goal is to provide multiple writing opportunities to students to allow them to practice their writing skills, you may want to consider using peer evaluation to reduce the grading burden. However, it is important to be aware that some students can be mistrusting of peer review, thinking that it places their grades in the hands of unqualified peers ². To reduce the likelihood of your students thinking this, it is important to explain the many positive outcomes that should result from them engaging with peer review.

In this guide, we consider some of the challenges and best practice strategies, before focusing specifically on using peer review to provide students with formative feedback and for grading purposes.

Incorporating Peer Review into Your Class

Some of the greater concerns about incorporating peer review into a science class are highlighted below (Table 1), but you can employ the strategies matched with these to negate or offset them.

Table 1: Best Practice Strategies to Deal with Specific Peer Review Implementation Concerns

Using Peer Review for Formative Assessment

Some specific strategies and things to consider that may help you handle peer review and address related concerns in your class include how you will:

1. Communicate the purpose of the activity with students to gain their support. Obviously, it is not a good idea to suggest that the students are “doing your work for you”. Instead, highlight the benefits of reading and critically analyzing other’s work. Discuss how being a peer reviewer will help them understand grading criteria better and spot weaknesses in writing structure and argumentation, which will in turn help improve their own work³. Emphasize how important it is for people to get feedback on their writing during the revision stage, and that you are helping facilitate that for the entire class.
2. Provide an appropriate incentive to students to encourage them to take the review process seriously. Depending on your class size, you may not have the resources to grade students’ reviews of each other’s work. However, you could set aside a component of the assignment grade to be determined by student perception of how useful the peer review is in helping them revise their work.
3. Train students on providing effective feedback ⁴. You may want to provide detailed guidance in the form of a checklist or rubric so that students know exactly what they should be considering when reviewing someone else’s work or at least provide prompts to make sure they pay attention to important elements of their peers' writing^{5, 6}.
4. Decide how many assignments to ask students to review. If they review two or more assignments, they can compare and contrast them as well as receive a wider range of feedback on their own work. Sometimes comparing different pieces of work helps students provide more objective feedback because they are better able to assess the strengths and weaknesses of writing when they see more than one example. While some research suggests students hold positive attitudes towards peer review after the purpose has been explained ⁷, other work shows that students trust the process more as they get more experience of it ⁸. As a result, incorporating more than one round of peer review should increase the chances that you students will have greater faith in what they are doing, and why they are being asked to do it.
5. Decide whether to include a face-to-face discussion to go along with the paper-based or software-based reviews. It can be useful for students to hear directly from their peers about what was unclear and excellent about their writing assignment. Face-to-face discussions allow for clarification and may even lead to paired brainstorming about revisions. Of course, this would mean that reviews are not anonymous, but there is little evidence that suggests blinding peer reviews is more useful to learners.
6. Encourage students to incorporate their reviewers’ comments into their work. You may want to have students write a cover page, similar to a letter to the editor of a journal, discussing how they incorporated their reviewers’ comments so that they must tackle this task.
7. Organize the logistics of the whole process. Software systems can do just this. These tools will greatly reduce the burden of organizing the reviews if you have a large class.

An Example of Peer Review in Communicating Science (SCIE 300) at UBC

Here we describe an example of how peer review is used in a third-year communicating science class at our institution. Students work individually to prepare a research paper written in scientific journal style. Their papers are based on a small-scale scientific investigation they performed in groups of 3-4. Four days after the draft papers are due as submissions to the course learning management system, students bring two copies of their paper to class. While students are working on an in-class activity, the instructor distributes the papers to the reviewers. We make sure that students in the same group are not assigned to review each other’s work and ensure that the review class peer discussions will be able to occur without anyone being left out of the discussion.

We give students a rubric and ask them, as homework, to mark up the papers as needed. Four days later, we pair students up in class to discuss their reviews. Each pair has ten minutes to discuss one paper, and then they switch.

The key features that make this process run smoothly are:

1. Students are accountable and therefore invested in the process because they need to submit their draft online before coming to class. There are a few buffer days for late submissions.
2. We provide a rubric to guide students’ reviews.
3. We provide guidance about exactly what the pairs should be discussing.

Do you have examples of effective peer review that you would like to share? Please contact us here if so.

Using Peer Review for Grading Purposes

While having students grade each other’s work can be a time-saving measure, it takes time and thought to incorporate peer evaluation into your class successfully. Here we describe some best practices to consider before implementing peer evaluation with your students. Benefits to peer evaluation include increased responsibility and autonomy, improvement in critical thinking, and improvement of structured thinking.

Many of the same considerations must be given when using peer review for grading purposes as for formative purposes (discussed above). However, there are some additional things to consider, and these are listed below:

1. As above (in the Using Peer Review for Formative Assessment section), it is not a good idea to suggest that the students are “doing your work for you”. Instead highlight the benefits of reading and critically analyzing others’ work.

Be aware that that some students may have negative perceptions of peer evaluation and perceive it as unfair (Smith et al. 2002, Kaufman and Schunn 2011). However, these perceptions may change positively as a result of participating in peer evaluation (Wen and Tsai 2006), so it is important to ensure that students get something positive from their first experiences of the process.

2. As above, it is important to provide an incentive to students to encourage them to take the grading process seriously.

However…

3. It is very important to limit the peer evaluation component of the students’ overall course grades to reduce the perception that their grade is mostly left up to a non-expert or to chance. This is critical when you are piloting peer evaluation in your class.


4. Be sure to have a plan in place for how to deal with student complaints. You may want to set up a formal grievance process.

5. Consider whether your peer evaluation is going to be anonymous or not. Most of the software systems below are anonymous. If not anonymous, consider how you will deal with conflicts of interest, such as two best friends being assigned to each other’s work.

Using Software or Calibrated Peer Review Software Systems

What follows is a brief overview of software systems you may want to consider. The benefit of using a software program to handle peer evaluation is that they can simplify the logistics of assigning, collecting, and returning grades, and providing feedback. Using these systems requires a considerable initial time commitment to set up the assignment and grading rubric(s), but the time savings come later when they construct grades for you and provide feedback directly to students. It is also very important to ensure you make it clear to your students that any software systems used for peer review offer learning benefits to them ⁹, as well as smoothing the assignment logistics for you.

1. Calibrated Peer Review. The CPR software was created at UCLA and takes users through a sequence of submitting their own work, training on grading other’s work (“calibration”) and grading their peer’s work. There is a bank of assignments and it is easy to create your own. Check out the extensive list of publications on the CPR website.
2. iPeer (UBC). iPeer is an open-source web-based software application that allows instructors to create assignments and rubrics, send reminders to students, and provide feedback. Evaluations can be based on rubrics, and the system can be used for evaluating the contributions of group members in team projects.
3. PeerMark (Turnitin). This component of Turnitin distributes student work for peer grading according to instructor-given criteria. Reviewers may comment and add editing remarks. Your institution may have a license with Turnitin, and it may be integrated with your learning management system (LMS). At UBC, Turnitin is not integrated with the LMS because Turnitin stores the data in the United States. Be sure only anonymous student work is uploaded to Turnitin if your institution is in Canada.
4. MyWritingLab (Pearson). This proprietary software allows for “facilitated peer review”, which means that students may provide comments on other students’ work, as well as grade it using an instructor-provided rubric.
5. Peerceptiv (formerly SWoRD Peer Assessment). Peerceptiv engages students in double-blind reviews using instructor-created rubrics. This system claims to motivate students, eliminate bias, and generate useful analytics for instructors. Student work is graded by three to six peers and students have the opportunity to “back evaluate” to rate the helpfulness and specificity of the review. SWoRD has been heavily researched by the University of Pittsburgh and is offered by Panther Learning.

Do you have other examples of peer evaluation software that you have had success with? Please contact us here if so.

Useful References

1. Dochy, F, Segers, M, Sluijsmans, D. The use of self-, peer and co-assessment in higher education: A review. Stud High Educ. 1999; 24(3):331–350.

2. Kaufman, JH, Schunn, CD. Students’ perceptions about peer assessment for writing: their origin and impact on revision work. Instr Sci. 2011; 39:387-406.

3. Cho, YH, Cho, K. Peer reviewers learn from giving comments. Instr Sci. 2011; 39:629-643.

4. Stanley, J. Coaching student writers to be effective peer evaluators. J Sec Lang Writ. 1992; 1(3):217-233.

5. Cho, K, Schunn, CD, Wilson, RW. Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives. 'J Educ Psych. 2006; 98(4):891-901.

6. Gan, MJS, Hattie, J. Prompting secondary students' use of criteria, feedback specificity and feedback levels during an investigative task. Instr Sci. 2014; 42:861-878.

7. Keaten, JA, Richardson, ME. A field investigation of peer assessment as part of the student group grading process, paper presented at the Western Speech Communication Association Convention. 1992; (pp. 1–34). Presented at the Western Speech Communication Association Convention, Albuquerque.

8. Smith, H, Cooper, A, Lancaster, L. Improving the quality of undergraduate peer assessment: A case for student and staff development. Innov Educ Teach Int. 2002; 39(1):71–81.

9. Wen, ML, Tsai, CC. University students’ perceptions of and attitudes toward (online) peer assessment. High Educ. 2006; 27(18):27–44.