Grading student papers with efficiency is a vital skill for instructors to hone, but providing effective feedback on these papers is every bit as important. Without detailed, personal feedback, students may feel as though their efforts have gone unnoticed, and, more importantly, they may not know how to address conceptual weaknesses in future work. However, instructors and TA’s have limited time available for grading, especially when teaching large classes.
In this podcast, we hear from two instructors with experience in balancing the grading load and providing feedback effectively and efficiently. They pass on their own tips to adopt or adapt, and also discuss other best-practice ideas to help plan how to provide feedback after grading different types of assignments.
We have created two complementary resources that include useful, summary tips that may prove helpful in shaping your grading and feedback plans, and in designing and calibrating rubrics to limit grading queries when the papers are already piled high on your desk. These are both available to download once you have contacted a site administrator here. Once you have provided your details (including a verifiable academic institution email address) you will receive a password that will enable you to download the materials.
Grading and Providing Effective Feedback Handy Hints
The following two handy hints should help you provide effective feedback while saving as much time as possible when grading.
1: Use an Annotation Key
When providing feedback on open-ended questions and essays, you might wish to use an annotation key to highlight similar errors, rather than explaining them again and again in your written feedback (e.g. choose one shorthand symbol for grammatical issues, another for content weaknesses, and another for poor logical development or argumentation etc.).
Spend some time explaining each error that has its own symbol when it first occurs, but afterwards simply mark repeated errors with the same annotated symbol. For example, if you notice a student shifts tense repeatedly within his/her sentences, and this makes the writing hard to follow, you should explain why this is the case at the point in the writing where this first occurs. You may wish to then provide an edited version of the sentence to make it clear what you mean, but try to refrain from editing too much of a student’s work (this is very time consuming and isn’t your job). When the student next shifts tense inappropriately, simply use the same symbol to indicate this.
Consider using different coloured pens for different errors. Depending on the assignment and its length, this can be more trouble than it is worth, and it does make things more time consuming. However, if an assignment is relatively short or if you are only assessing skills relating to a few concepts, it may provide a quick and easy way to group similar errors and/or feedback for improvement together.
Always remember to include the annotation key so that students understand which specific error/weakness relates to each symbol and/or colour.
Use the same symbols and/or colours for the same errors/weaknesses for all assignments that you grade. This will help you save time and may also encourage students to work together to improve their own writing via peer review.
2: Provide a Brief Feedback Summary
When you provide a final summary of feedback, try to make comments that could be used to improve the piece of writing, rather than attempting to edit the writing or list each and every specific weakness, which your annotation key and its symbols should already do.
Focus your feedback on one or two key areas of improvement. If a student’s work needs discussion beyond this, consider asking/requiring them to addend office hours to go over feedback in person because this will often be quicker and easier for both parties. When choosing what to focus on, target higher-order concerns over lower-order concerns (see Table 1).
Table 1: Higher-order concerns are more important in dictating the quality of written work, whereas lower-order concerns can usually be addressed more easily in future writing assignments once students are aware of their specific areas of weakness.
| Example Higher-Order Concerns | Example Lower-Order Concerns |
|---|---|
| Lack of organization (paragraph issues) | Overuse of jargon |
| Poor thesis and development statements | Overuse of the passive voice |
| Weak evidence provided for any claims | Use of inappropriate analogies |
| No citations, or errors in citations | Lack of punctuation |
| Poor logical development/argumentation | Inappropriate style for numbers and units |
| Poor grammar | Spelling errors |
It is easy to overlook the need to praise students for things they do well, especially when time is limited, but it is important to let them know what they are good at. This not only encourages them but also makes sure they don’t spend time trying to improve something that is already satisfactory, so make sure you add positive elements to your feedback.
Encourage students to keep lists of the things they need to improve, so that they focus on these the next time it comes to completing a similar assignment. Suggest that they should read your feedback from past assignments when editing/revising their work to reduce the chance that they make similar errors again (this is common).
Grading and Providing Feedback: Training Teaching Assistants – Calibration
A major concern when grading assignments in large classes/sections is the need to divide the grading load among instructors and teaching assistants, while maintaining consistency. This guide is designed to help you run a calibration workshop to achieve that consistency.
Before
1: Design a specific rubric
Design a rubric that provides an objective guide as to how every mark should be distributed. Start by stating what a student needs to include in their answer to obtain 1 mark in a question. Then, show specifically what additional material they need to include to obtain all variations in marks, up to the maximum available. Such an approach should leave a grader in no doubt as to what they should award any given answer.
If possible, do this for open-ended essay questions, as well as short-answer questions. Providing ambiguous criteria, such as award between 5 marks (for exceptional transitions) and 0 marks (for no transitions or very weak transitions) will result in very confused graders. Conversely, stating something quantifiable (such as count the number of individual errors in transitions and deduct 1 mark for each from a starting 5 marks) makes it easier for graders to be objective.
2: Select a handful of papers to be graded by everyone
If the assignment has been completed in previous years or terms, select a variety of papers that showcase a range of grades (some high, some low, and some in the middle). Try to choose 3-5 papers to give enough of a sample and provide variety without providing so many that graders are overworked before the real grading begins.
If the assignment is a new one, compile some fictional, expected answers that should obtain a range of grades when graded with your rubric.
Ask your graders to work alone and use your rubric to grade these assignments.
3: Hold a workshop where all graders discuss the grades they assigned
Go through each graded question with everyone, and discuss the grades that were assigned. Where these differ, discuss the reasons for such differences with the graders and troubleshoot the rubric (and graders’ interpretations) together until everyone is confident they would award the same grades for each question.
4: Revise rubric
It may be necessary to make changes to the rubric where graders provided different grades for certain questions. If the same answer to a given question sees a wide range of grades from graders, this is an indication that the rubric needs to be improved for this question.
Make sure all graders contribute to this process and leave with the new rubric, confident that they can use it objectively.
Although this stage seems like adding another big time commitment to the process, it can save a lot of time further down the line; you will need to spend considerably less time double-checking grading, answering questions about interpreting the rubric and/or results etc., if you tighten up the rubric before grading begins.
During
5: Encourage open discussion among graders as they grade
There will likely be ambiguities that only arise when students complete assignments due to a lack of clarity in certain sections of the rubric, or due to unforeseen – but not technically incorrect – answers arising.
If graders communicate these with one another (over email, in focus groups, or in class planning time), they will help to devise objective solutions much faster. Flagging troublesome answers with one another will draw in feedback from other graders, and a majority vote may prove helpful in deciding how to grade such responses.
After
6: Compare mean grades and variation around these for each grader
There will probably be inherent variation in grades (and in the mean grade and the variation around this) in different sections. However, when section sizes are large, this should be relatively minimal.
If one section does have a significantly higher mean grade, it would be wise to review a handful of papers graded from that section (perhaps even passing these out to graders from other sections). If these different graders grade the papers a little lower, it might be that a standardised adjustment needs to be made to all papers. At the least, some further investigation should take place to make sure these students have not received artificially high grades.
7: Compare random papers from each grader
Even if mean grades are similar between sections, it is good practice to compare papers from different graders to make sure there is close agreement between them. Some variation is to be expected, but if different graders differ greatly in the grades they provide for each assignment, then further investigation is merited.
General Tips
A: Consider randomly assigning papers to graders
Making all assignments anonymous to the grader removes any subjectivity in how they will approach their grading. It can also pay to split assignments up from different sections, or have graders only grade assignments that come from sections they don’t teach.
B: Eliminate all ambiguity/subjectivity from rubrics
Try to provide clear, non-ambiguous scales for all questions, so that every mark can be accounted for. Try to avoid framing subjective scales and instead provide a quantifiable means of grading every answer. For example, try not to ask graders to award 1 - 3 marks based on an answer being poor – excellent; instead ask the graders to award 3 marks for an answer with no mistakes, 2 for one with one or two mistakes, and 1 for an answer with more than two mistakes, for example.
C: Advise graders to grade the same question back-to-back
When grading an assignment, it will save time and enhance objectivity if graders work through the same question sequentially on different students’ assignments, rather than working through each assignment from Question A to Question X. It is easier to memorize the specific grading criteria for a given question on the rubric than to work with different questions, one after another.
