All learning paths Learn See Try Make Run a check round on shared work What this lesson is about Give the same three pieces of work to everyone, compare the marks, and find the badly written task.
Six people in your circle marked the same answers. On the second one, the marks ran from one to five. What now?
A Ask the six markers to talk it through until they land on one mark. B Give everyone the same three pieces of work, and look for the badly written task first. C Read the second answer again yourself and decide which of the six marks was right.
Check my recall Why this matters Reviewers who differ are often reading a task that allows both answers. Research on disagreement warns that averaging the marks throws that information away. The spread is the part worth reporting.
Run a check round on shared work, report the spread, and find the badly written task behind it.
Reviewers who differ are often reading a task that allows both answers. Research on disagreement warns that averaging the marks throws that information away. The spread is the part worth reporting.
Give everyone the same three pieces of work, and look for the badly written task first. Report the spread of marks, not the average, because the average hides the split. Name nobody in the result, or the second round will not be honest. Task Six people in a circle marked the same three answers out of five. On the second answer the marks ran from one to five.
Weak approach Works out the average of the six marks on each answer and writes them in a row. A note says who marked hardest, and the one-to-five split on the second answer never appears.
Stronger approach Shows the range the six marks covered on each answer, one to five on the second, with nobody named. Then rewrites that task so both readings cannot stand.
Why the stronger approach works The spread is what the round was for. Naming a person turns the next round into a test, and the quiet members stop taking part.
Try a changed situation Your family is judging four cooks at a village fair and wants one score for each. Which rule still applies?
Make something yourself Give three pieces of the same work to three people you know. Write the spread of their marks, name the task you found was badly written, and show how you rewrote it.
Lesson 5 of 7 on Teaching and reviewing next Teach one lesson to five people Part 2 ends in a work sample you can send
Read the primary or official source: Davani et al., Dealing with Disagreements (arXiv)