Path: Working well on data projects
The guideline decides, not your taste
Apply a written guideline you privately disagree with, and raise the disagreement through the right channel.
- Starting level
- Some basics
- Time
- 15 minutes
- Public lesson guide
- EN + हिंदी
You will learn to
Apply a guideline you disagree with, and move the disagreement to where it can change the guideline.
Why this matters
Your labels are compared with everyone else's. A quiet exception makes the data disagree with itself and leaves no record of why.
Three things to remember
- Where the guideline decides the case, the guideline wins over your taste.
- Send your disagreement as a separate note, not as a changed label.
- Quote the guideline line you applied whenever a case was close.
See one example
The guideline treats a short answer as complete when it names all three required facts. A reviewer finds one such answer too blunt.
Try the judgement yourself
Both answers respond to the task above. One of them does the job better. Which, and why?
Make something yourself
Describe a rule you have been asked to follow that you disagreed with. Write how you applied it anyway, and the separate note you would send about the gap.
Read further
The source is optional. The complete teaching and practice are available inside AI2Bharat.
Read the primary or official source: OpenAI InstructGPT labeler guidelines