Skip to the lessons

Path

AI trainer essentials

Practise the main work of human-feedback projects. Judge answers, rank them, explain your choice, and rewrite.

9 lessons197 minutes

Prepares you for
It prepares you for human-feedback and model-evaluation work. AI data companies list these roles as trainer, evaluator or annotator — the person who marks up the data. The last four lessons are also the shape of rating work. A written scale, hidden checked items, and a round that keeps a team together.
What you learn
You read an instruction and its rubric — the checklist that says what a good answer needs. You compare two model answers against that rubric, not against length or confidence. You write the short reason for your choice. You handle the cases the brief really does not decide. Then you rate on a five-level scale with written anchors. You write the gold answer yourself, and you run the round that brings a team back onto one scale.
Why this order
Reading the rubric comes before comparing answers, and comparing comes before explaining. Each lesson uses the skill from the one before it. The scale, the gold answer and the round sit last. Each of them is work that other people are handed.
  1. Read the instruction and rubric18 minutes
  2. Compare two model responses22 minutes
  3. Explain and improve a judgment24 minutes
  4. Handle an ambiguous instruction20 minutes
  5. Review the conversation, not one reply22 minutes
  6. Rate on a five-level scale22 minutes
  7. Gold items, and what pauses your queue20 minutes
  8. Write the answer others are rated against24 minutes
  9. Run a check round and reset the scale25 minutes

Curriculum reviewed: