All learning paths Learn See Try Make Write a task from your own work What this lesson is about Turn one real problem from your working week into a task with one answer another person can check.
A project asks you for three test questions from the medical shop you have run for nine years. Where do you start?
A Set a task only somebody inside your trade could set. B Pick three questions a pharmacy course sets in its exam, because those already have marked answers. C Write three questions about medicines that anybody careful could look up and answer.
Check my recall Why this matters A subject expert is paid to set tasks only somebody inside the trade could set. A benchmark — a fixed set of tasks a system is judged on — is only as good as its tasks. A task whose answer two experts would argue about teaches nobody anything.
Write one task from your own trade that has an answer another person can check.
A subject expert is paid to set tasks only somebody inside the trade could set. A benchmark — a fixed set of tasks a system is judged on — is only as good as its tasks. A task whose answer two experts would argue about teaches nobody anything.
Set a task only somebody inside your trade could set. Name the facts that settle the answer: the amount, the date, the rule. If two experts could both be right, the task is not finished. Task A wholesale dealer is asked to write one task about a returned consignment.
Weak approach Writes: a shop has returned goods and is unhappy. What should the distributor do to keep the customer? Adds that the answer should be polite and practical.
Stronger approach Writes: a retailer returns forty cartons on day sixty-two of a sixty-day credit note. Names the return window, the restocking charge and who pays the freight. The answer is one line: refuse the credit, take the stock back at the restocking charge.
Why the stronger approach works Both are about a return. Only the second names the facts that settle it, so two dealers would mark it the same way.
Try a changed situation A school clerk asks for a task about a parent who wants a child moved to another section mid-term.
Make something yourself Write one task from a real week in your trade. Give the facts that settle it — the amount, the date, the rule — and write its answer in two lines.
Lesson 1 of 6 on Your own trade, as AI work next Adapt a scoring guide to your trade Part 1 ends in a work sample you can send
Read the primary or official source: Mialon et al., GAIA: a benchmark for General AI Assistants (arXiv)