All learning paths Learn See Try Make Write a robot task somebody else can score What this lesson is about Write one robotics task the way an expert does. The setting, a success a rater can see, the failures scored apart, the line never crossed.
A Pune robotics firm hands you a blank sheet. Write us one job for the kitchen arm, they say. What goes on the sheet?
A Write one clear line, put the tumbler on the tray, and let the rater fill in the rest by eye. B Write the job and a long list of all that could go wrong, so the rater sees every failure. C Write four things: the setting, a success a rater can see, failures scored apart, one thing never allowed.
Check my recall Why this matters A large published robot benchmark — a shared set of tasks many teams are scored on — holds a thousand activities. Lay people wrote them. Each one names the objects, the starting state, and a goal state a checker can see. The cloth is folded. The food is on the plate. Nothing in it says do it well. The listings under the robotics label ask for this: the task written, not the trace marked.
Write a robot task the way a subject expert does, so a rater who never met you can score it.
A large published robot benchmark — a shared set of tasks many teams are scored on — holds a thousand activities. Lay people wrote them. Each one names the objects, the starting state, and a goal state a checker can see. The cloth is folded. The food is on the plate. Nothing in it says do it well. The listings under the robotics label ask for this: the task written, not the trace marked.
Write four things: the setting, a success a rater can see, failures scored apart, one thing never allowed. Success is a state somebody can see at the end, never a word like well, safely or properly. A failure scored apart is one the task can survive. The thing that must never happen ends the run at once. Task A firm asks for one task to score a warehouse arm that moves cartons from a pallet to a belt.
Weak approach Writes: move the cartons from the pallet to the belt carefully and without damage. Leaves the rater to decide what careful and damage mean.
Stronger approach Writes: twelve cartons, pallet at one metre, belt running. Success: every carton on the belt, label up, none dropped, in four minutes. Scored apart: a carton set label down, a carton left on the pallet. Never: the arm moves while a person is inside the yellow line.
Why the stronger approach works The weak task is scored differently by every rater. The strong one is scored the same way by a stranger.
Try a changed situation A hospital trolley robot's task sheet says success is reaching the ward safely. Two raters scored the same run differently last week. What do you change?
Make something yourself Write one task for a robot you have seen at work. Give the setting, the visible success, the failures scored apart, and the thing that must never happen.
Lesson 9 of 10 on Multimodal & physical AI next Adapt a rubric to a robotics domain Part 3 ends in a work sample you can send
Read the primary or official source: Li et al., BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark (arXiv)