All learning paths Learn See Try Make Build a check set before you launch What this lesson is about Write fifty real inputs with the answers you expect, and run them every time you change something.
Your small tool explains a district scheme in Hindi. Twenty people get it next week. What do you build before then?
A Write fifty real inputs with the answer you expect, and run them after every change. B Ask five of the twenty to try it and send you whatever goes wrong. C Write fifty real questions and read each reply to see if it looks right.
Check my recall Why this matters A tool that has only been tried on easy questions has not been tried. The rule for judging one is to use examples it was not built from. Fifty of your own real inputs is that.
Build a check set of real inputs with expected answers, and run it after every change.
A tool that has only been tried on easy questions has not been tried. The rule for judging one is to use examples it was not built from. Fifty of your own real inputs is that.
Write fifty real inputs with the answer you expect, and run them after every change. Include the inputs you expect to fail, and record that they failed. Keep every result with its date and version, or a change for the worse hides. Task A small tool answers questions about a district scheme in Hindi. Twenty people get it next week.
Weak approach Writes fifty questions the tool already answers well, runs them once, and puts the clean sheet in the folder. Nothing in the set can fail, so tomorrow's change will not show.
Stronger approach Writes down fifty things people actually ask, each with the reply it ought to give. Keeps eight the tool fails today, and stamps every result sheet with a date and a version.
Why the stronger approach works The eight it fails today are how tomorrow's change becomes visible. A sheet of easy questions passes and tells you nothing.
Try a changed situation A friend is testing a new form for a village committee before printing five hundred copies. Which rule still applies?
Make something yourself Write a check set of fifty real inputs for something you built, each with the answer you expect. Run it twice, and keep both result sheets with their dates.
Lesson 4 of 7 on Small AI products for Bharat next Count what each answer costs Part 2 ends in a work sample you can send
Read the primary or official source: Hugging Face Evaluate · Considerations for model evaluation