Path: How post-training works
Evaluation closes the loop
Use held-out tasks and explicit criteria to check whether a change helped without creating new failures.
- Starting level
- Some basics
- Time
- 20 minutes
- Public lesson guide
- EN + हिंदी
You will learn to
Design a small held-out evaluation that checks improvement and regression.
Why this matters
A change can improve one metric while making accuracy, safety, or a subgroup worse.
Three things to remember
- Keep evaluation tasks separate from development examples.
- Measure the intended benefit and likely regressions.
- Inspect important languages, domains, and user groups.
See one example
A team changes a model to make answers shorter.
Check one prompt used during development.
Compare a held-out set before and after for length, correctness, safety, and important subgroups.
A multidimensional held-out test can reveal both the intended gain and hidden regressions.
Make something yourself
Create a five-task evaluation plan with one success metric, one safety check, and one subgroup check.
Read further
The source is optional. The complete teaching and practice are available inside AI2Bharat.
Read the primary or official source: OpenAI HealthBench