Learn
See
Try
Make

Test and review generated code

What this lesson is about

Use the requirements and your own tests to find the first failure. Wrong result, odd input, or a safety problem.

A function that should find the largest number starts its count at zero, and the style is neat. What do you test?

Why this matters

Generated code can read well and still miss odd inputs, state changes, or safety limits.

Turn a code requirement into a test that shows the first real failure.

Generated code can read well and still miss odd inputs, state changes, or safety limits.

Test what was required, not how the code was written.
Include the edges, empty inputs, and the ways it can fail.
Fix the real cause before you rewrite everything.

Task

Review a function that finds the largest number and starts at zero.

Weak approach Writes four tests and runs them: three positive lists and a mixed one, all passing. Reads the code closely and calls it tidy. No test has an all-negative list or an empty one, so the zero it starts from is never reached.

Stronger approach Run an all-negative case. It returns zero instead of the largest input. Start from the first item, or handle an empty input.

Why the stronger approach works Both reviewers tested, and one of them tested four times. Tests that pass are the ones the code was already written for.

Try a changed situation You are checking a shop's billing sheet formula, not a program. What do you try on it?

Make something yourself Write one test that fails, a note on the real cause, and the smallest repair.

Lesson 1 of 8 on Coding & reasoning evaluationnextFind the first invalid reasoning stepPart 1 ends in a work sample you can send

Read the primary or official source: Chen et al., Evaluating Large Language Models Trained on Code (arXiv)