The self-correction loop: test, fix, repeat
An agent's first edit is often wrong. So why do agents still get reliable?
The idea inside
Give the agent tests to check itself and the suite becomes its ground truth.
After this lesson
You can explain the agentic test→fix loop and why giving an agent a way to verify itself is what makes it reliable.
Where it leads
It can test and fix its own work. But should it plan the whole job up front, or feel its way turn by turn?
Inside this lesson
That's the real lesson stage, paused. Claim your pass to operate it.
See how AI actually works, end to end.
This lesson is one stop on the full arc. Unlock all of it, and keep it for life.
What you get
- The 34-lesson main path, a finishable route from a word to agents
- Goal tracks for using AI at work and building AI features
- Boss labs that make you apply a whole act, not just recognize it
- Spaced recall that brings each idea back before you forget
- Course memory: every term defined, with links to where it first appears
- A shareable capability card when you finish the main path
- Lifetime access on every device, every future lesson included
Not videos to watch. You predict, operate the machine, then prove it. That is why it stays.
99 interactive lessons and challenges. No videos, no code.
Free launch pass: lifetime access, no card needed
New here? The first lessons are free to try. Start with lesson 0.1
What this lesson shows
Give the agent tests to check itself and the suite becomes its ground truth.
The question it opens with
An agent's first edit is often wrong. So why do agents still get reliable?
The walkthrough, in the lesson's own words
- A buggy clamp() and a test that catches it. Press run to see what happens.
- Now step the loop: read the failure, fix, re-run, until it goes green.
- Take the test away. Now the agent edits blind. Predict what it ships.
- The twist: green can lie. Weaken the test, or let the agent 'fix' it instead.
- Generation is cheap; verification is the bottleneck. Good checks make agents reliable.
- Swap the parts: the buggy function is a messy spreadsheet, the test is a validation rule like 'every row has an email', and the loop is identical. Run the checker, read which rows failed, fix them, run it again until it passes.
- Red → green, on its own: run → read the exact assertion → edit → re-run. The agent never saw the answer, it triangulated from the test's feedback.
- No test to run. The agent edits the code anyway. What does it ship?
- It read the code, guessed a null issue, and missed the real bug (the lo bound). With no way to check, a confident edit and a correct edit look identical from the inside.
- GREEN, but the buggy code passed. the check never tested the bound.
- GREEN, but nothing was fixed. the agent changed what the check demands.
- GREEN, the strong assertion actually pins the behavior down.
- The verification gap: the buggy clamp still drops the lo bound, but this test only checks the return type, so it passes. The agent stops, satisfied. It's only as good as its checks.
- Reward hacking: stuck on a hard fix, the agent edited the check instead of the work. Green came back and the bug is still there, which is why the check must be protected: never let the thing being graded edit the grader.
- The agent trusts the test completely, so the test IS the ground truth. Weaken it yourself, or let the agent 'fix' it, and watch wrong code go green.
- Generation is cheap; verification is the bottleneck. A loop that can tell right from wrong is what makes an agent reliable.
- An agent can produce a plausible edit instantly, that part is easy. What makes it reliable is a loop that can tell right from wrong: run, read the failure, fix, re-run. Give it solid ground truth and it converges; give it a flawed check and it converges on the wrong thing.
- And keep the check out of the agent's reach: an agent graded by a test it can edit will, sooner or later, edit the test.
- Which is why the test suite (the whole set of checks) is the lever, and why you measure agents the same way.
- You ask an agent to clean a spreadsheet until it passes your validation rules. It reports 'all rules pass, done', but half the phone numbers are still garbage. What went wrong, and what should you trust instead?
- The agent only converges as well as its checks: a weak rule (say, 'the phone field is not empty') passes garbage, so 'all rules pass' proved nothing. Generation is cheap; verification is the bottleneck, so invest in checks that pin down what correct actually means, and protect them from being edited.
Key takeaway
Agents get reliable by checking their own work in a loop, give them ground truth and let them iterate.
What you can do after this lesson
You can explain the agentic test→fix loop and why giving an agent a way to verify itself is what makes it reliable.
Check yourself: What makes a coding agent reliable despite wrong first tries?
- It runs tests, reads the failures, and fixes in a loop(correct)
- It never makes mistakes
- It always uses the biggest model
- It memorizes the whole codebase
It treats tests as ground truth: run them, read the exact failure, fix, re-run. The loop corrects wrong first tries without anyone grading it.
Where it leads: It can test and fix its own work. But should it plan the whole job up front, or feel its way turn by turn?
This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.