Structured outputs: guaranteed JSON

What this lesson shows

Constrained decoding allows only valid next tokens, so the JSON always parses.

The question it opens with

Your code needs clean JSON, but the model sometimes adds prose or breaks the schema.

The walkthrough, in the lesson's own words

• Advance the model along the rail: pick the next token. Illegal tokens are blocked.
• Flip the constraint off and watch the same rail derail into invalid JSON.
• Parseable by construction, but still check the values make sense.
• Valid shape ≠ correct value. This matches the schema perfectly, but 450°C isn't real. Structure guaranteed; content isn't.
• After parsing, still validate the values: range checks, enums, business rules, or a second model pass. Shape and truth are two different problems, which is why 7.12's ship gate runs both checks: parse checks catch a broken shape, and only evals catch a wrong value.
• Illegal tokens get probability zero before sampling, so the output can only ever be valid JSON. It parses every single time.
• Unmasked, any token can win the draw, including the ones that produce prose. The same rail derails and the receiving software chokes on it.
• The softmax from 3.3 turns scores into next-token probabilities. Constrained decoding zeroes out every token that would break the schema before that draw, then renormalizes the survivors back to sum 1, so only legal tokens can be sampled.
• At each step the decoder is masked to only schema-valid next tokens, so the JSON always parses. But a valid shape can still hold a wrong value. Same idea you met in 7.1: get reliable structure from the model, then verify the substance yourself.
• The constraint is free at decode time, masking illegal tokens costs nothing. But field order matters: if the schema demands the answer before a reasoning field, the model must commit before it has thought, and accuracy drops. The fix is schema design, put the reasoning field first.
• You wire an LLM into software that needs clean JSON, but sometimes it adds a sentence before the JSON and the parser breaks. What makes this dependable?
• Structured outputs: constrain decoding to a schema so the model can only emit valid JSON, instead of hoping a prompt keeps it tidy. Any AI feature that feeds another system (forms, function calls, pipelines) leans on this to stay reliable.
• "string" means any text, "number" any number; the | means sky must be one of these three words (an "enum" = a fixed list of allowed values).
• Every token on the rail was schema-legal, so the string parses by construction.
• The receiving software reads it without error, no safety net needed. Constrained decoding forced it to conform.
• Software can't act on a nicely written paragraph; it needs answers machines can reliably read, the same shape every time. The schema works like a bouncer with a rulebook (called a grammar): at every position it only lets in tokens that keep the output valid. Try a greyed token first: see why the bouncer blocks it. Then emit the legal one.

Key takeaway

You can make outputs machine-parseable by construction, while still checking the values make sense.

What you can do after this lesson

You can explain constrained decoding / structured outputs, and why valid shape still needs value validation.

Where it leads: You can build and ship it, but how do you read the scores that claim one model beats another?

This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.