A first guess: just the last word
How much context is enough?
The idea inside
Condition on the previous word(s); n-grams.
After this lesson
You can explain how an n-gram works and why one word of memory isn't enough.
Where it leads
Is this prediction good? By how much?
Inside this lesson
That's the real lesson stage, paused. Claim your pass to operate it.
See how AI actually works, end to end.
This lesson is one stop on the full arc. Unlock all of it, and keep it for life.
What you get
- The 34-lesson main path, a finishable route from a word to agents
- Goal tracks for using AI at work and building AI features
- Boss labs that make you apply a whole act, not just recognize it
- Spaced recall that brings each idea back before you forget
- Course memory: every term defined, with links to where it first appears
- A shareable capability card when you finish the main path
- Lifetime access on every device, every future lesson included
Not videos to watch. You predict, operate the machine, then prove it. That is why it stays.
99 interactive lessons and challenges. No videos, no code.
Free launch pass: lifetime access, no card needed
New here? The first lessons are free to try. Start with lesson 0.1
What this lesson shows
Condition on the previous word(s); n-grams.
The question it opens with
How much context is enough?
The walkthrough, in the lesson's own words
- One clue this time, the previous word. After “drinks ___”, what follows?
- Invented counts that show the idea; a real table is counted from real text.
- One word back is enough to pick a word that fits.
- Last lesson (2.1), the model ignored the sentence and just grabbed common words like
- The ranking shifts, each previous word has its own list.
- One word of memory buys grammar, but it forgets everything older, to stay on topic it must look back at
- Look back two words instead of one and it's a trigram; n words back, an n-gram. More context means more grammar, but the table grows fast, and even n words is a fixed, short window.
- Older chatbots and phone autocomplete write a sentence that reads grammatical but drifts off topic, sometimes looping the same phrase. What's the limit they hit?
- They pick the next word from only the last word or two, like a bigram model. One word of memory buys grammar but forgets the topic, so staying coherent over a paragraph needs looking back at many words at once.
Key takeaway
Better, but brittle: it still wanders.
What you can do after this lesson
You can explain how an n-gram works and why one word of memory isn't enough.
Where it leads: Is this prediction good? By how much?
This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.