Where the map comes from

What this lesson shows

Words that fill the same blanks get pulled together, so predicting the next word learns the map.

The question it opens with

Lesson 1.1 handed you a finished map of word meanings. But nobody drew it. So where did it come from?

The walkthrough, in the lesson's own words

• Tap two scattered words, then guess: will they end up in the same cluster, or split up?
• Same blanks, different word. Which one fits all three?
• Press play and watch the map build itself, one sentence at a time.
• There it is: the Lesson 1.1 map, learned from scratch.
• This map is scrambled. Nobody has placed the words yet. Make a call before the model does.
• Here are three sentences the word dog fits. Which other word slips into all of them?
• Each sentence pulls the words that share it closer together. Watch the scattered words sort themselves into two clusters.
• Nobody placed these. Predicting the next word pulled words that keep the same company together. The map was learned, not given.
• These two get pulled into the same cluster: they keep the same company in real sentences. Nobody placed them there.
• These two end up in different clusters: they almost never share a sentence, so nothing pulls them together.
• Exactly, cat slips into every sentence dog does; car and boat don't.
• Not quite, the answer is cat. It slips into every blank dog does.
• Words that keep the same company mean similar things, and the model can measure that with no dictionary, just by reading. Sharing the same blanks is the only clue it needs.
• A sped-up cartoon: real training takes billions of reads, and nobody knows the final map in advance.
• Each read is one downhill step from 2.4: the miss nudges the words' coordinates instead of a single weight.
• Nobody placed these. Predicting the next word, over and over, pulled words that share contexts together, and the Lesson 1.1 map fell out. Learned, never given.
• Search and recommendation tools surface results that match your meaning, not just your exact words, so a query for couch finds sofa. Where does that sense of similarity come from?
• From embeddings: each word or item gets a position learned by predicting context, so things that keep the same company land near each other. Nobody hand-labels the synonyms, the closeness falls out of the prediction task.

Key takeaway

You watched the embedding map build itself from raw text.

What you can do after this lesson

You can explain how a model learns word meanings: words used in similar contexts end up with similar vectors (the distributional hypothesis).

Where it leads: Now it can predict and represent meaning, but it still just autocompletes text. How does it become a helpful assistant?

This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.