Where the map comes from
Words that fill the same blanks get pulled together, so predicting the next word learns the map.
1Remember the map from Lesson 1.1? Nobody drew it. Let's find out who did.
kittencatpuppydoghorsebicyclecarboatbustruck
the catchNo dictionary, no labels, only raw text. So how does the model learn that cat belongs next to dog?
the ideaThe secret is a game you already know from this act: predict the next word.
→ continue← backR replay
Now it can predict and represent meaning, but it still just autocompletes text. How does it become a helpful assistant?
2.6 From predictor to assistantCommon questions
What is "Where the map comes from" about?
Words that fill the same blanks get pulled together, so predicting the next word learns the map.
What problem does it solve?
Lesson 1.1 handed you a finished map of word meanings. But nobody drew it. So where did it come from?
What will I be able to do after this lesson?
You can explain how a model learns word meanings: words used in similar contexts end up with similar vectors (the distributional hypothesis).
What comes next?
Now it can predict and represent meaning, but it still just autocompletes text. How does it become a helpful assistant?