Embeddings: meaning as coordinates
More dimensions = more nuance; an embedding is a learned coordinate list.
1The model stores no words, only numbers. Here's what a word looks like inside it.
man
0.28 0.26
woman
0.62 0.26
king
0.28 0.72
the ideaA word's position on the map is just its list of coordinates, that list is its embedding.
Real models use far more than 2
Here each word gets 2 numbers, so it fits on a flat map. Real models give each one 768 numbers in GPT-2 small , room for far more shades of meaning than a flat map.
→ continue← backR replay
How does the machine actually measure 'nearness'?
1.3 Measuring meaning with one numberBuilds on1.1Turning words into numbers
Common questions
What is "Embeddings: meaning as coordinates" about?
More dimensions = more nuance; an embedding is a learned coordinate list.
What problem does it solve?
A 2-D map can't cleanly separate king / queen / man / woman.
What will I be able to do after this lesson?
You can explain what a word embedding is and read an analogy as vector math.
What comes next?
How does the machine actually measure 'nearness'?