Embeddings: meaning as coordinates
More dimensions = more nuance; an embedding is a learned coordinate list.
The same arrow that took man to woman. Drop its tail on king.
the idea Each word is just a list of coordinates, its position on this map, that list is its embedding.
See the coordinates
Invented numbers that show the idea, not measurements from a real model.
Here each word gets 2 numbers, so it fits on a flat map. Real models give each one 768 numbers in GPT-2 small , room for far more shades of meaning than a flat map.
How does the machine actually measure 'nearness'?
1.3 Measuring meaning with one numberBuilds on1.1Turning words into numbers
Common questions
What is "Embeddings: meaning as coordinates" about?
What question does this lesson answer?
What will I be able to do after this lesson?
What comes next?
What this lesson shows
More dimensions = more nuance; an embedding is a learned coordinate list.
The question it opens with
Words need more than a flat map: how do you write 'king relates to queen as man relates to woman' as math?
The walkthrough, in the lesson's own words
- Same move, new word. Now name where it lands.
- Move the copied arrow onto king, same direction, same length.
- In map language: start at king, walk backwards along the man arrow (that is the minus), then forwards along the woman arrow (the plus).
- That single arrow IS "− man + woman", so what is king + that move?
- Exactly, the +woman move turns the king into the queen.
- Each word is just a list of coordinates, its position on this map, that list is its
- Here each word gets 2 numbers, so it fits on a flat map. Real models give each one
- “man → woman” is one step along the gender axis.
- Same axis, same shift, so it turns king into queen too.
- A word is a position in this space. That position is its embedding.
- Ask a chatbot "Paris is to France what ___ is to Japan" and it answers Tokyo. What move did it just make?
- The same arrow-copying you just did. The France → Paris arrow means 'go from a country to its capital'; walk that same arrow from Japan and you land on Tokyo. Word relationships are directions on the map.
- In a multilingual model, yes. It learns one shared map, so 'dog' and 'perro' land near each other, and the same geometry holds: rey − hombre + mujer ≈ reina. Meaning, not spelling, sets the position, which is why related words can sit close even across languages.
- A 2-D map of man, woman, king and queen as labelled points.
- The copied arrow now starts at king and its tip reaches queen.
- Drag the dashed arrow so its tail sits on king.
Key takeaway
king − man + woman ≈ queen, seen as vector arithmetic.
What you can do after this lesson
You can explain what a word embedding is and read an analogy as vector math.
Check yourself: Word embeddings can do 'king − man + woman ≈ queen'. What does that reveal?
- Directions in the space carry meaning, like gender or royalty(correct)
- The model memorized that exact equation
- The letters were added and subtracted
- It's a coincidence of word length
Meaning becomes geometry: consistent directions encode features like gender, so moving along the 'gender' direction turns king into queen. The model never memorized the equation.
Prefer reading first? This lesson has a companion explainer: EmbeddingsWord vector analogies
Where it leads: How does the machine actually measure 'nearness'?
This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.