Query, Key, Value: run one attention lookup by hand

1. 1One word asks a question. Tap the earlier word whose Key you think answers it.
2. 2See the match scored: a dot product between the Query and each Key, the same operation used to compare embeddings.
3. 3Watch the winning word's Value flow into the asking word, so it now knows what its subject is.

Attention is a search, and this widget lets you run one lookup by hand. Every word starts as one embedding. The model multiplies that embedding by three learned grids of numbers to make three views of the same word: a Query, which asks a question, a Key, which advertises what the word contains, and a Value, which is the content it hands over when it is chosen.

When drinks holds up its Query, every earlier word offers its Key. The score for each pair is a dot product, multiply the matching slots and add them up, exactly the similarity trick from the embeddings lessons. The highest score wins the biggest share, and that word's Value flows into drinks. In the sentence on the stage, cat wins, so drinks now carries the knowledge that a cat is doing the drinking.

The numbers on the bars and the numbers in the expanded view are the same numbers. Nothing hidden happens between them. That is the point of running it by hand: the whole of attention at one position is a few multiplications and an addition.

This is the machinery that disambiguates words by context. In the sentence I left the bank to fish by the river, bank sends a Query, river answers with a Key that scores high, and the Value of river shifts bank toward the water sense. Multi-head attention runs many of these lookups in parallel, and a transformer stacks dozens of layers of them. Once the single lookup is clear, the rest of the architecture is repetition.

Paste this into any HTML page, course platform, wiki or slide tool that accepts an iframe. It runs with no account, sets no cookies of its own, and links back here.

Attribution is built in. If you write about it, a link to this page is all we ask.

What is the difference between a Key and a Value?

The Key is what a word advertises so others can decide whether to attend to it. The Value is what the word actually delivers once chosen. Scoring uses Keys; the blend that updates the asking word uses Values. Both come from the same embedding through different learned matrices.

Is the dot product really all there is to the score?

For one head at one position, yes: multiply matching slots and add. Real models scale that sum by the square root of the dimension and run softmax over all the scores so they become shares that add to one. The widget shows the raw scores so you can check the arithmetic yourself.

Can I put this widget in a lecture or a course page?

Yes. The embed snippet on this page drops the stage into any HTML page or slide tool that accepts an iframe. It runs with no account and links back to the full lesson.