Where do query, key and value come from?

All three are made from the same token vector, multiplied by three separate weight matrices that were learned during training. Nobody writes them by hand. The model discovers, over billions of examples, which projections make a useful question, a useful label, and useful content to pass on.

What is the difference between a key and a value?

The key is what a token advertises so others can decide whether to look at it. The value is what they actually receive when they do. Splitting the two lets a token be easy to find for one reason and contribute something else entirely.

Why divide the scores by the square root of the dimension?

Dot products of long vectors grow large, and large scores make softmax collapse onto a single token, which stalls learning. Dividing by the square root of the key dimension keeps scores in a range where the gradients stay useful. It is numerical hygiene, not a modelling idea.

Why does attention need three vectors instead of one?

Because asking, being found and delivering are different jobs. If one vector did all three, a word could only be matched on the same features it hands over. Separate Query, Key and Value matrices let a word be looked up by one property, such as being the subject, and contribute another, such as its meaning.

Are Query, Key and Value the same in every layer?

No. Every layer and every attention head has its own three learned matrices, so the same token gets a different Query, Key and Value at each of them. That is how one layer can track grammar while another tracks meaning or position.

What is the difference between self-attention and cross-attention?

In self-attention, the Queries, Keys and Values all come from the same sequence, so a text attends to itself. In cross-attention, the Queries come from one sequence and the Keys and Values from another, such as a translation model's output looking at its input, or an image model looking at your prompt.

Plain-language explainer

Query, Key and Value, explained

Query, Key and Value are the three vectors attention derives from each token. The Query is what a token is looking for, the Key is what it offers to be matched against, and the Value is the information it passes on when chosen. Matching Queries to Keys produces attention scores; the scores weight the Values that get blended together.

What do Query, Key and Value actually mean in attention?

Attention is a search. Each token asks a question, its Query. Each token also advertises what it contains, its Key, and carries the content it would hand over, its Value. Matching a Query against every Key produces scores, softmax turns the scores into shares, and the token's new representation is the share-weighted blend of the Values. All three roles are computed from the same token vector through learned weight matrices. Nobody writes them; training finds them.

Last reviewed October 23, 2026

Reading is the slow way. Start with a free lesson you can operate right now.

Start free: How attention blends meaning →

Free, no code, no signup.

Ten seconds, no lesson: play with the standalone widget, and embed it in your own page.

Then go deeper: Query, Key, Value Locked

How Query, Key and Value work, step by step

Every token enters an attention layer as one vector of numbers. The layer multiplies that vector by three learned weight matrices to get three new vectors: the Query, the Key and the Value. Then four things happen.

Score: take the current token's Query and compute its dot product with the Key of every token it can see. A high number means a good match.
Scale: divide each score by the square root of the Key's length, so long vectors do not produce huge scores.
Normalize: run softmax over the scores, so they become weights between 0 and 1 that add up to 1.
Blend: multiply each token's Value by its weight and add them up. That sum is the token's new vector, now carrying information from the tokens it matched.

A worked example with tiny numbers

Take the sentence 'the cat sat' and work out attention for 'sat'. Real vectors have dozens or hundreds of numbers. Two are enough to see the mechanism. Say the Query for 'sat' is [1, 2], and the three Keys are the [0, 1], cat [2, 1] and sat [1, 0].

The dot products are 2 for 'the', 4 for 'cat' and 1 for 'sat'. Divide by the square root of 2, the vector length, and softmax turns them into weights of 0.18, 0.73 and 0.09. 'Sat' pays most of its attention to 'cat'. A verb has found its subject.

Now the Values: the [1, 0], cat [0, 4], sat [2, 2]. Weighted and added, they give [0.35, 3.11]. The new vector for 'sat' is mostly made of what 'cat' had to offer. That is the whole trick: the Key decided who gets heard, and the Value decided what they say.

Attention for the word 'sat'. Query = [1, 2]. Each score is divided by 1.41 before softmax. Weights are rounded and add up to 1.

Token	Key	Query · Key	Weight	Value
the	[0, 1]	2	0.18	[1, 0]
cat	[2, 1]	4	0.73	[0, 4]
sat	[1, 0]	1	0.09	[2, 2]

Why it matters in real models

Real models run this lookup in parallel, many times over. GPT-2 small splits each 768-number token vector across 12 attention heads of 64 numbers, and repeats that in 12 layers. Llama 3 8B uses 32 layers with 32 Query heads of 128 numbers each. Every head has its own three matrices, so each one can learn a different kind of match: one follows grammar, another tracks names, another looks for the previous line.

Keys and Values are also what the KV cache stores. To save memory, Llama 3 lets four Query heads share one set of Keys and Values, an idea called grouped-query attention. It has 8 Key-Value heads per layer instead of 32, so the cache is four times smaller and long chats stay affordable.

And the scoring step sets the cost of long context. Every token scores every earlier token, so the work grows with the square of the length. A 1,000-token prompt means about half a million Query-Key scores per head in each layer. At 100,000 tokens it is about five billion. Researchers who study how models work have traced real behavior to these matches. Anthropic's induction heads, for example, use Query-Key matches to find the word that followed an earlier copy of the current word, then copy that word forward. It is a big part of how models continue patterns they saw earlier in the prompt.

What people get wrong

Q, K and V are stored properties of words. They are produced fresh, per position and per layer, by learned matrices multiplying the token's vector.
The model looks words up in a fixed table. Scores are computed from the live context, which is why the same word attends differently in every sentence.
Attention chooses the answer. It only decides which information flows where. Scoring the next word happens later, at the top of the stack.

Where you see it in real products

Attention heatmaps in AI visualizations are these Query-Key scores drawn as color.
The reason 'bank' resolves to river or money per sentence is this per-context matching.
Long-document question answering works because Queries can fish the one relevant passage out of thousands of Keys.

Common questions

Where do query, key and value come from?: All three are made from the same token vector, multiplied by three separate weight matrices that were learned during training. Nobody writes them by hand. The model discovers, over billions of examples, which projections make a useful question, a useful label, and useful content to pass on.
What is the difference between a key and a value?: The key is what a token advertises so others can decide whether to look at it. The value is what they actually receive when they do. Splitting the two lets a token be easy to find for one reason and contribute something else entirely.
Why divide the scores by the square root of the dimension?: Dot products of long vectors grow large, and large scores make softmax collapse onto a single token, which stalls learning. Dividing by the square root of the key dimension keeps scores in a range where the gradients stay useful. It is numerical hygiene, not a modelling idea.
Why does attention need three vectors instead of one?: Because asking, being found and delivering are different jobs. If one vector did all three, a word could only be matched on the same features it hands over. Separate Query, Key and Value matrices let a word be looked up by one property, such as being the subject, and contribute another, such as its meaning.
Are Query, Key and Value the same in every layer?: No. Every layer and every attention head has its own three learned matrices, so the same token gets a different Query, Key and Value at each of them. That is how one layer can track grammar while another tracks meaning or position.
What is the difference between self-attention and cross-attention?: In self-attention, the Queries, Keys and Values all come from the same sequence, so a text attends to itself. In cross-attention, the Queries come from one sequence and the Keys and Values from another, such as a translation model's output looking at its input, or an image model looking at your prompt.

Related explainers

Attention Softmax KV cache Transformers Positional encoding

One idea at a time, in your inbox

New lessons and explainers, written the way these pages are. Now and then, not daily, and never a sales sequence.

We email you a confirmation link first. Unsubscribe in one click, any time. Privacy.

Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.