Do I actually need a vector database?

Not to start. Up to a few tens of thousands of chunks, a plain array in memory or a Postgres extension is fine and far simpler to run. A dedicated store earns its place once you need scale, metadata filtering and live updates together.

How is vector search different from a normal database search?

A keyword search finds documents containing your words. A vector search finds documents whose meaning sits near your question, so how do I cancel can match a page about ending a subscription. Strong systems run both and merge the results.

What is an embedding index?

A structure that finds near neighbours without comparing your query against every stored vector. It trades exactness for speed, returning almost always the right neighbours in a fraction of the time. That approximation is why two runs can return slightly different results.

Plain-language explainer

Vector databases, explained

What is a vector database, and why does every RAG stack have one?

A vector database stores embeddings, the number-lists that encode meaning, and answers one question fast: which stored items sit closest to this new vector? That is search by meaning instead of by keyword. Embed your documents once, embed the user's question at query time, fetch the nearest chunks, and paste them into the model's context. That fetch step is the retrieval in RAG, and the specialized indexes exist because comparing millions of vectors naively would be far too slow.

Last reviewed July 25, 2026

Do not just read it. Operate the mechanism yourself in a short interactive lesson.

See it work: RAG: retrieval as a callback to similarity →

Free, no code, no signup.

What people get wrong

It is a database of facts the AI knows. It stores your content as vectors; the model itself stays frozen and separate.
Vector search understands the documents. It only ranks by geometric closeness. Badly chunked content retrieves badly, no matter the model.
You always need one. A small corpus can go straight into the context window; the database earns its keep at scale.

Where you see it in real products

Every 'chat with your documents' product runs one under the hood.
Tools like Pinecone or the pgvector extension are this exact component.
Semantic search in note apps, finding the note you described but never titled, is vector search.

Common questions

Do I actually need a vector database?: Not to start. Up to a few tens of thousands of chunks, a plain array in memory or a Postgres extension is fine and far simpler to run. A dedicated store earns its place once you need scale, metadata filtering and live updates together.
How is vector search different from a normal database search?: A keyword search finds documents containing your words. A vector search finds documents whose meaning sits near your question, so how do I cancel can match a page about ending a subscription. Strong systems run both and merge the results.
What is an embedding index?: A structure that finds near neighbours without comparing your query against every stored vector. It trades exactness for speed, returning almost always the right neighbours in a fraction of the time. That approximation is why two runs can return slightly different results.

Related explainers

Embeddings RAG Word vector analogies Transformers RAG vs fine-tuning

One idea at a time, in your inbox

New lessons and explainers, written the way these pages are. Now and then, not daily, and never a sales sequence.

We email you a confirmation link first. Unsubscribe in one click, any time. Privacy.

Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.