Plain-language explainer
Vector databases, explained
What is a vector database, and why does every RAG stack have one?
A vector database stores embeddings, the number-lists that encode meaning, and answers one question fast: which stored items sit closest to this new vector? That is search by meaning instead of by keyword. Embed your documents once, embed the user's question at query time, fetch the nearest chunks, and paste them into the model's context. That fetch step is the retrieval in RAG, and the specialized indexes exist because comparing millions of vectors naively would be far too slow.
Do not just read it. Operate the mechanism yourself in a short interactive lesson.
See it work: RAG: retrieval as a callback to similarity →Free, no code, no signup.
What people get wrong
- It is a database of facts the AI knows. It stores your content as vectors; the model itself stays frozen and separate.
- Vector search understands the documents. It only ranks by geometric closeness. Badly chunked content retrieves badly, no matter the model.
- You always need one. A small corpus can go straight into the context window; the database earns its keep at scale.
Where you see it in real products
- Every 'chat with your documents' product runs one under the hood.
- Tools like Pinecone or the pgvector extension are this exact component.
- Semantic search in note apps, finding the note you described but never titled, is vector search.
Related explainers
Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.