The first time I shipped a feature backed by a vector database, I spent an afternoon confused about why my results made no sense. I had treated it like a keyword index. It is not one. A vector database stores meaning, or at least a numerical approximation of meaning, and it answers a different kind of question than the databases I had used for years. This post is the explanation I wish someone had handed me before I started.
What an embedding actually is
An embedding is a list of floating point numbers that represents a piece of content. You take some text, an image, or an audio clip, run it through a model, and out comes a fixed length array. A typical text embedding might have 384, 768, or 1536 dimensions. Each number on its own means nothing you can interpret. Together they place the content at a specific point in a high dimensional space.
The useful property is that similar content lands in nearby positions. The sentence “my laptop will not turn on” and the sentence “my computer is dead” produce vectors that sit close together, even though they share almost no words. A keyword search would miss that connection entirely. An embedding captures it because the model that produced the vectors learned, from enormous amounts of text, that those phrases mean roughly the same thing.
This is the whole trick. We convert the fuzzy human notion of “these two things are about the same topic” into a geometry problem. Once it is geometry, a computer can solve it fast.
How similarity is measured
Closeness in this space is usually measured with cosine similarity or, equivalently for normalized vectors, dot product. Cosine similarity looks at the angle between two vectors and ignores their length. Two vectors pointing in the same direction score near 1.0, vectors at right angles score near 0, and opposite vectors score near -1.0. Some systems use Euclidean distance instead, which measures straight line distance. The choice depends on how the embedding model was trained, and getting it wrong quietly degrades your results.
Here is the part that surprises people coming from relational databases. There is no exact match. Every query returns a ranked list of approximately relevant items with a score attached. You decide where to cut the list off. That mental shift, from “the row that matches” to “the rows that are most similar,” is the single biggest adjustment.
Generating embeddings in practice
You rarely train an embedding model yourself. You call one. Here is a small example that turns a few sentences into vectors and measures how close they are.
from sentence_transformers import SentenceTransformer
import numpy as np
model = SentenceTransformer("all-MiniLM-L6-v2")
sentences = [
"my laptop will not turn on",
"my computer is dead",
"what time does the store close",
]
vectors = model.encode(sentences, normalize_embeddings=True)
def cosine(a, b):
return float(np.dot(a, b))
print("laptop vs computer:", cosine(vectors[0], vectors[1]))
print("laptop vs store:", cosine(vectors[0], vectors[2]))
Run that and the first pair scores high while the second pair scores low. The model never saw these exact sentences during training. It learned the relationships from context, and that generalization is what makes the whole approach work.
Why you need a specialized database
You could store vectors in a normal column and compare your query against every row. That is called a brute force or exact search, and it works fine up to maybe a hundred thousand vectors. Past that it falls apart, because comparing one query against ten million vectors on every request is too slow for anything interactive.
Vector databases solve this with approximate nearest neighbor indexes. The most common is HNSW, which stands for Hierarchical Navigable Small World. It builds a layered graph where each vector links to its neighbors, and a query walks the graph instead of scanning everything. You trade a tiny bit of accuracy for an enormous speed gain. Other index types like IVF and product quantization make different tradeoffs between memory, speed, and recall.
- Recall is the fraction of the true nearest neighbors your index actually returns. You tune it up or down depending on how much latency you can spend.
- Latency is how long a query takes. HNSW typically answers in single digit milliseconds even over millions of vectors.
- Memory matters because HNSW graphs are large. Quantization shrinks vectors at some cost to precision.
Metadata filtering is where it gets real
Pure similarity search is rarely enough on its own. In real applications you want “find documents similar to this query, but only from this user’s workspace, written in the last 90 days.” That means combining vector search with traditional filters on metadata. How well a database handles this combination, called filtered or hybrid search, is one of the things I care about most when I evaluate options, and it is a major theme in my notes on choosing a vector database.
Done badly, filtering forces the engine to either scan everything or return too few results because the filter eliminated most of the candidates the index found. Done well, the filter is applied during the graph traversal so you still get fast, accurate results inside the subset you care about.
A mental model that holds up
I think of a vector database as a search engine for meaning rather than words. A keyword index answers “which documents contain these tokens.” A vector index answers “which documents are about the same thing as this query.” Those are complementary, not competing, which is why the strongest systems I have built use both at once and merge the rankings.
The embeddings carry the semantics. The index makes search fast at scale. The metadata filters keep results relevant to the actual user. Once those three pieces clicked for me, the rest of the field stopped feeling like magic and started feeling like engineering.
Where this leads
The reason vector databases exploded in popularity is that they are the storage layer underneath retrieval augmented generation. When you want a language model to answer questions about your own documents, you embed those documents, store the vectors, and retrieve the relevant ones at query time. I walk through that entire pattern in building RAG systems with vector databases, and it builds directly on everything here. If you want the broader picture of how this fits into shipping real systems, my write up on practical AI engineering covers the surrounding decisions.
Start by generating a few embeddings and printing the similarity scores yourself. Seeing related sentences score high and unrelated ones score low does more to build intuition than any diagram. Everything else is detail on top of that one idea.
Leave a Reply