How does a vector database find similar information?

Finding nearest neighbours among millions — without checking them all.

Picture it first

Imagine a huge library with a million books but no time to look at each one. Instead, there are signposts: 'Animals → this way', then 'Dogs → that shelf', then 'Puppies → third row'. You hop from sign to sign and quickly reach the right shelf. Very rarely you might miss a great book on a different shelf, but you save hours.

Now the same idea, in simple QA words

A vector database is a special database that answers one question very fast: 'Which stored pieces of text are closest in meaning to this question?' Checking millions of pieces one by one would be too slow, so it uses shortcuts, like signposts, to jump quickly to the right area. These shortcuts are extremely fast but may very rarely miss the best match. It can also filter, for example 'only HR documents' or 'only 2026 files'.

How the story connects to the real thing

In the storyIn the AI system
A library with a million booksA database with millions of text pieces
Signposts: AnimalsDogs → Puppies → Index shortcuts that jump to the right area
Reaching the right shelf in secondsFinding similar text in milliseconds
Rarely missing a book on another shelfOccasionally missing the true best match

Real-world example

An e-commerce company stores 50 million product embeddings. A shopper searches 'shoes for monsoon office commute'. A brute-force comparison would take seconds; the HNSW index returns the top 20 similar products in milliseconds, filtered to 'in stock' and 'size 9'.

Step by step, in plain words

  1. All text pieces are stored as numbers, with labels.
  2. A question arrives (also as numbers).
  3. Filters are applied (e.g. only HR documents).
  4. Shortcuts jump to the closest area.
  5. The top matches are returned with a similarity score.

How a tester tests it

  • Measure recall@K of the ANN index against exact brute-force search on a sample — tuning parameters trade recall for speed.
  • Test metadata filters for leakage: can a 'confidential' document ever appear for a normal user?
  • Load-test query latency at production data size, not on 100 sample documents.
  • Test deletes and updates: deleted vectors must not be returned.

Try it yourself

1) Install Chroma (pip install chromadb), add 10 sentences, and query with a paraphrase. 2) Add a metadata field and try a filtered query.

Interview angle

“What does 'approximate' in ANN mean for testing?”

It means the index can miss the true nearest chunk in exchange for speed. So I measure recall@K against an exact search baseline, and I re-measure after changing index parameters like HNSW's ef_search or after the data grows. A drop there can look like 'the bot got dumber' when nothing in the prompt changed.