Vector Search & Embeddings for Enterprise

What an embedding is

An embedding model maps text to a fixed-length vector — commonly 768, 1024 or 1536 dimensions — positioned so that semantically similar text lands nearby.

“How do I reset a charge point?” and “procedure for restarting a charging station” share almost no words and end up close together. That is the entire value proposition, and it is why vector search finds documents keyword search misses.

It is also why it misses documents keyword search finds. Exact identifiers — 4002, IEC_62196_T2_COMBO, a part number — carry little semantic signal. The embedding of a passage containing an error code is dominated by the surrounding prose. This is the structural reason hybrid retrieval exists.

Distance metrics

Metric	Measures	Use when
Cosine	Angle between vectors, magnitude ignored	The default for text; almost always correct
Dot product	Angle and magnitude	Vectors are normalised — then identical to cosine but faster
Euclidean (L2)	Straight-line distance	Rarely right for text embeddings

Match the metric to what the model was trained with. Most text embedding models are trained for cosine similarity. Using L2 on vectors trained for cosine produces results that look plausible and rank subtly wrong — which is worse than being obviously broken, because nobody investigates.

Approximate nearest neighbour

Exact nearest-neighbour search compares the query against every vector. At ten million chunks that is ten million distance computations per query, which does not work.

Every production vector store therefore uses an approximate index. Approximate means it can miss the true nearest neighbour, and that is a deliberate, tunable trade.

Index	How it works	Character
HNSW	Navigable small-world graph, searched hierarchically	Excellent recall and latency; high memory; the common default
IVF	Partition into clusters, search the nearest few	Lower memory; recall depends on how many clusters you probe
Flat	Brute force, exact	Correct by definition; fine below ~100k vectors
PQ / quantised	Compress vectors	Large memory savings, some accuracy loss; combine with the above

The knobs that matter

Parameter	Raising it	Cost
HNSW `M`	More graph edges, better recall	Memory and build time
HNSW `efConstruction`	Better index quality	Build time only
HNSW `efSearch`	Better recall per query	Query latency — tune this at runtime
IVF `nprobe`	More clusters searched, better recall	Query latency

efSearch and nprobe are runtime parameters, which means recall is something you can dial per query. Fast autocomplete can run low; a compliance search can run high. Most teams never touch them and accept whatever the library defaults to.

Measure recall against a flat index. Build an exact index over a sample, run your golden queries against both, and compare. Without that number you do not know whether your retrieval layer is missing 1% of correct results or 30%.

Metadata filtering: where correctness lives

Enterprise search is almost never unrestricted. Results must be scoped by tenant, department, document type, date, and above all by who is asking.

Approach	Mechanism	Problem
Post-filter	Retrieve top-k, then discard	Ask for 10, all belong to another tenant, return nothing — and you already read their data
Pre-filter	Restrict the candidate set, then search	Correct, but can degrade to brute force on a narrow filter
Filtered ANN	Filter evaluated inside graph traversal	What good engines do; what you should require

Post-filtering is a security defect, not a performance choice. If unauthorised documents are retrieved and then dropped, that content was in your process memory, in your logs, and one caching layer or reordered code path away from a live leak. Access control belongs inside the search query. Evaluate any vector store on whether it supports filtered ANN properly before anything else.

Freshness

Vector indexes are not free to update, and how a store handles change varies enormously.

Insert is usually cheap.
Delete is often a tombstone — the vector stays in the graph until compaction, so deleted content can still be retrieved and must be filtered out.
Update is generally delete plus insert, inheriting the tombstone behaviour.
Re-embedding after a model change means rebuilding everything — vectors from different models are not comparable.

That last point deserves planning. Changing embedding model is not a config change; it is a full re-index and a re-run of your evaluation set. Version the embedding model alongside the index and never mix generations in one collection.

For a knowledge base backing customer-facing answers, event-driven re-indexing on source change beats a nightly batch. A withdrawn policy that stays answerable for a day is a compliance problem, not a latency one.

Choosing a store

Option	Fits	Watch
Azure AI Search	Azure-native stacks; hybrid search and semantic ranking built in	Cost at scale; tier limits
pgvector	Already on Postgres; modest corpora; transactional consistency	Index build time; performance beyond a few million vectors
Qdrant / Weaviate / Milvus	Large corpora, self-hosted, filtering-heavy	You own the operational burden
Pinecone	Managed, low operational effort	Cost; data residency

For a .NET and Azure estate, Azure AI Search is usually the pragmatic answer — hybrid retrieval and reranking are built in rather than assembled, and it sits inside the same identity and network boundary as the rest of the platform. That last point matters more than benchmark numbers: a vector store outside your compliance boundary is a data-residency conversation before it is an engineering one.

If you are already on Postgres and your corpus is in the low millions, pgvector avoids introducing a new datastore for no measured benefit. Start there and move when you have evidence, not before.

Cost

Three components, and teams routinely model only the first:

Embedding generation — one-off per chunk, plus every re-index. Cheap per call, significant across millions of chunks and unbounded if you re-embed carelessly.
Storage — a 1536-dimension float32 vector is ~6 KB. Ten million chunks is ~60 GB before index overhead, and HNSW graphs can add substantially more.
Query — charged per operation or per provisioned unit. Hybrid search doubles the query count.

Two levers with real impact: dimensionality reduction, where some models support shortened embeddings at modest accuracy cost, and quantisation, which can cut memory several-fold. Both should be evaluated against your golden set rather than adopted on vendor claims.

Production lessons

Measure recall against a flat index. If you do not know your recall, you do not know your system works.
Require filtered ANN. Post-filtering is a data-leak pattern.
Version the embedding model with the index. Never mix generations.
Match the distance metric to the model. Mismatches fail quietly.
Handle tombstones explicitly so deleted content cannot be retrieved before compaction.
Tune efSearch per use case rather than accepting one default everywhere.
Start with the datastore you already run. Add a specialised vector database when you have measured a reason.
Re-index on source change, not on a timer.

Frequently asked questions

What is a vector embedding?

A fixed-length numeric representation of text — commonly 768 to 1536 dimensions — positioned so that semantically similar text lands nearby. It lets a search engine match meaning rather than words, so a question and a differently-worded answer can be matched.

Why is vector search bad at exact identifiers?

Because error codes, part numbers and field names carry little semantic signal. The embedding of a passage containing them is dominated by the surrounding prose, so the identifier itself barely influences the vector. This is why hybrid retrieval combining vector and keyword search exists.

What is the difference between HNSW and IVF?

HNSW builds a navigable small-world graph and gives excellent recall and latency at the cost of high memory. IVF partitions vectors into clusters and searches only the nearest few, using less memory but with recall depending on how many clusters are probed.

Why is post-filtering a problem in vector search?

Because unauthorised documents are retrieved before being discarded, meaning restricted content passed through your process and possibly your logs. It is a security defect rather than a performance choice — access control belongs inside the search query via filtered ANN.

What happens if you change embedding model?

You must re-embed and rebuild the entire index, because vectors produced by different models are not comparable. Version the embedding model alongside the index and never mix generations within one collection.