numCandidates tuning for quantized HNSW indexes

Parent: Atlas Vector Search automatic quantization and rescoring · Published reference · snapshot 2026-10-30

Source-anchored research for numCandidates tuning for quantized HNSW indexes. Run status: COMPLETED.

These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.

How it works

Treat graph exploration and rescoring depth as distinct controls when comparing engines. Qdrant exposes hnsw_ef separately from quantization oversampling; Elastic retrieves candidates using num_candidates and rescores a top k-times-oversample set. These factors do not define a portable Atlas numCandidates multiplier. (medium confidence) [source]

How-to and procedures

Use full-precision exact nearest-neighbor results as the retrieval-recall reference and measure latency while sweeping candidate settings. Atlas documents ENN for measuring ANN recall and states that automatically quantized indexes use full-fidelity vectors for ENN; Elastic requires measuring recall and latency on the actual data. (medium confidence) [source]

Measurements and reference values

In MongoDB’s 15.3-million-vector benchmark, a roughly 3%-selective filter required about four times as much work for binary quantization at lower limits to achieve 90–95% recall. Filtered and unfiltered queries need separate tuning; this multiplier is specific to that benchmark. (low confidence) [source]
In MongoDB's larger benchmark, a roughly 3% selective filter required about four times the work for binary search at lower limits to reach 90–95% recall. Binary rescoring also reduced throughput under concurrency; these observations require workload-specific validation. (low confidence) [source]

Problems, failure modes and limitations

In the cached MongoDB operator reference, ANN numCandidates must be at least limit and at most 10,000. ENN with exact:true supplies a separate accuracy baseline; automatic-quantization ENN uses full-fidelity vectors. (low confidence) (verified as of 2026-10-14) [source]
MongoDB binary quantization rescores retrieved candidates with retained full-precision vectors; its scalar path in these experiments does not. Wider binary candidate pools can therefore repair final ordering, but rescoring cannot recover a true neighbor absent from the pool. (medium confidence) [source]
Qdrant exposes hnsw_ef separately from quantization oversampling and rescoring. Its example sets hnsw_ef=128, limit=10, oversampling=3.0 and rescore=true. This illustrates separate graph-exploration and reranking controls; those example values are not a direct translation of MongoDB numCandidates. (low confidence) (verified as of 2026-10-30) [source]
TCO implication, inferred from the evidence: compare configurations at matched recall and production concurrency. Binary rescoring and larger candidate budgets add query work; MongoDB’s benchmark reports reduced concurrent throughput from rescoring, while Qdrant likewise treats oversampling and rescoring as accuracy/performance tradeoffs. Compression alone cannot establish serving-cost savings. (medium confidence) [source]
MongoDB recommends at least 20 times limit as an initial numCandidates setting, then workload-specific tuning. Its documented ANN range requires limit <= numCandidates <= 10000. This is guidance and a parameter constraint, not a quantized-recall guarantee. (low confidence) (verified as of 2026-10-14) [source]
In MongoDB binary-quantization search, the accumulated top numCandidates are rescored with full-fidelity vectors before returning limit results. Qdrant independently documents retrieving extra quantized candidates and rescoring them using original vectors. (medium confidence) [source]

Comparisons and alternatives

MongoDB numCandidates controls the HNSW candidate priority-queue size, rather than an exact count of all visited vectors. MongoDB recommends starting at 20 times limit and tuning for the dataset; quantized vectors may require more exploration than float32. (low confidence) (verified as of 2026-10-14) [source]
The August 2026 Winata experiment used 75,000,649 visual embeddings, 512 dimensions, 300 queries, and document-ID recall@72. At 10,000 candidates it reported binary mean recall of 95.8% versus scalar 73.1%. It excluded self-matches and used independently recomputed float32 cosine ground truth; it tested no query-time filters. (low confidence) [source]
Atlas numCandidates controls the HNSW search priority-queue size. Increasing it explores more of the graph and can improve recall at increased latency; Elastic documents the same direction of trade-off for its HNSW num_candidates control. (medium confidence) [source]
HighLevel reports that increasing numCandidates worsened its retrieval latency rather than fixing the bottleneck. It separately found scalar quantization retained quality better than binary on its slow-query sample. This is one operator's experience, not a universal Atlas outcome or controlled cross-engine benchmark. (low confidence) [source]

Facts and statements

Increasing numCandidates does not guarantee recovery of quantization loss. MongoDB benchmarks and an independently executed 75-million-vector experiment both show scalar recall plateauing while binary recall continues improving with additional candidates. This scopes blanket advice to raise candidates slightly after quantization. (medium confidence) [source]
Measure ANN recall against exact full-precision neighbors using top-k document-ID overlap, keeping query vectors and k fixed. MongoDB describes ENN as the accuracy baseline, and Elastic demonstrates the same control using a separate flat float32 field. Nearest-neighbor recall does not itself establish application relevance. (medium confidence) [source]
HighLevel’s May 4, 2026 production write-up reports that increasing MongoDB numCandidates, including an example increase from 100 to 2,000, worsened latency. This is a workload-specific counterexample to treating a larger candidate budget as a speed optimization; the write-up does not isolate that test’s quantization setting. (low confidence) [source]
OneUptime’s example compares quantized ANN IDs with unquantized ANN IDs at numCandidates=100 and labels the overlap recall. That measures agreement with another approximate result set. MongoDB’s ENN baseline and Elastic’s flat-search control are better suited to measuring absolute nearest-neighbor recall; the example’s 150/300-candidate settings do not establish portable recall guarantees. (medium confidence) [source]
Quantized Atlas vectors may require more candidates than float32 to maintain recall. MongoDB observed substantially greater candidate requirements for binary quantization in its 15.3M-vector, 2048-dimensional benchmark; the measured requirement is workload-specific. (medium confidence) [source]
In MongoDB's 5.5M-document multidimensional benchmark, scalar recall plateaued as candidates increased, while binary recall improved with a latency penalty especially above 1000 candidates. More candidates therefore did not uniformly repair quantization loss. (low confidence) [source]

Children

Quantization ranking error versus HNSW exploration error (frontier)
Exact-neighbor ground truth and recall distributions (frontier)
Filtered candidate tuning under concurrency (frontier)

Frontier under this node: Exact-neighbor ground truth and recall distributions, Filtered candidate tuning under concurrency, Quantization ranking error versus HNSW exploration error

← the whole tree · 3D view· how to read this page