# Embedding generation and reranking token spend in retrieval TCO > Source-anchored research for Embedding generation and reranking token spend in retrieval TCO. Run status: COMPLETED. Parent: [Platform Consolidation TCO (Atlas Search/Vector Search vs Separate Search or Vector DB)](https://llms-explorer.com/tree/platform-consolidation-tco-atlas-search-vector-search-vs-separate-search-or-vector-db/) · 5 facets · 16 facts · page: https://llms-explorer.com/tree/embedding-generation-and-reranking-token-spend-in-retrieval-tco/ ## Structure and components - Consolidating retrieval into Atlas does not remove embedding and reranking inference spend: automated embedding meters processed tokens, while native reranking meters tokens processed by the $rerank stage. Treat these as explicit TCO components alongside the retrieval deployment. (high confidence) (verified as of 2026-10-14) — [source](https://www.mongodb.com/docs/vector-search/query/native-reranking/billing.md) ## How it works - Returning fewer ranked results is distinct from submitting fewer candidates. Voyage top_k controls returned results; its token formula uses all submitted documents. Inference: lowering top_k alone does not lower this input-token count when the same documents and query are processed. (high confidence) — [source](https://www.mongodb.com/docs/vector-search/query/native-reranking/billing.md) - Illustrative workload calculation, not a vendor benchmark: a 20-token query reranked against 100 passages of 500 tokens processes 52,000 tokens per request under the documented formula; one million such requests process 52 billion tokens. Multiply by the applicable model rate and deduct eligible credits separately. (high confidence) (verified as of 2026-10-14) — [source](https://www.mongodb.com/docs/vector-search/query/native-reranking/billing.md) ## Problems, failure modes and limitations - Bifrost issue #4239, reported June 10, 2026 against transports/v1.5.11, describes Cohere reranking costs appearing null because search_units were not mapped into usage. Treat this as a reported integration failure at the named revision, not proof that the present release remains affected; reconcile usage with upstream billing. (low confidence) (verified as of 2026-06-10) — [source](https://github.com/maximhq/bifrost/issues/4239) - Anthropic's September 2024 Contextual Retrieval experiment reranked 150 retrieved chunks and sent 20 to generation. Its reported 67% retrieval-failure reduction compares the combined contextual-embedding, contextual-BM25 and reranking pipeline against its baseline; it is not the isolated benefit of reranking or evidence that every workload needs that candidate depth. (low confidence) — [source](https://www.anthropic.com/research/contextual-retrieval) ## Comparisons and alternatives - Reranking's added processing cost alone does not determine end-to-end TCO. Anthropic describes reducing generation context through reranking while acknowledging added runtime latency and cost; the BrowseComp-Plus study finds favorable reranking-versus-reasoning budget trade-offs. Compare total model spend and answer quality at matched workloads rather than assuming a universal net saving or net penalty. (high confidence) — [source](https://www.anthropic.com/research/contextual-retrieval) - A single-machine Jacar experiment on Spanish MIRACL reported 3,420 average reranking tokens for 20 candidates versus 9,066 for 50 on a 20-query sample, with no nDCG improvement from the larger candidate set in its quality evaluation. These are workload-specific measurements, not Atlas pricing or a general throughput forecast. (low confidence) — [source](https://jacar.es/en/local-reranking-llama-cpp-pgvector/) - Reranking's quality payoff is workload-dependent: Anthropic reported improved retrieval failure rates with contextual retrieval plus reranking, whereas the EncouRAGe preprint abstract reports marginal reranking gains with higher latency. The experiments use different pipelines and datasets, so this scopes universal ROI claims rather than establishing a direct contradiction. (medium confidence) — [source](https://www.anthropic.com/research/contextual-retrieval) - The BrowseComp-Plus study Rerank Before You Reason reports that moderate reranking can achieve comparable accuracy at lower effective token cost than increasing search-time reasoning. This supports measuring the combined retrieval-and-reasoning budget, but its benchmark does not establish Atlas-versus-separate-database savings. (low confidence) — [source](https://aclanthology.org/2026.findings-acl.1289/) ## Facts and statements - Atlas automated embedding accounts for tokens during initial index creation, document inserts and updates, and queries. Budget corpus ingestion separately from recurring document-change and query-embedding traffic; this source does not establish that every ordinary index rebuild necessarily regenerates embeddings. (low confidence) (verified as of 2026-10-14) — [source](https://www.mongodb.com/docs/vector-search/crud-embeddings/automated-embedding/billing.md) - Voyage reranker token accounting repeats the query for every candidate: total tokens = query_tokens × documents_reranked + sum(document_tokens). Thus candidate count, passage length, and query length directly affect metered reranking work. (high confidence) (verified as of 2026-10-14) — [source](https://www.mongodb.com/docs/vector-search/query/native-reranking/billing.md) - Atlas Automated Embedding and Native Reranking each document one-time organization-level free-token allocations shared across projects and clusters; those allocations do not refresh. Treat introductory credits separately from steady-state TCO. (high confidence) (verified as of 2026-10-14) — [source](https://www.mongodb.com/docs/vector-search/crud-embeddings/automated-embedding/billing.md) - Billing units differ across providers: MongoDB documents token-based native reranking, while Cohere documents reranking charges by searches and embedding charges by tokens embedded. Normalize each workload through the provider's actual billing unit before comparing cost. (high confidence) (verified as of 2026-10-14) — [source](https://www.mongodb.com/docs/vector-search/query/native-reranking/billing.md) - Context enrichment can add a preprocessing inference step and enlarge embedding inputs. Anthropic's contextual-retrieval method prepends roughly 50–100 generated tokens to each chunk before embedding; its historical cost example assumes prompt caching. Include these tokens and the contextualizer workload if that design is used. (low confidence) — [source](https://www.anthropic.com/research/contextual-retrieval) - Atlas Native Reranking documents processed tokens as query_tokens × numDocsToRerank + the sum of tokens across all reranked documents; it bills the $rerank stage separately from other Voyage AI operations. (low confidence) (verified as of 2026-10-14) — [source](https://www.mongodb.com/docs/vector-search/query/native-reranking/billing.md) - Derived budgeting method: model embedding cost as initial corpus tokens plus tokens processed for subsequent content changes and queries, multiplied by the applicable model rate. Keep corpus rebuild scenarios separate from normal monthly traffic; MongoDB's API billing also explicitly counts document and query tokens. (medium confidence) (verified as of 2026-10-14) — [source](https://www.mongodb.com/docs/vector-search/crud-embeddings/automated-embedding/billing.md)