Time-series bucket closure under cache pressure
Parent: WiredTiger cache eviction and dirty ratio · Published reference · snapshot 2026-10-30
↓ Facts as markdownall context files
Source-anchored research for Time-series bucket closure under cache pressure. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
How it works
- BucketCatalog memory reclamation archives idle buckets when timeseriesIdleBucketExpiryMemoryUsageThreshold is exceeded. Archiving keeps a small record for reopening; if pressure remains after idle buckets are archived, archived entries are removed. Buckets removed this way remain eligible for query-based reopening. (low confidence) [source]
- One independent user experiment reported that a workload producing 25,000 buckets before other collections were loaded later produced about 420,000 buckets and 415,000 cache-pressure closures. This is a workload observation, not a universal threshold or proof of a dirty-ratio trigger. (low confidence) (verified as of 2023-10-19) [source]
- Bucket-catalog memory reclamation is a separate path. Its memory threshold causes idle buckets to be archived or their retained entries removed, and serverStatus exposes numBucketsClosedDueToMemoryThreshold and numBucketsArchivedDueToMemoryThreshold separately from numBucketsClosedDueToCachePressure. (medium confidence) (verified as of 2026-10-30) [source]
- The inspected rollover code includes a large-measurement exception: below the configured minimum measurement count, a bucket can exceed its effective ordinary size ceiling, but a separate absolute ceiling bounded by the cache-derived maximum can still trigger cache-pressure rollover. (low confidence) (verified as of 2026-10-30) [source]
Measurements and reference values
- MongoDB 8.0 documents the BucketCatalog memory allocation as 2.5% of available system memory and lists exceeding this allocation separately from active buckets not fitting the storage-engine cache. These are distinct conditions, so changing the catalog threshold is not established as a remedy for every dynamic cache-pressure closure. (medium confidence) [source]
Problems, failure modes and limitations
- MongoDB 8.0 defines numBucketsClosedDueToCachePressure as closures caused by exceeding a dynamic size limit derived from storage-engine cache size and numBuckets. It defines numBucketsClosedDueToMemoryThreshold separately for active buckets exceeding the BucketCatalog memory allowance. (medium confidence) [source]
- MongoDB 8.0 defines numBucketsClosedDueToCachePressure as closures caused by a dynamic bucket-size limit derived from storage-engine cache size and numBuckets. The implementation classifies an insertion exceeding the effective cache-derived size ceiling as kCachePressure when that ceiling is tighter than the ordinary size ceiling. (medium confidence) (verified as of 2026-10-30) [source]
- A cache-pressure bucket-closure counter is not a direct dirty-ratio measurement. The documented counter and inspected rollover branch use a cache-derived bucket-size limit, while PMM separately charts modified/unmodified page evictions and cache page states. Correlating these metrics is a diagnostic inference; one cannot infer a specific dirty-cache threshold crossing from the bucket counter alone. (medium confidence) (verified as of 2026-10-30) [source]
- Cache-sizing remedies depend on the hosting service. IBM Cloud Databases for MongoDB Gen 2 states that customers cannot configure its internal/file-system cache ratio directly; customers can scale total memory, and its performance guide suggests memory scaling when disk I/O utilization warrants additional caching. This service-specific option does not establish that extra memory resolves every bucket-closure incident. (low confidence) (verified as of 2026-10-30) [source]
Comparisons and alternatives
- In the implementation snapshot, the cache-derived size calculation takes cache capacity and active-bucket cardinality, rather than a measured WiredTiger dirty percentage. Consequently, a cache-pressure bucket closure alone does not prove that a dirty-ratio eviction trigger fired. (medium confidence) (verified as of 2026-10-30) [source]
- SERVER-69899 explains the design tradeoff: many open or archived buckets growing to the normal 125 KiB limit can cause repeated disk writes. Dynamically reducing bucket sizes targets roughly half the WiredTiger cache, at the cost of worse disk usage and query performance. The implementation applies a minimum-size floor, so the half-cache target is a heuristic rather than an unconditional memory bound. (medium confidence) [source]
- For diagnosis, inspect the distinct bucket closure counters and correlate them with storage-engine measurements. Percona PMM provides cache Used versus Max and separate rates for modified and unmodified page evictions; those measurements add context to the bucket counters rather than making the counters interchangeable. (medium confidence) (verified as of 2026-10-29) [source]
- SERVER-105187 scopes the dynamic-sizing heuristic: its calculation combines global bucket cardinality across collections and a fixed relationship to WiredTiger cache size, which may not suit every workload. The ticket describes raising timeseriesBucketMinSize to timeseriesBucketMaxSize as effectively disabling dynamic bucketing, but presents this as internal behavior rather than a generally validated tuning recommendation. (low confidence) (verified as of 2025-07-14) [source]
- For operational investigation, PMM's WiredTiger Details dashboard separates cache usage, configured maximum, cache reads and writes, and modified versus unmodified eviction rates. These panels provide storage-engine context for a time-series closure-counter increase; they do not measure bucket closure itself. (low confidence) (verified as of 2026-10-29) [source]
Facts and statements
- The master implementation snapshot computes a dynamic size as storageCacheSizeBytes divided by twice workloadCardinality, floors it at gTimeseriesBucketMinSize, and caps the ordinary effective maximum at gTimeseriesBucketMaxSize. Zero cardinality returns the ordinary maximum. The call site supplies global numActiveBuckets, so pressure can reflect activity beyond one collection. (low confidence) (verified as of 2026-10-30) [source]
- SERVER-89233 records a historical sharded-cluster symptom where bucket counts approached document counts and numBucketsClosedDueToCachePressure increased. Its ticket lists affected versions 7.0.4 and 7.0.5 and fix versions 8.1.0-rc0 and 8.0.5. These ticket fields do not establish every affected or fixed release. (low confidence) (verified as of 2025-04-25) [source]
- A July 30, 2021 +N Consulting walkthrough independently demonstrated a timeseries section in collection statistics containing size, time, count, and memory-threshold closure counters. It is historical evidence that bucket closure reasons were observable; it does not validate the later dynamic cache-pressure formula. (low confidence) [source]
Children
- Cache-derived bucket size minimum and workload cardinality (frontier)
- BucketCatalog idle-bucket archival and query-based reopening (frontier)
- Version-scoped singleton-bucket regression (frontier)
- Cache-derived dynamic bucket size limits (frontier)
- Bucket-catalog memory reclamation (frontier)
- Time-series bucket closure diagnostics (frontier)
Frontier under this node: Bucket-catalog memory reclamation, BucketCatalog idle-bucket archival and query-based reopening, Cache-derived bucket size minimum and workload cardinality, Cache-derived dynamic bucket size limits, Time-series bucket closure diagnostics, Version-scoped singleton-bucket regression