Quantile Estimation (DDSketch)
Parent: Client Side Operations Timeout (CSOT) · Published reference · snapshot 2026-10-15 · viability 2.70 / 5 (skip)
↓ Facts as markdownall context files
Depth-first rabbithole dossier for Quantile Estimation (DDSketch); source-anchored research pack.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Definitions
- [low] The bucket index for DDSketch is calculated as ceil(ln(value) / ln(gamma)), where gamma is defined as (1 + alpha) / (1 - alpha) and alpha represents the desired relative accuracy. Sources: https://infoq.com/articles/adaptive-hedged-requests-p99-latency [source]
How it works
- [low] DDSketch is capable of handling endpoints that process over 10M points per second, providing accurate latency quantiles for these high-throughput scenarios. Sources: https://ar5iv.labs.arxiv.org/html/1908.10693 [source]
- [low] The MongoDB $percentile accumulator requires quantile elements in p to be numeric values in the range 0.0 to 1.0 inclusive, and because its t-digest approximation uses pseudorandomness, computed percentile values on the same dataset might differ on each run. Sources: https://www.mongodb.com/docs/manual/reference/operator/aggregation/percentile.md [source]
Problems, failure modes and limitations
- [high] DDSketch is the first fully-mergeable, relative-error quantile sketching algorithm with formal guarantees, providing O(1) constant-memory quantile estimation with relative-error guarantees of plus or minus one percent. Sources: https://ar5iv.labs.arxiv.org/html/1908.10693, https://infoq.com/articles/adaptive-hedged-requests-p99-latency [source]
Facts and statements
- [high] DDSketch has a critical-path overhead of approximately thirty-five nanoseconds per request, making it suitable for real-time per-host latency tracking, and can accurately maintain upper half order statistics of over a million samples with a sketch size of 273 with probability greater than 0.99991. Sources: https://ar5iv.labs.arxiv.org/html/1908.10693, https://infoq.com/articles/adaptive-hedged-requests-p99-latency [source]
- [low] Adding a hundred million values to an empty DDSketch takes less than 5 seconds on a 3.1GHz MacBook Pro, while merging two DDSketches containing up to fifty million values each takes around 10 microseconds or less. Sources: https://ar5iv.labs.arxiv.org/html/1908.10693 [source]
- [low] A tumbling window fix uses two DDSketches rotating at a fixed interval, such as thirty seconds, to manage data aging. Sources: https://infoq.com/articles/adaptive-hedged-requests-p99-latency [source]
Related concepts
- DDSketch — is a part of Quantile Estimation (DDSketch)
- Quantile — is a part of Quantile Estimation (DDSketch)
- Estimation — is a part of Quantile Estimation (DDSketch)
Children
- No children recorded.