Atlas load-shedding backpressure and retry-budget handling
Parent: MongoDB Driver Internals · Published reference · snapshot 2026-10-29
↓ Facts as markdownall context files
Source-anchored research for Atlas load-shedding backpressure and retry-budget handling. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Structure and components
- Do not independently restart retry budgets at every application or service layer. Nested retries multiply database traffic; coordinate a total attempt budget and deadline across driver retries and application resubmissions. (high confidence) [source]
How it works
- The cached Atlas IWM documentation describes ingress queueing separately from optional load shedding: enabling shedding bounds the ingress queue and rejects arrivals when it is full; disabling shedding allows the queue to grow. (low confidence) (verified as of 2026-10-14) [source]
- SERVER-130672 documents stale test expectations after SERVER-129657 corrected overload counters that previously incremented even when maxTimeMS expired during backoff before retry dispatch. Observing an overload label was not proof that a retry occurred. (low confidence) (verified as of 2026-07-14) [source]
- Atlas Load Shedding rejects operations during prolonged overload. A backpressure-aware driver can retry eligible failures, but the application must decide when to abandon work that the cluster continues rejecting. (high confidence) (verified as of 2026-10-14) [source]
- With optional Load Shedding enabled, Atlas bounds each relevant ingress queue and rejects arrivals when it is full. Disabling this setting leaves ingress queueing and other supported IWM protections active, so disabling shedding does not remove overload latency. (high confidence) (verified as of 2026-10-14) [source]
How-to and procedures
- Avoid independent retries in multiple application layers around driver retries: retry counts multiply through dependency stacks. Google and Amazon both recommend concentrating retries to prevent this amplification. (high confidence) [source]
Measurements and reference values
- Specified backoff is U(0,1) × min(10000 ms, base × 2^retryNumber), with a 100 ms default base and first retry numbered one. A positive server baseBackoffMS replaces the base; Node implements these rules. (medium confidence) (verified as of 2026-10-27) [source]
Problems, failure modes and limitations
- The client-backpressure specification defaults maxAdaptiveRetries to two retries, excluding the initial attempt. Node operation execution converts that option to maxAdaptiveRetries + 1 total attempts after an overload error. (medium confidence) (verified as of 2026-10-27) [source]
- Atlas resilience tests temporarily enable Load Shedding and reject a small random fraction of operations even without overload, then restore the previous setting. Such a test checks error handling, not proof of goodput or stability under sustained overload. (low confidence) (verified as of 2026-10-14) [source]
- The specified maxAdaptiveRetries default is two retries, separate from the initial attempt. MongoDB also recommends at most two application overload retries, balancing error reduction against added load. (high confidence) (verified as of 2026-10-27) [source]
- The Java killCursors overload-retry task remained unresolved when checked; its description reports no retry wrapper and says cursor close swallows the failure. Therefore the specification alone does not establish complete command coverage in every driver. (low confidence) (verified as of 2026-10-19) [source]
Comparisons and alternatives
- Node checks whether the selected overload backoff exceeds remaining CSOT time and propagates the error instead of sleeping and dispatching another attempt when that check fails. (low confidence) (verified as of 2026-10-27) [source]
- During sustained rejection, throttle application requests and reduce admitted work instead of continuously replenishing failed operations. MongoDB documents client request throttling, while Google describes client-side rejection based on recent acceptance and rejection rates. This is application policy, not proof of automatic driver concurrency control. (high confidence) [source]
Facts and statements
- SystemOverloadedError signals overload, not retry safety. RetryableError identifies a safely retryable rejection; NoWritesPerformed states that no writes occurred. Without a safety guarantee, an application must assess idempotency before resubmitting. (high confidence) (verified as of 2026-10-27) [source]
- Once overload occurs, the specified cap counts all retries, including non-overload failures. CSOT must still leave time for another attempt; an available deadline does not permit unlimited overload retries. (low confidence) (verified as of 2026-07-08) [source]
- The overload specification extends retries to commands such as updateMany, createCollection, getMore and runCommand. Reads require retryReads, writes require retryWrites, and runCommand requires both. Connection establishment and monitoring are excluded. (medium confidence) (verified as of 2026-07-08) [source]
- A per-operation retry cap does not bound aggregate retry traffic. Application designers can add a shared retry budget or local retry token bucket; Google describes per-client budgets and Amazon describes token-bucket throttling. These are external design patterns, not established Atlas-driver guarantees. (high confidence) [source]
- Reserve prompt overload retries for latency-sensitive or business-critical operations. MongoDB recommends logging background failures and retrying them at a higher level after longer delays; Google similarly distinguishes traffic that can tolerate delayed completion. (high confidence) [source]
- Exponential backoff should include jitter so clients do not retry together. A maximum delay alone still permits repeated retry traffic at the cap; bounded attempts and stopping unproductive retries are also necessary. (high confidence) [source]
Children
- Application admission control under sustained Atlas overload (frontier)
- Cross-layer retry amplification and shared retry budgets (frontier)
- Application-wide overload retry budgets (frontier)
- Deadline-aware retry admission (frontier)
- Adaptive application concurrency under sustained overload (frontier)
Frontier under this node: Adaptive application concurrency under sustained overload, Application admission control under sustained Atlas overload, Application-wide overload retry budgets, Cross-layer retry amplification and shared retry budgets, Deadline-aware retry admission