Pinned MongoDB 7.0.22 migration clone concurrency and buffer boundaries

Parent: Chunk migration commit, range deleter and orphan cleanup internals · Published reference · snapshot 2026-11-02

↓ Facts as markdown all context files

Source-anchored research for Pinned MongoDB 7.0.22 migration clone concurrency and buffer boundaries. Run status: COMPLETED.

These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.

How it works

The validator rejects settings when the concurrency feature flag is disabled, using an enabled check that ignores FCV. It rejects nonpositive values and, when test commands are disabled, values greater than `500`; therefore `500` passes its numeric condition. Test commands relax only the upper bound. The diagnostic instead says the level must be positive and less than the maximum, whose value is `500`. (high confidence) [source 1] [source 2] [source 3] [source 4]
The fetcher constructor captures `_chunkMigrationConcurrency` from `chunkMigrationConcurrency.load()` when `mongo::feature_flags::gConcurrencyInChunkMigration.isEnabledAndIgnoreFCVUnsafe()` is true, otherwise it captures `1`; its comment says the flag has no upgrade/downgrade concerns. Both minimum and maximum inserter pool sizes use the captured value. `fetchAndScheduleInsertion()` derives its fetcher count from that stored value, and each scheduled inserter receives it as the argument stored in `_threadCount`. A runtime parameter change does not resize these constructed pools or change this captured inserter argument. (high confidence) [source 1] [source 2] [source 3] [source 4]
`fetchAndScheduleInsertion()` shuts down and joins the fetcher pool. The header says insertions can still be running when it returns. The destructor calls inserter `waitForIdle()`, then `shutdown()`, then `join()`; its comment explains that joining first can spawn a thread beyond the pool maximum to finish pending work. The destination caller places the fetcher in an inner scope before checking interruption and reading clone progress. (high confidence) [source 1] [source 2] [source 3] [source 4] [source 5]
The fetcher passes captured `_chunkMigrationConcurrency` to each inserter, whose constructor stores `threadCount` in `_threadCount`. The insertion loop reaches `awaitReplication` only when `_writeConcern.needToWaitForOtherNodes()` is true, `_threadCount == 1`, and the shared secondary-throttle ticket is obtained. Consequently, donor compatibility forcing a single fetching worker does not by itself re-enable this wait when captured insertion concurrency exceeds `1`. (high confidence) [source 1] [source 2] [source 3] [source 4] [source 5] [source 6]
For the historical version boundary, MongoDB issue `SERVER-95324` lists `7.0.23` among its fix versions and describes making `chunkMigrationConcurrency` a no-op because of excessive resource consumption and, in some instances, increased orphaned documents. MongoDB’s release announcement dated August 14, 2025 lists that disablement in `7.0.23`. Percona separately lists the upstream removal in its `7.0.24-13` release notes. These attributed later-version notices constrain extrapolation beyond the selected historical implementation. (high confidence) [source 1] [source 2] [source 3]

Problems, failure modes and limitations

The buffer parameter `chunkMigrationFetcherMaxBufferedSizeBytesPerThread` is an `AtomicWord<int>` settable at startup and runtime, with default expression `4 * BSONObjMaxInternalSize` and validator `validateChunkMigrationFetcherMaxBufferedSizeBytesPerThread`. Its validator accepts `0` as no limit and otherwise requires at least `BSONObjMaxInternalSize`. The declaration describes a buffered-document limit per thread. (high confidence) [source 1] [source 2] [source 3] [source 4]
For a nonzero tracker budget, `waitUntilSpaceAvailableAndAdd` first rejects a single response larger than the budget with error `8120100`, then waits interruptibly under the mutex until `_currentSize + sizeBytes` fits and adds the reservation. `remove` checks that the tracked count covers the removal, subtracts it and notifies one waiter. When the budget equals `kUnlimited`, declared as `0`, both methods return without tracking. (high confidence) [source 1] [source 2] [source 3] [source 4]
The fetcher creates `_secondaryThrottleTicket` with capacity `1`. In the eligible insertion-wait branch, the inserter tries to acquire a ticket and uses `runWithoutSession` around `awaitReplication` for its last operation and supplied write concern. `WriteConcernFailed` logs a timeout warning and continues; other non-OK replication statuses are asserted. Failed ticket acquisition reaches `dassert(false)`. On the normal loop path, each iteration then applies `migrateCloneInsertionBatchDelayMS.load()` as a separate delay; its declaration defaults to `0`. These source predicates and the configured delay do not establish adaptive replication-lag control or a workload optimum. (high confidence) [source 1] [source 2] [source 3] [source 4] [source 5]
The fetching worker and the nonempty-batch path of the real inserter create cancelable operation contexts from the inner context cancellation token and check outer and worker interruption. Their `DBException` handlers kill the inner operation with distinct errors, `6718400` for fetching and `6718402` for insertion, then log the failure. The destination caller checks interruption after the fetcher destructor scope. (medium confidence) [source 1] [source 2] [source 3] [source 4] [source 5]
`BasicEmptyFetchingTest` sets the feature flag true and concurrency `30`, creates `MigrationBatchFetcher<MigrationBatchMockInserter>` with donor parallel support true and buffer limit `0`, and supplies one terminal response per worker. The mock insertion method only checks `invariant(status.isOK())`; its constructor and thread-creation method have empty bodies. These are fixture definitions, not real insertion or replication-wait assertions. (medium confidence) [source 1] [source 2] [source 3] [source 4] [source 5]
`BasicFetching` likewise uses concurrency `30`, donor parallel support true, buffer limit `0`, and the mock inserter. Its responder supplies `8` nonterminal batches followed by one terminal response per worker; the document helper builds three documents with matching `_id` and `X` values. The command callback asserts `_migrateClone` targets `test.foo`. These literal fixtures do not select a workload-optimal concurrency or exercise finite-buffer backpressure. (medium confidence) [source 1] [source 2] [source 3] [source 4] [source 5] [source 6]

Comparisons and alternatives

The parameter prose says per thread, but the pinned destination caller passes the loaded value directly to one MigrationBatchFetcher, whose constructor initializes one _bufferSizeTracker with that value. All its fetchers reserve through that tracker. The quoted implementation does not multiply the budget by captured concurrency. Treat this as shared tracked batch bytes within the fetcher object, rather than a proven resident-memory ceiling per thread. (high confidence) [source 1] [source 2] [source 3] [source 4] [source 5] [source 6]

Facts and statements

The GitHub reference for the selected historical release uses a release label prefixed with the letter r and points to annotated tag object `ef98696d6009abf1b6f73a935c9f7bb3fa3ea4c4`. That object repeats the same release label and resolves to commit `dacdbc3df2fbe579b03336a2f01fc9aedf406a41`. The implementation observations below are confined to that resolved release source. (high confidence) [source 1] [source 2]
At the pin, `featureFlagConcurrencyInChunkMigration` declares `feature_flags::gConcurrencyInChunkMigration`, default `true`, and version `6.3`. The `chunkMigrationConcurrency` parameter is an `AtomicWord<int>` settable at startup and runtime, defaults to `1`, and uses `validateChunkMigrationConcurrency`. Its declaration describes recipient insertion threads and parallel recipient-to-source `_migrateClone` requests. (high confidence) [source 1] [source 2] [source 3] [source 4] [source 5]
The header says `parallelMigrateCloneSupported` in `_recvChunkStart` advertises donor synchronization support and omission makes parallel requests unsafe. The implementation sets the fetcher pool minimum and maximum to `_isParallelFetchingSupported ? _chunkMigrationConcurrency : 1`; this compatibility fallback affects fetching while the inserter pool remains sized by the captured concurrency. (high confidence) [source 1] [source 2] [source 3] [source 4]
Each fetching worker receives `nextBatch` before reserving `nextBatch.objsize()` bytes, then constructs the inserter with `nextBatch.getOwned()` and schedules it. The scheduled callback installs `ON_BLOCK_EXIT` to call the tracker removal with `batchSize` around the callback body containing `inserter.run(status)`. The tracked amount covers admitted responses awaiting or performing insertion, while fetched responses also exist before admission. This response-byte accounting does not establish a per-thread resident-memory ceiling. (medium confidence) [source 1] [source 2] [source 3] [source 4]
The destination manager copies `cloneRequest.parallelFetchingSupported()` to `_parallelFetchersSupported`. Its initial bulk-clone scope constructs `MigrationBatchFetcher<MigrationBatchInserter>` with that value and `chunkMigrationFetcherMaxBufferedSizeBytesPerThread.load()`, calls `fetcher.fetchAndScheduleInsertion()`, exits the fetcher scope, checks interruption and reads the maximum cloned optime. The caller explicitly marks the destructor scope as semantically significant. (medium confidence) [source 1] [source 2] [source 3]
MongoDB issue `SERVER-87412` reports a non-thread-safe jumbo-chunk fetch path with `chunkMigrationConcurrency` greater than `1` and describes invariant-triggered server restarts. Its affected-version field is `5.0.0`, and its resolution is Won’t Fix. This attributed tracker report supports a caution against unconditional concurrency safety or performance guarantees; its stated affected version does not identify an incident in the selected historical deployment. (low confidence) [source]

Children

No children recorded.

← the whole tree · 3D view· how to read this page