Pinned secondary packed-container expansion and writer-key routing boundaries
Parent: Pinned pipelined oplog applier in-flight backpressure and publication boundaries · Published reference · snapshot 2026-11-02
↓ Facts as markdownall context files
Source-anchored research for Pinned secondary packed-container expansion and writer-key routing boundaries. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Structure and components
- Expansion copies the original raw entry with its object field removed and rebuilds only that field; the replacement object is appended last, so BSON field order can change. The metadata fixture covers shared-value arrays, paired arrays, integer ranges and key-array deletes, asserting preservation of optime, entry timestamp, wall-clock time and container identity. (high confidence) [source 1] [source 2]
How it works
- The standalone container path runs only when derived-operation storage is supplied. It expands one container entry, moves the resulting vector into that storage, distributes the derived entries with serial routing disabled, and continues without distributing the original entry. An empty expansion follows the same continue path and contributes no writer operation; an unexpanded scalar falls through to ordinary writer assignment. (high confidence) [source 1] [source 2]
- DistinctKeysInOneExpansionHashDifferently expands three selected byte keys and asserts pairwise unequal hashes. It prevents the selected equality fixtures from passing merely because those keys all receive a key-independent hash; it does not assert that arbitrary distinct keys have collision-free hashes or distinct writer indices. (low confidence) [source]
Comparisons and alternatives
- Packedness depends on array representation, not the number of resulting operations: a scalar entry yields no expansion, a one-element packed array still requires scalarization, and an empty packed array yields an engaged optional containing an empty vector. Callers must drop the empty expansion rather than pass through the packed entry. The bulk expansion separately tracks whether any entry was packed, even when the total operation count stays unchanged. Ordinary scalar container operations bypass expansion. The scalar fixture asserts one surviving operation and no derived operations. The empty-array fixture asserts engaged empty expansions for insert and delete and checks that only the unrelated scalar insert survives writer-vector derivation. The one-element nested fixture asserts that the surviving container operation has a scalar key even though expansion leaves the operation count unchanged. (high confidence) [source 1] [source 2] [source 3]
- Bulk expansion preserves relative order and leaves entries that need no expansion intact. It first counts the expanded size, builds a separate vector, copies unchanged entries instead of moving from the input, and commits with a vector swap. Thus a throw during construction leaves the original vector contents intact; the replacement is all-or-nothing. (high confidence) [source 1] [source 2]
- The helper assertExpansionHashesMatchScalars compares raw entry hashes of expanded operations with separately constructed scalar entries in order. ExpandedBytesKeyArrayHashesLikeScalarEntries, ExpandedIntKeyRangeHashesLikeScalarEntries, ExpandedPairedArraysHashLikeScalarEntries and ExpandedKeyArrayDeleteHashesLikeScalarEntries apply it to the four packed shapes. This checks hash equality before writer-count modulo reduction. These are selected source-fixture assertions, rather than execution results or measurements of application parallelism. (low confidence) [source]
Facts and statements
- The expansion interface requires packed container entries to become single-key entries before writer distribution. Container routing hashes the container identity with the scalar key hash, so an insert and a delete of the same key use the same hash domain; hashing a packed entry as one unit would break that correspondence. Writer assignment rejects entries whose key or value remains packed. The declared expansion contract preserves relative order and says operations touching a container key are applied by the same thread in oplog order. (high confidence) [source 1] [source 2]
- A packed byte-key array insert expands into one scalar insert per key with the shared value, including an absent value. A packed key-array delete expands into one scalar delete per key. BytesKeyArrayInsertIsSplitPerKey constructs three packed keys and a scalar delete, then asserts four container operations and equal writer indices for the matching insert and delete. KeyArrayDeleteIsSplitPerKey likewise checks the expanded operation count and equal writer indices for the selected shared key. (medium confidence) [source 1] [source 2]
- For an integer base key paired with an array of values, expansion constructs consecutive integer keys by adding each value index to the base key. The IntKeyRangeInsertIsSplitPerKey fixture uses base 100 and asserts three container operations and that the expanded insert at base + 1 and a scalar delete of that key share a writer. (high confidence) [source 1] [source 2]
- For paired key and value arrays, expansion checks equal lengths and emits each key with the value at the same index. The PairedKeyAndValueArraysAreSplitPairwise fixture checks the resulting operation count and equal writer assignment for the second key's insert and scalar delete. (medium confidence) [source 1] [source 2]
- In the direct terminal applyOps extraction branch, the dispatcher extracts operations, bulk-expands packed container operations, stores the extracted operations, then assigns writers with serial=false. This branch is selected for an unlinked entry or for the single-entry retryable-batch case; other linked entries use the chain-reading path. The transaction-operation dispatch helper also expands its vector before storing it and routing derived operations. These are caller placements before writer selection. The nested applyOps fixture asserts four container operations, equal writer indices for the shared-key insert and delete, and preservation of the ordinary collection insert. (medium confidence) [source 1] [source 2] [source 3]
- Derived-operation storage owns the expanded entries while writer vectors receive pointers to them. The dispatch storage helper explicitly retains extracted operations until batch application completes. The ordinary applier keeps its derived-operation vector in scope across writer construction, worker scheduling and the wait for the worker pool to become idle. (high confidence) [source 1] [source 2] [source 3]
- Ordinary CRUD routing uses the namespace and, for non-capped or clustered collections, the document identifier hash with the collection collator. Non-clustered capped collections omit the document identifier from this hash; capped inserts are marked to avoid bulk insertion. Container routing instead combines container identity with the scalar key hash, ignores the key field name and uses no collator. These are separate routing domains. (low confidence) [source]
- Writer selection reduces either the forced writer identifier or the computed entry hash modulo the number of writers. In derived-operation serial mode, the first operation selects a writer and subsequent operations receive that writer identifier as the override. The direct terminal applyOps extraction branch and the standalone expansion path pass serial routing disabled; scalar expansion therefore does not itself impose document-to-container serial ordering. (high confidence) [source 1] [source 2]
- The death fixture PackedEntryReachingWriterAssignmentFails passes a packed key-array insert directly to hashing and expects the diagnostic "container op packing multiple keys reached writer thread assignment". The implementation enforces the corresponding invariant before hashing. (medium confidence) [source 1] [source 2]
Children
- No children recorded.