Pinned event-mongodb payload recovery republishing and reset exclusion boundaries
Parent: Change Data Capture Patterns · Published reference · snapshot 2026-11-02
↓ Facts as markdownall context files
Source-anchored research for Pinned event-mongodb payload recovery republishing and reset exclusion boundaries. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
How it works
- For each loaded payload entry, recovery adds its original message ID to the handled set before checking EventName. An empty EventName causes a skip with no Send and no recovery-count increment; that original ID remains excluded from reset during the same pass. The code comment identifies entries written before EventName was added as the reason for this guard. (low confidence) [source]
Problems, failure modes and limitations
- A Send error skips marking and clearing, retains the payload for a later recovery cycle, and leaves the original ID in the same-pass reset exclusion set. The failing-publisher memory fixture asserts that RecoverOnce returns zero recovered and no pass error in that case. (medium confidence) [source 1] [source 2]
- After Send succeeds, recovery calls MarkProcessed on the original ID before ClearPayload. A MarkProcessed error retains the payload, skips the count increment, and preserves same-pass reset exclusion. A later recovery cycle can send the payload again; the runner comment explicitly warns that a crash between republishing and marking can also cause another republish. (low confidence) [source]
- After MarkProcessed returns nil, recovery ignores the ClearPayload error and increments recovered. In MongoStateManager, MarkProcessed updates status to completed while ClearPayload unsets only payload. Thus, if marking updates an existing document and clearing then fails, the retained payload alone does not satisfy LoadStalePayloads, which requires processing status. (medium confidence) [source 1] [source 2]
- The positive batch budget for reset is batchLimit minus the count of successful payload recoveries, while LoadStalePayloads receives the original batchLimit. Skipped payload entries do not consume that success count. ListStale receives the remaining limit before exclusion, so excluded IDs can reduce useful reset work without a refill. A zero batchLimit means unlimited; the memory zero-limit fixture asserts recovery of all three stale entries. (medium confidence) [source 1] [source 2]
- LoadStalePayloads, ListStale or bulk ResetStale errors propagate as pass errors. In contrast, individual Send, MarkProcessed and Reset failures are recorded and skipped, and ClearPayload errors are ignored. RecoverOnce returns the successful payload total if its reset helper reports an error. (low confidence) [source]
Comparisons and alternatives
- MongoStateManager stores payload on the existing message document, with metadata and event name set only when nonempty; nil data or an empty payload returns without writing. LoadStalePayloads selects processing documents whose updated_at precedes the stale cutoff and whose payload field exists, applies a positive query limit, and reconstructs payload, metadata and event name. These predicates use updated_at rather than expires_at. (low confidence) [source]
- MongoStateManager MarkProcessed filters by message ID and returns nil when UpdateOne returns no error; it does not inspect matched or modified counts. Therefore, the runner counts a nil marking result rather than independently verifying a completed-state transition. (medium confidence) [source 1] [source 2]
- Reset exclusion contains all IDs loaded by the payload phase, including failed sends, failed marking and missing event names. With a nonempty exclusion set, reset uses ListStale and individual Reset calls instead of the optional bulk-reset path. It skips excluded IDs after listing; it does not request replacements for excluded results. (low confidence) [source]
- The MongoDB payload lifecycle fixture acquires a message, stores payload, metadata and event name, then asserts that the stale loader returns all those fields. After ClearPayload it asserts an empty stale-payload result. The fixture setup says tests skip when MongoDB is unavailable or short mode is set; the lifecycle fixture exercises state storage operations rather than RecoveryRunner. (low confidence) [source]
Facts and statements
- Payload-aware recovery requires both a coordinator implementing PayloadStore and a configured Publisher. RecoverOnce first republishes loaded stale payload entries, then resets remaining eligible stale states. Without a Publisher, it uses basic reset; the memory BasicReset fixture asserts that both stale messages become acquirable again. (medium confidence) [source 1] [source 2]
- For a payload entry with an event name, recovery generates a new event ID and calls Send with that ID and the stored event name, payload and metadata. The implementation comment says the new ID permits WorkerPool to acquire fresh state; the old message ID is used for the subsequent MarkProcessed call. (low confidence) [source]
- The memory payload-republish fixture asserts a combined recovery count for one payload entry and one entry without payload, one Send with the expected event name and payload, retention of the original state as completed, and reacquisition of the reset entry. Its publisher records Send calls and returns nil; these assertions describe recovery bookkeeping. (low confidence) [source]
- The memory mixed-phase fixture supplies two stale payload entries and one entry without payload. It asserts three recovered, two publisher calls, and successful reacquisition of the entry reset in the second phase. (low confidence) [source]
- The recovery source defines a sequence of Send, MarkProcessed and ClearPayload, and explicitly allows repeat republishing across a crash before marking. The memory republish fixture checks Send records and coordinator state; the MongoDB payload fixture checks stored fields and clearing. This evidence establishes those recovery and storage boundaries, without establishing atomic completion of a sink business effect and a checkpoint. (medium confidence) [source 1] [source 2] [source 3]
Children
- No children recorded.