Delete propagation and reconciliation for $merge-maintained views
Parent: Read Offload via Caching and Precomputed Views ($merge/$out, Computed Pattern) · Published reference · snapshot 2026-10-31
↓ Facts as markdownall context files
Source-anchored research for Delete propagation and reconciliation for $merge-maintained views. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
How it works
- A MongoDB delete event identifies the deleted document through documentKey, including _id and the shard key for sharded collections, but omits fullDocument because the document no longer exists. Design implication: a one-document projection can use a retained source-to-target identity mapping to locate the target deletion. (medium confidence) [source]
Problems, failure modes and limitations
- Do not run broad stale-row cleanup after a failed build: $merge can retain earlier writes after an error, leaving valid target rows without the new marker. (medium confidence) [source]
- A complete rebuild using $out replaces an existing target collection atomically when the aggregation completes, so rows absent from the rebuilt results disappear. MongoDB builds a temporary collection and renames it over the destination. $out cannot target a sharded collection; $merge can. (high confidence) [source]
- For grouped summaries, a deleted document's identity alone may not identify its former group or contribution. Design inference: retain that mapping or consume an available pre-image, then recompute affected groups and explicitly remove groups whose result becomes empty. A fresh read cannot retrieve the deleted source document. (medium confidence) [source]
- A resume token permits recovery only while the oplog retains the required history. Debezium documents snapshot-based recovery when a saved position is unavailable, subject to snapshot configuration. Design inference for materialized views: after an unrecoverable event gap, rebuild or compare the complete intended key set; merely upserting surviving source rows cannot identify missed deletions. (medium confidence) [source]
Comparisons and alternatives
- Meltwater's published implementation subscribes to insert, update, replace, and delete operations and relays events through application code to downstream consumers. MongoDB documents delete as a distinct event. Design implication: a deletion propagation path must preserve delete events rather than assume every notification contains a replacement document. (high confidence) [source]
Facts and statements
- MongoDB Server $merge processes incoming aggregation results; whenMatched: replace replaces individual matched documents, not the entire target. Target rows absent from the new result can survive. (medium confidence) [source]
- whenNotMatched: "discard" discards an incoming result that has no target match. It does not delete target documents lacking incoming results. The database $merge action lists provide no document-delete action. (high confidence) [source]
- Database aggregation $merge processes emitted result documents; it does not remove target documents absent from the results. If deletion removes the last source member of a group, that group emits no replacement row and its old summary can remain indefinitely. (high confidence) [source]
- Codevra demonstrates serialized mark-and-sweep reconciliation: stamp every emitted summary with the current refresh identifier, then delete older identifiers only after a complete successful refresh. Cleanup must stay within the refreshed report scope. Failed, partial, or overlapping builds make broad cleanup unsafe without coordination. (low confidence) [source]
- Codevra demonstrates publishing a completed generation through a conditional pointer update. Absent rows disappear from the active generation; old generations need reader-aware retention, and generation tags do not freeze source reads. (low confidence) [source]
- A delete change event includes documentKey but omits fullDocument. Recovering non-key grouping information may require a retained mapping or fullDocumentBeforeChange; MongoDB requires pre-images to have been enabled and retained. (low confidence) [source]
- whenMatched: "replace" replaces individual matching documents, not the whole collection. Default "merge" combines fields and can preserve obsolete target fields that the new generated row omits. (high confidence) [source]
- Codevra demonstrates a separate-generation alternative: build rows under a new generation, publish an active-generation pointer after completion, and retain older generations for readers before garbage collection. Missing groups disappear from the active generation. A generation marker does not itself freeze source reads into a snapshot. (low confidence) [source]
- Pre-images must have been enabled before the deletion and requested through fullDocumentBeforeChange. They can expire independently before consumption, and oplog removal also removes the associated images. Design implication: pre-images are a bounded recovery dependency, so a reconciliation path must handle missing historical grouping data. (high confidence) [source]
Children
- No children recorded.