Change-stream point-in-time images and retention lifecycle
Parent: MongoDB Replication · Published reference · snapshot 2026-10-27
↓ Facts as markdownall context files
Source-anchored research for Change-stream point-in-time images and retention lifecycle. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
How it works
- A background process removes expired pre-images from config.system.preimages asynchronously. Expiration makes images eligible for cleanup; it is not a promise that physical deletion happens at the exact expiry second. (high confidence) [source]
- Pre-images occupy config.system.preimages, and a background process removes them asynchronously. Enabling image capture consumes additional storage and processing time. (medium confidence) [source]
- Treat lookup-derived before/after comparisons cautiously: rapid updates can make an event delta and its looked-up full document represent different operations. Prefer point-in-time post-images when exact per-update document states are required. (medium confidence) [source]
Problems, failure modes and limitations
- Image recording must be enabled on each collection with changeStreamPreAndPostImages before the operation occurs. create, db.createCollection(), and collMod support enabling it; enabling it later cannot supply images for earlier operations. (high confidence) [source]
- Request pre-images with fullDocumentBeforeChange and post-images with fullDocument. For each option, whenAvailable permits null for an unavailable image, whereas required terminates the stream with an error if the required image is missing. (low confidence) [source]
- Enabling images adds storage and processing work. Including full images also enlarges change events; Airbyte and Debezium warn that large events can exceed the 16 MiB BSON limit and fail. Image capture therefore requires both retention and event-size planning. (high confidence) [source]
- Request pre-images with fullDocumentBeforeChange. For both image options, "required" raises an error when the requested image is unavailable, whereas "whenAvailable" returns the image only if available; fullDocumentBeforeChange defaults to "off". (medium confidence) [source]
- Removing an event from the oplog also makes its corresponding images subject to deletion regardless of a longer expireAfterSeconds setting. An image retention setting cannot extend image history beyond the associated oplog history. (medium confidence) [source]
- Returning full images increases event size. Large documents combined with image fields and event metadata can exceed the 16 MiB BSON event limit, including when a lookup returns the full document. (medium confidence) [source]
Comparisons and alternatives
- fullDocument: updateLookup reads the current majority-committed document rather than guaranteeing the event-time document. Intervening updates can change its contents; a later deletion can make the lookup return null. Airbyte documents that multiple events can consequently contain the same final state. (high confidence) [source]
- For update events, fullDocument: "whenAvailable" or "required" requests the point-in-time post-image. This supports consumers that need intermediate document states instead of the latest document version returned by lookup. (medium confidence) [source]
Facts and statements
- Pre-images represent documents before update, replacement, or deletion; inserts have no pre-image. Post-images represent documents after insert, replacement, or update; deletes have no post-image. (high confidence) [source]
- For update events, MongoDB's implementation reconstructs point-in-time post-images from the stored pre-image and the event's update delta. It does not need to store a second full post-image for each update. (low confidence) [source]
- changeStreamOptions.preAndPostImages.expireAfterSeconds controls image retention cluster-wide. Its default off ties retention to the corresponding oplog events; off does not mean indefinite retention. Oplog eviction expires associated images regardless of a longer configured image-retention interval. (high confidence) [source]
- Budget image retention for consumer lag and recovery time, and monitor oplog history separately. An event can remain readable after its required image expires, so event availability alone does not guarantee image availability. (high confidence) [source]
- The historical C# driver issue CSHARP-4801 reproduced an expired pre-image causing FullDocumentBeforeChange access to cast BsonNull to BsonDocument and throw. The issue records fix version 2.22.0; it is evidence of a historical driver null-handling bug, not a current server retention defect. (low confidence) [source]
- MongoDB 6.0 introduced configurable change-stream pre- and post-images. A pre-image describes the document before an update, replacement, or deletion; a post-image describes it after insertion, replacement, or update. Inserts have no pre-image and deletes have no post-image. (medium confidence) [source]
- changeStreamOptions.preAndPostImages.expireAfterSeconds controls image retention cluster-wide. Its default "off" ties retention to the corresponding oplog events; it does not mean indefinite archival retention. (medium confidence) [source]
Children
- Optional versus required image availability (frontier)
- Pre-image storage and asynchronous expiration (frontier)
- Lookup races versus point-in-time post-image reconstruction (frontier)
- Consumer lag and image retention budgeting (frontier)
Frontier under this node: Consumer lag and image retention budgeting, Lookup races versus point-in-time post-image reconstruction, Optional versus required image availability, Pre-image storage and asynchronous expiration