Change Data Capture Patterns
Parent: MongoDB Atlas Triggers and Functions · Published reference · snapshot 2026-10-23 · skill mongodb-cohort-f6ac572aba6f
↓ Facts as markdownall context files
Source-anchored research for Change Data Capture Patterns. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Structure and components
- Atlas database Triggers run on a serverless compute layer that scales independently of the database server. Ordered Triggers wait to execute a Function for a particular event until the Functions of previous events have finished executing. (low confidence) [source]
How it works
- Atlas database Triggers consume MongoDB change streams and dispatch change events to Functions or AWS EventBridge; change streams expose majority-committed database changes. (high confidence) [source]
- Ordered Triggers wait for each Function execution to finish before starting the next event. Unordered Triggers run concurrently and lose execution-order guarantees, so choose ordering according to downstream consistency needs and expected processing lag. (high confidence) [source]
- Atlas Trigger Auto-Resume can restart after its resume token disappears from the oplog, but does not process missed events in the gap. Plan an explicit reconciliation or state backfill when correctness requires recovering missing changes. (low confidence) (verified as of 2026-10-14) [source]
- fullDocument with updateLookup is a lookup of current document state, not a guaranteed image at the instant of the update event. Later changes may already be present, and deletion before lookup can yield null; use event deltas or appropriately configured pre/post-images when exact change state matters. (high confidence) [source]
- Profile the CDC consumer as well as the database. Estuary's engineering write-up identifies synchronous batch fetching and BSON-to-JSON conversion as bottlenecks; its implementation used bounded prefetching and a Rust transcoder. These results describe that connector, not an Atlas Trigger throughput guarantee. (low confidence) [source]
- Atlas Database Triggers consume MongoDB change-stream events and pass matching events to a linked Function. They provide an event-driven CDC entry point for reacting to database writes. (high confidence) [source]
- Filter relevant operations and fields in the change-stream pipeline or Trigger match expression, and project only required event fields to reduce downstream work. A direct change-stream consumer must preserve the event _id resume token. (high confidence) [source]
- For state replication, bootstrap existing documents with a snapshot before continuing the live change stream. Debezium records a starting offset, emits snapshot read events, and streams from an offset; interrupted snapshots can restart, so sinks must tolerate repeated records. This describes Debezium, not an automatic Atlas Trigger backfill. (low confidence) [source]
- MongoDB supports document pre- and post-images, with collection configuration and retention requirements. Debezium can include pre-images when both MongoDB collection settings and connector capture mode enable them. (high confidence) [source]
- Ordered Atlas Triggers process events sequentially; unordered Triggers execute concurrently. Choose ordering for order-sensitive derived state, and account for replay of already processed concurrent events after suspension. (high confidence) [source]
- A low-traffic filtered CDC feed still needs checkpoint progress. Debezium heartbeat messages let offsets be committed when only excluded collections change; Atlas also maintains heartbeats to keep Trigger resume tokens fresh. These are distinct mechanisms with a shared recovery purpose. (high confidence) [source]
How-to and procedures
- Use a Trigger match expression to restrict which change events invoke work, and a projection to reduce event payload size. Filtering on updateDescription.updatedFields can avoid invoking the Function for unrelated field updates. (high confidence) [source]
Problems, failure modes and limitations
- Size oplog retention for the longest expected interruption and processing backlog. Atlas warns that slow ordered Triggers can fall behind until events leave the oplog; MongoDB requires retained history to locate a resume token. (high confidence) [source]
- Flink CDC issue #1879 reports an immediate 'resume token was not found' failure when incremental snapshots were enabled with Flink CDC 2.3.0, Flink 1.16.0 and MongoDB 4.4. This historical connector report is not evidence of a current Atlas Trigger defect. (low confidence) [source]
- resumeAfter cannot reopen a change stream after an invalidate event such as a collection drop or rename. startAfter can start a new stream after that invalidate event, provided the required oplog history remains available. (low confidence) [source]
Comparisons and alternatives
- Resume from a retained token or server position rather than opening a fresh stream after a restart. Recovery requires enough oplog history; size retention for the expected interruption and catch-up period, and monitor lag against the available history window. (high confidence) [source]
Facts and statements
- For replication and migration, bootstrap existing data with a snapshot and then consume ongoing changes. Debezium records an oplog position before copying collections and subsequently streams from its recorded offset; CDC keeps a migration target current while applications keep writing. (high confidence) [source]
- Design downstream effects to tolerate repeated events. Atlas documents at-least-once event processing and possible duplicate delivery after failures; Debezium likewise documents replay after a crash before offset persistence. Neither establishes exactly-once external side effects by itself. (high confidence) [source]
- Resume a custom CDC consumer using its saved change-stream token; Spring Data exposes getResumeToken and resume options. A token is a position bookmark, and successful historical resumption still requires the corresponding oplog history. (high confidence) [source]
- For consumers that require old document contents, enable pre-images deliberately. Atlas stores pre-images at collection level with storage and compute overhead; Debezium can use MongoDB pre-images to populate the before field. (high confidence) [source]
- MongoDB change streams require a replica set or sharded cluster. Applications can consume collection or database streams; MongoDB also supports deployment-wide streams. (high confidence) [source]
- An update fullDocument lookup can return a newer majority-committed document than the state immediately after the triggering update. Use updateDescription for the event’s delta; do not treat a lookup result as an immutable historical post-image. (high confidence) [source]
- Spring Data exposes imperative listener containers and reactive change-stream subscriptions. It supports resume tokens or server timestamps and warns that an Instant can lack sufficient resume precision; use BsonTimestamp where necessary. (low confidence) [source]
Children
- Snapshot-to-stream handoff (frontier)
- Idempotent change-event consumers (frontier)
- Oplog retention and CDC recovery (frontier)
- Change-event document images (frontier)
- Snapshot-and-stream bootstrapping (frontier)
- Resume-token checkpointing (frontier)
- Idempotent CDC consumers (frontier)
- Document pre-images and post-images (frontier)
- CDC reconciliation after history loss (frontier)
Frontier under this node: CDC reconciliation after history loss, Change-event document images, Document pre-images and post-images, Idempotent CDC consumers, Idempotent change-event consumers, Oplog retention and CDC recovery, Resume-token checkpointing, Snapshot-and-stream bootstrapping, Snapshot-to-stream handoff