Atlas Stream Processing streaming materialized views
Parent: Read Offload via Caching and Precomputed Views ($merge/$out, Computed Pattern) · Published reference · snapshot 2026-10-31
↓ Facts as markdownall context files
Source-anchored research for Atlas Stream Processing streaming materialized views. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
How it works
- The pattern suits dashboards, running totals, and counts when computed results are read more frequently than source data changes. (low confidence) [source]
- Source-driven $lookup enrichment does not automatically revisit already-written output when only the reference collection changes. This scopes claims that the maintained collection always reflects every related collection. (low confidence) (verified as of 2026-10-14) [source]
Problems, failure modes and limitations
- Bean describes the target collection and processor as separately managed: other writers can change the collection, and dropping it can yield incomplete recreated results. This is a demo-author warning, not an independently verified incident. (medium confidence) (verified as of 2026-06-06) [source]
Comparisons and alternatives
- A streaming materialized view is a stored Atlas collection maintained continuously by a stream processor; applications query that collection instead of rerunning the source transformation for each read. (medium confidence) [source]
- The implementation requires every $group to run inside a window; the tutorial uses one-second tumbling windows. Treat that interval as an example configuration rather than a freshness SLA. (medium confidence) (verified as of 2026-10-14) [source]
- The documented aggregate pipeline combines a change-stream $source, event-effect computation, a window containing $group, and a $merge sink. It is an assembled pipeline pattern rather than a declaration that automatically manages the source, processor, and target together. (medium confidence) [source]
Facts and statements
- A CQRS-style deployment serves a query-shaped collection and moves repeated joins or transformations into stream processing; actual savings require workload measurement. (medium confidence) [source]
- The documented change-delta example requires source pre/post-images and requests both fullDocument and fullDocumentBeforeChange as required. (low confidence) (verified as of 2026-10-14) [source]
- Pre-existing source documents need initialization: the tutorial seeds the target with a batch aggregation before streaming. It permits initialSync as an alternative for pipelines without a window stage. (medium confidence) (verified as of 2026-10-14) [source]
- Only incremental aggregations qualify for this pattern; aggregations requiring a full collection scan do not. (low confidence) (verified as of 2026-10-14) [source]
- For read offload, precompute a query-shaped read model: the stream processor performs transformations and enrichment while application reads retrieve the prepared documents. This exchanges repeated read-time work for maintenance compute and storage. (medium confidence) [source]
- The manufacturing-mes-dashboard repository explicitly says its materialized views use batch refresh and that Atlas Stream Processing was not implemented; its benchmark figures therefore do not validate streaming-view performance. (low confidence) (verified as of 2026-10-30) [source]
- The documented windowed pattern batches event effects before updating running totals. Its window interval trades update frequency against freshness; continuous execution does not imply zero processing lag. (medium confidence) [source]
- Evaluate the pattern for repeated expensive reads, and measure maintenance cost and acceptable staleness. Microsoft independently cautions that rapidly changing sources, simple source queries, or strict consistency can make materialization unsuitable; this is architectural guidance, not an Atlas benchmark. (low confidence) [source]
Children
- Streaming-view initialization with initialSync versus batch seeding (frontier)
- Reference-collection change propagation in streaming joins (frontier)
- Streaming-view window interval and observed freshness (frontier)
- Target collection lifecycle and ownership for stream-maintained views (frontier)
- Streaming view initialization and processor lifecycle (frontier)
- Window duration and materialization freshness (frontier)
- Reference-collection enrichment freshness (frontier)
Frontier under this node: Reference-collection change propagation in streaming joins, Reference-collection enrichment freshness, Streaming view initialization and processor lifecycle, Streaming-view initialization with initialSync versus batch seeding, Streaming-view window interval and observed freshness, Target collection lifecycle and ownership for stream-maintained views, Window duration and materialization freshness