# Canary and Shadow-Traffic Comparison > Source-anchored research for Canary and Shadow-Traffic Comparison. Run status: COMPLETED. Parent: [MongoDB Performance Regression Detection and Testing Methodology](https://llms-explorer.com/tree/mongodb-performance-regression-detection-and-testing-methodology/) · 5 facets · 12 facts · page: https://llms-explorer.com/tree/canary-and-shadow-traffic-comparison/ ## Definitions - Shadow traffic is not automatically side-effect-free. A state-changing request may execute on both versions, and shared state such as a cache can alter or contaminate performance measurements. Isolate candidate data and downstream effects, or make replayed operations safe to repeat. (high confidence) — [source](https://aws.amazon.com/blogs/networking-and-content-delivery/mirror-production-traffic-to-test-environment-with-vpc-traffic-mirroring/) ## How it works - A canary routes a controlled share of production traffic to a changed version while the control serves the rest. Users in the canary slice receive candidate responses, so defects can affect that slice; teams compare candidate and control signals and can stop or roll back when the candidate regresses. (high confidence) — [source](https://medium.com/netflix-techblog/automated-canary-analysis-at-netflix-with-kayenta-3260bc7acc69) - AWS VPC Traffic Mirroring is not universally zero-impact: AWS documentation says mirrored packets count toward source-instance bandwidth and may be dropped when bandwidth or packet-rate limits are exceeded. The AWS blog's no-performance-impact statement is specific to its out-of-band architecture and should not be generalized to every mirroring setup. (medium confidence) (verified as of 2026-10-23) — [source](https://docs.aws.amazon.com/vpc/latest/mirroring/traffic-mirroring-network-limitations.html) ## Problems, failure modes and limitations - Shadow runs exercise the candidate on copied production inputs but cannot show how users respond to candidate answers because only the incumbent response reaches them. Replayed state-changing requests can duplicate effects; AWS calls out idempotency, and InfoQ recommends read-only replicas or staging data for downstream state. (high confidence) — [source](https://aws.amazon.com/blogs/networking-and-content-delivery/mirror-production-traffic-to-test-environment-with-vpc-traffic-mirroring/) ## Comparisons and alternatives - In Envoy v1.36.11, RequestMirrorPolicy mirrors all requests when runtime_fraction is unset and can restrict mirroring to a configured fraction when it is set. Sampling can limit candidate load, with the trade-off that fewer requests are measured. (low confidence) (verified as of 2026-10-01) — [source](https://www.envoyproxy.io/docs/envoy/v1.36.11/api-v3/config/route/v3/route_components.proto) ## Facts and statements - A canary routes a portion of live requests to a candidate, so some users receive candidate responses; shadow traffic copies production requests to the candidate while the incumbent serves the client and the shadow response is discarded. (high confidence) — [source](https://medium.com/netflix-techblog/automated-canary-analysis-at-netflix-with-kayenta-3260bc7acc69) - A useful canary comparison uses a baseline running the production version and configuration, created alongside the candidate; match deployment time, instance count, traffic type, and traffic amount. Comparing against long-running production can confound results through cache warmup, heap, or load-balancing differences. (high confidence) — [source](https://cloud.google.com/blog/products/devops-sre/canary-analysis-lessons-learned-and-best-practices-from-google-and-waze) - Canary analysis should compare configured baseline and candidate time-series metrics that represent service health, including latency, errors, and saturation. Netflix's Kayenta description validates and cleans metric data, classifies individual metrics, and aggregates results; it describes Mann–Whitney U confidence intervals as its primary comparison algorithm. (high confidence) (verified as of 2018-04-10) — [source](https://medium.com/netflix-techblog/automated-canary-analysis-at-netflix-with-kayenta-3260bc7acc69) - Google and Waze recommend validating a canary analysis configuration with known-good and known-bad app versions, and running an A/A check with the same version on both sides that should pass. (low confidence) — [source](https://cloud.google.com/blog/products/devops-sre/canary-analysis-lessons-learned-and-best-practices-from-google-and-waze) - Google and Waze's Spinnaker guidance recommends at least 50 time-series data points per metric for statistically relevant canary analysis; depending on monitoring granularity, this can require several hours of analysis. (low confidence) (verified as of 2019-01-14) — [source](https://cloud.google.com/blog/products/devops-sre/canary-analysis-lessons-learned-and-best-practices-from-google-and-waze) - Shadowing leaves the stable version on the client response path and sends a copy of each selected request to the candidate; the candidate response is discarded and does not delay the primary response. This exercises the candidate on real request inputs without exposing its output to users. (high confidence) — [source](https://aws.amazon.com/blogs/networking-and-content-delivery/mirror-production-traffic-to-test-environment-with-vpc-traffic-mirroring/) - Canary metrics should be broken down by candidate and control population, and their aggregation intervals should fit within the canary duration. Service-wide metrics can dilute a regression affecting a small traffic slice, while a time window longer than the canary can mix unrelated events into the comparison. (high confidence) — [source](https://sre.google/workbook/canarying-releases/)