ML Model Monitoring
Parent: Machine Learning · Published reference · snapshot 2026-09-08 · skill da-analytical-methods
↓ Facts as markdownall context files
The analytical-methods stage of the data-analysis discipline — the techniques
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Data Analysis: Analytical Methods
- The analytical-methods stage of the data-analysis discipline - the techniques that turn a prepared dataset into findings, predictions, and decisions. This is the "do the analysis" layer that sits between data acquisition and communication: cleaning and exploring data, fitting statistical and ML models, running experiments and causal estimates, forecasting, detecting anomalies, engineering features, and the specialized modeling disciplines (CLV, survival, Bayesian, conformal/UQ, causal discovery, prescriptive optimization, recommenders). [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#data-analysis-analytical-methods]
- It does not re-derive the underlying probability and inference theory (that is da-1-foundations-theory) and it does not own the surrounding process, platform, or communication stages - see the cross-hub note below. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#data-analysis-analytical-methods]
How to use this hub
- This hub consolidates 16 analytical-methods sub-skills as on-demand references under references/. Treat the routing table as an index, not as the answer: [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#how-to-use-this-hub]
- Identify which method the task calls for. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#how-to-use-this-hub]
- Find the matching row below. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#how-to-use-this-hub]
- Read the listed references/<name>.md before giving a deep answer - the table lines are deliberately shallow and exist only to route. For multi-method tasks (e.g. EDA → feature engineering → modeling → evaluation), read each relevant reference in sequence. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#how-to-use-this-hub]
Sub-skill routing table
- This hub absorbs 16 former standalone skills as on-demand reference files. When a task matches a row, Read the listed references/ file before answering - do not rely on this table alone for depth. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#sub-skill-routing-table]
Cross-hub note
- This hub is the methods stage of a six-hub data-analytics family. Route elsewhere when the task is not "run the analysis": [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#cross-hub-note]
- Theory beneath a method (distributions, Bayes' theorem, CLT, estimation theory, levels of measurement, correlation vs. causation) → da-1-foundations-theory. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#cross-hub-note]
- Process / lifecycle (CRISP-DM, problem framing, success metrics, stakeholder handoff) → da-2-data-analysis-lifecycle. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#cross-hub-note]
- Getting the data (sources, collection methods, sampling design) → da-3-data-acquisition-sampling. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#cross-hub-note]
- Pipelines and platform (ETL, warehousing, OLAP, semantic layer, governance, observability) → da-data-engineering-platform. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#cross-hub-note]
- Showing and applying the result (visualization, reporting, applied/domain analytics, ethics and privacy) → da-applied-and-communication. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#cross-hub-note]
- Writing up findings (stakeholder narratives, KB articles, runbooks, executive summaries) → technical-writing-craft (structured docs) or content-and-marketing-writing (TAM replies, customer-facing narratives). [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#cross-hub-note]
- When a request spans stages, start in the hub that owns the decision the user is currently making and hand off explicitly. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#cross-hub-note]
Cross-hub map — where every data-analytics topic lives
- This family is split across these hubs. If a task's deep material is not in this hub's Sub-skill routing table, it is a reference file under a sibling hub below - activate that hub or Read its references/<name>.md directly. Every former standalone skill in this family is now a reference under one of these hubs (nothing was deleted). [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-analytical-methods/#cross-hub-map-where-every-data-analytics-topic-lives]
Children
- Drift Taxonomy (data/covariate, concept, prediction/output, label/prior, feature drift) (frontier)
- Drift Detection Tests (PSI, KL/JS divergence, KS, Chi-square, Wasserstein/EMD, L-infinity, MMD, C2ST) (frontier)
- Sequential/Streaming Concept-Drift Detectors (DDM, EDDM, ADWIN, Page-Hinkley, CUSUM) (frontier)
- Performance Monitoring with Delayed/Absent Ground Truth (proxy metrics, two-loop monitoring, label lag) (frontier)
- Performance Estimation Without Labels (NannyML CBPE, DLE, M-CBPE) (frontier)
- Training-Serving Skew Detection (frontier)
- Slice/Segment-Based Performance Monitoring and Fairness Drift (frontier)
- Input Outlier/Adversarial Detection (Alibi Detect) (frontier)
- Alerting, Retraining Triggers, and the Monitoring->Retraining Loop (frontier)
- Model-Monitoring Tooling (Evidently, Arize, Fiddler, WhyLabs/whylogs, NannyML, Seldon/Alibi Detect, SageMaker Model Monitor, Vertex AI Model Monitoring, MLflow) (frontier)
- Model Monitoring vs Data Observability vs LLM Observability (frontier)
Frontier under this node: Alerting, Retraining Triggers, and the Monitoring->Retraining Loop, Drift Detection Tests (PSI, KL/JS divergence, KS, Chi-square, Wasserstein/EMD, L-infinity, MMD, C2ST), Drift Taxonomy (data/covariate, concept, prediction/output, label/prior, feature drift), Input Outlier/Adversarial Detection (Alibi Detect), Model Monitoring vs Data Observability vs LLM Observability, Model-Monitoring Tooling (Evidently, Arize, Fiddler, WhyLabs/whylogs, NannyML, Seldon/Alibi Detect, SageMaker Model Monitor, Vertex AI Model Monitoring, MLflow), Performance Estimation Without Labels (NannyML CBPE, DLE, M-CBPE), Performance Monitoring with Delayed/Absent Ground Truth (proxy metrics, two-loop monitoring, label lag), Sequential/Streaming Concept-Drift Detectors (DDM, EDDM, ADWIN, Page-Hinkley, CUSUM), Slice/Segment-Based Performance Monitoring and Fairness Drift, Training-Serving Skew Detection