ML Model Monitoring
Parent: Machine Learning · researched 2026-05-31T17:20:24.227Z· 24 sources · 11 concepts · skill da-analytical-methods
The analytical-methods stage of the data-analysis discipline — the techniques
Data Analysis: Analytical Methods
- The analytical-methods stage of the data-analysis discipline - the techniques [source]
- that turn a prepared dataset into findings, predictions, and decisions. This is [source]
- the "do the analysis" layer that sits between data acquisition and [source]
- communication: cleaning and exploring data, fitting statistical and ML models, [source]
- running experiments and causal estimates, forecasting, detecting anomalies, [source]
- engineering features, and the specialized modeling disciplines (CLV, survival, [source]
- Bayesian, conformal/UQ, causal discovery, prescriptive optimization, [source]
- It does not re-derive the underlying probability and inference theory (that is [source]
- da-1-foundations-theory) and it does not own the surrounding process, [source]
- platform, or communication stages - see the cross-hub note below. [source]
How to use this hub
- This hub consolidates **16 analytical-methods sub-skills as on-demand [source]
- references** under references/. Treat the routing table as an index, not as [source]
- Identify which method the task calls for. [source]
- Find the matching row below. [source]
- Read the listed references/<name>.md before giving a deep answer - the [source]
- table lines are deliberately shallow and exist only to route. For [source]
- multi-method tasks (e.g. EDA → feature engineering → modeling → evaluation), [source]
- read each relevant reference in sequence. [source]
Sub-skill routing table
Cross-hub note
- This hub is the methods stage of a six-hub data-analytics family. Route [source]
- elsewhere when the task is not "run the analysis": [source]
- Theory beneath a method (distributions, Bayes' theorem, CLT, estimation [source]
- theory, levels of measurement, correlation vs. causation) → [source]
- da-1-foundations-theory. [source]
- Process / lifecycle (CRISP-DM, problem framing, success metrics, [source]
- stakeholder handoff) → da-2-data-analysis-lifecycle. [source]
- Getting the data (sources, collection methods, sampling design) → [source]
- da-3-data-acquisition-sampling. [source]
- Pipelines and platform (ETL, warehousing, OLAP, semantic layer, [source]
- governance, observability) → da-data-engineering-platform. [source]
- Showing and applying the result (visualization, reporting, applied/domain [source]
- analytics, ethics and privacy) → da-applied-and-communication. [source]
- Writing up findings (stakeholder narratives, KB articles, runbooks, [source]
- executive summaries) → technical-writing-craft (structured docs) or [source]
- content-and-marketing-writing (TAM replies, customer-facing narratives). [source]
- When a request spans stages, start in the hub that owns the *decision the user [source]
- is currently making* and hand off explicitly. [source]
Cross-hub map — where every data-analytics topic lives
- This family is split across these hubs. If a task's deep material is not in this hub's Sub-skill [source]
- routing table, it is a reference file under a sibling hub below - **activate that hub or Read its [source]
- references/<name>.md directly**. Every former standalone skill in this family is now a reference under one [source]
- of these hubs (nothing was deleted). [source]
Children
- Drift Taxonomy (data/covariate, concept, prediction/output, label/prior, feature drift) (frontier)
- Drift Detection Tests (PSI, KL/JS divergence, KS, Chi-square, Wasserstein/EMD, L-infinity, MMD, C2ST) (frontier)
- Sequential/Streaming Concept-Drift Detectors (DDM, EDDM, ADWIN, Page-Hinkley, CUSUM) (frontier)
- Performance Monitoring with Delayed/Absent Ground Truth (proxy metrics, two-loop monitoring, label lag) (frontier)
- Performance Estimation Without Labels (NannyML CBPE, DLE, M-CBPE) (frontier)
- Training-Serving Skew Detection (frontier)
- Slice/Segment-Based Performance Monitoring and Fairness Drift (frontier)
- Input Outlier/Adversarial Detection (Alibi Detect) (frontier)
- Alerting, Retraining Triggers, and the Monitoring->Retraining Loop (frontier)
- Model-Monitoring Tooling (Evidently, Arize, Fiddler, WhyLabs/whylogs, NannyML, Seldon/Alibi Detect, SageMaker Model Monitor, Vertex AI Model Monitoring, MLflow) (frontier)
- Model Monitoring vs Data Observability vs LLM Observability (frontier)
Frontier under this node: Alerting, Retraining Triggers, and the Monitoring->Retraining Loop, Drift Detection Tests (PSI, KL/JS divergence, KS, Chi-square, Wasserstein/EMD, L-infinity, MMD, C2ST), Drift Taxonomy (data/covariate, concept, prediction/output, label/prior, feature drift), Input Outlier/Adversarial Detection (Alibi Detect), Model Monitoring vs Data Observability vs LLM Observability, Model-Monitoring Tooling (Evidently, Arize, Fiddler, WhyLabs/whylogs, NannyML, Seldon/Alibi Detect, SageMaker Model Monitor, Vertex AI Model Monitoring, MLflow), Performance Estimation Without Labels (NannyML CBPE, DLE, M-CBPE), Performance Monitoring with Delayed/Absent Ground Truth (proxy metrics, two-loop monitoring, label lag), Sequential/Streaming Concept-Drift Detectors (DDM, EDDM, ADWIN, Page-Hinkley, CUSUM), Slice/Segment-Based Performance Monitoring and Fairness Drift, Training-Serving Skew Detection