Data Analysis Foundations and Theory
Parent: Data Analysis · Published reference · snapshot 2026-09-08 · skill da-1-foundations-theory
↓ Facts as markdownall context files
Conceptual grounding for a data analysis effort. This skill answers the
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Data Analysis: Foundations & Theory
- Conceptual grounding for a data analysis effort. This skill answers the before-you-pick-a-tool questions: what is data analysis, how does it relate to neighboring fields, what kind of analysis is called for, what can the data support given how it was measured, and what assumptions ride underneath. It does not execute techniques - it scopes and frames them. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#data-analysis-foundations-theory]
Sub-skill routing table
- This hub consolidates 37 foundations sub-skills as on-demand references - match the task to the table and Read the listed references/<name>.md before answering deep questions. The overview below is enough for framing and scoping; load the reference when a question needs depth. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#sub-skill-routing-table]
1. What data analysis is (and its scope)
- Data analysis is the systematic process of inspecting, cleaning, transforming, and interpreting data to extract useful information, support conclusions, and aid decision-making. In practice it is bounded and goal-directed. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#1-what-data-analysis-is-and-its-scope]
- Distinguish four neighboring terms - they overlap but are not synonyms: [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#1-what-data-analysis-is-and-its-scope]
- Data analysis - the act of evaluating data to answer a defined question. The process verb. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#1-what-data-analysis-is-and-its-scope]
- Data analytics - the broader practice/field built around analyzing data; typically about the past and present, with more elementary statistics. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#1-what-data-analysis-is-and-its-scope]
- Data science - a wider, multidisciplinary field that includes analytics but reaches into ML, forecasting, and large-scale data engineering; more oriented toward predictive models. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#1-what-data-analysis-is-and-its-scope]
- Statistics - the mathematical discipline of collecting, describing, and drawing inferences from data under uncertainty. Data analysis uses statistics as a toolkit. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#1-what-data-analysis-is-and-its-scope]
- Rule of thumb: data analysis is the activity; analytics is the field around it; data science extends it toward modeling and engineering; statistics supplies the inferential mathematics. State which definition you are using. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#1-what-data-analysis-is-and-its-scope]
Analysis vs. synthesis
- Analysis breaks a whole into parts; synthesis recombines parts into a new integrated whole or recommendation. Name which mode you are in to avoid presenting raw decomposition as a conclusion. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#analysis-vs-synthesis]
Quantitative vs. qualitative
- Quantitative analysis operates on numeric measurements (interval and ratio scales). [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#quantitative-vs-qualitative]
- Qualitative analysis operates on categorical/textual data (nominal and ordinal scales). The split is about method too, not just data type. Many efforts are mixed-method. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#quantitative-vs-qualitative]
2. The four families of analysis
- The four form a maturity progression but are not strictly sequential per project. "Diagnostic" maps loosely onto exploratory work, but don't conflate the marketing taxonomy with Tukey's exploratory/confirmatory split. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#2-the-four-families-of-analysis]
3. The analysis lifecycle / process
- The de facto reference is CRISP-DM (six phases you can revisit): [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#3-the-analysis-lifecycle-process]
- Business Understanding - define the question and success criteria. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#3-the-analysis-lifecycle-process]
- Data Understanding - collect, describe, explore, verify quality. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#3-the-analysis-lifecycle-process]
- Data Preparation - select, clean, construct, integrate, format. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#3-the-analysis-lifecycle-process]
- Modeling - choose technique, build, assess. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#3-the-analysis-lifecycle-process]
- Evaluation - check against business goal; review process. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#3-the-analysis-lifecycle-process]
- Deployment - deliver, monitor, report. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#3-the-analysis-lifecycle-process]
- The phases are iterative, not a one-way pipeline. A lighter generic framing - define → collect → clean → analyze → interpret → communicate - works for non-mining work. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#3-the-analysis-lifecycle-process]
4. Exploratory vs. confirmatory analysis
- Exploratory Data Analysis (EDA) - Tukey's approach for summarizing a dataset's main characteristics, often with graphics, to generate hypotheses and check assumptions. Techniques: box plots, stem-and-leaf, histograms, scatter plots. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#4-exploratory-vs-confirmatory-analysis]
- Confirmatory Data Analysis (CDA) - classical hypothesis testing: pick a model before examining the data, then assess inference precision. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#4-exploratory-vs-confirmatory-analysis]
5. Measurement theory and levels of measurement
- Stevens' four levels (1946): [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#5-measurement-theory-and-levels-of-measurement]
- Nominal and ordinal are categorical/qualitative; interval and ratio are continuous/quantitative. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#5-measurement-theory-and-levels-of-measurement]
- Common pitfalls: computing a mean of ordinal codes (median is safer); treating an arbitrary numeric label as quantitative; forgetting interval scales lack a true zero (ratios are meaningless). Know the controversy: Velleman & Wilkinson (1993) and Luce (1997) contested Stevens' typology. Treat the level of measurement as a useful first filter, not an iron law. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#5-measurement-theory-and-levels-of-measurement]
6. The role of theory and assumptions
- Data does not interpret itself. Every analysis rides on assumptions: representative sample, measurements meaning what labels claim, model preconditions holding. Two practices: [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#6-the-role-of-theory-and-assumptions]
- State assumptions explicitly and tie each to the analysis family and measurement level. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#6-the-role-of-theory-and-assumptions]
- Match method to question and to data. Misalignment - a prescriptive recommendation on descriptive data, or a mean on ordinal categories - is the most common foundational error. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#6-the-role-of-theory-and-assumptions]
Quick decision checklist
- Term check - analysis, analytics, data science, or statistics? State the definition. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#quick-decision-checklist]
- Family - descriptive, diagnostic, predictive, or prescriptive? [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#quick-decision-checklist]
- Mode - exploratory or confirmatory? Don't mix on the same data. [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#quick-decision-checklist]
- Lifecycle - which CRISP-DM phase; what's next? [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#quick-decision-checklist]
- Measurement - what level is each variable; which statistics are licensed? [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#quick-decision-checklist]
- Assumptions - what must be true; have I checked? [archived source: https://llms-explorer.com/sources/mdb-context-hub/da-1-foundations-theory/#quick-decision-checklist]
Where this helps
- Framing a new analysis request before picking any tool, to establish which of descriptive, diagnostic, predictive, or prescriptive work is actually being asked for. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Resolving a disagreement about scope or terminology on a team by grounding the conversation in the actual definitional distinctions between analysis, analytics, data science, and statistics. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Choosing which statistics are valid to compute on a given variable, based on whether it's nominal, ordinal, interval, or ratio measured. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Deciding whether an exploratory finding needs a separate confirmatory pass on fresh data before it's presented as a validated conclusion. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
How to apply this
- Before starting any analysis, explicitly state which of the four terms you're using — data analysis, analytics, data science, or statistics — since they overlap but answer different scoped questions. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Classify your effort as descriptive, diagnostic, predictive, or prescriptive up front, and check that the technique you're reaching for actually matches that family. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Decide explicitly whether you're in exploratory or confirmatory mode before touching the data, and if both are needed, reserve a holdout or fresh sample for the confirmatory step. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Identify the measurement level of each variable before choosing a statistic, since the level licenses which statistics are meaningful to compute on it. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Antipatterns
- Running exploratory and confirmatory analysis on the same dataset without a holdout, a classic double-dipping pattern that introduces systematic bias into the confirmatory result. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Computing the mean of ordinal-coded categories, like a 1-5 rating scale, as if it were an interval-scale quantity, when the median is the statistic actually licensed by that measurement level. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Presenting a raw decomposition of data as if it were already a synthesized conclusion or recommendation, conflating the analysis step with the synthesis step that has to follow it. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Applying a prescriptive-style recommendation on top of purely descriptive data, without first doing the diagnostic or predictive work that would actually justify prescribing an action. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Limitations
- The four-family taxonomy of descriptive, diagnostic, predictive, and prescriptive is described as a maturity progression, but the pack itself notes it isn't strictly sequential per project. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Stevens' four levels of measurement (1946) are foundational but contested in places — the pack flags that a controversy exists over their strict application, without fully resolving it. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- This material deliberately stays at the conceptual, before-you-pick-a-tool level; deep technique-level guidance for any one family or lifecycle phase lives in separate, dedicated sub-skills. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
- Data analysis, analytics, data science, and statistics are treated here as a rule-of-thumb distinction rather than a universally agreed taxonomy, and different organizations use the terms with real variation. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.