# Augmented Analytics and LLM-Assisted Analysis > How AI augments or automates the analytical loop — preparing data, finding insights, answering natural-language questions, and explaining results — instead of a human writing every query and reading e Parent: [Data Analysis](https://llms-explorer.com/tree/data-analysis/) · 18 facets · 93 facts · page: https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/ ## Augmented Analytics & LLM-Assisted Analysis - How AI augments or automates the analytical loop - preparing data, finding insights, answering natural-language questions, and explaining results - instead of a human writing every query and reading every chart. The audience is an analyst, data/BI engineer, or product owner deciding whether and how to put an LLM between users and data, and how to keep the answers correct. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#augmented-analytics-llm-assisted-analysis) - Scope boundary. This skill is about the AI/LLM layer over analytics. It deliberately does not re-teach: — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#augmented-analytics-llm-assisted-analysis) - The semantic/metrics layer itself → da-18-semantic-layer-headless-bi. Here we only cover how an LLM is grounded in that layer. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#augmented-analytics-llm-assisted-analysis) - Unstructured-text NLP (topic modeling, sentiment, NER, text embeddings) → da-36-text-analytics-nlp. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#augmented-analytics-llm-assisted-analysis) - Generic ML / LLM training, fine-tuning, eval theory → da-7-machine-learning. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#augmented-analytics-llm-assisted-analysis) - Generic RAG pipeline architecture with no numeric/analytical angle → rag-architecture. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#augmented-analytics-llm-assisted-analysis) ## 1. Augmented analytics → agentic analytics (the Gartner arc) - Gartner (Rita Sallam, 2017) defined augmented analytics as using ML/AI to assist data preparation, insight generation, and insight explanation to augment how people explore and analyze data. Gartner's evaluation criteria span six capabilities: ML-assisted insight discovery, NLP/NLQ querying, automated explanations (NLG), data-prep assistance, GenAI integration, and augmented data science. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#1-augmented-analytics-agentic-analytics-the-gartner-arc) - The 2025 evolution is agentic analytics (Gartner Market Guide for Agentic Analytics, Feb 2025): AI agents that don't just assist but autonomously plan, investigate, and act. Gartner predicts ~75% of analytics content will use GenAI for contextual intelligence by 2027, evolving toward "autonomous analytics" managing a slice of business processes. The four classic analytics tiers map onto this: descriptive (what happened) → diagnostic (why) → predictive (what will) → prescriptive (what to do) - augmented/agentic analytics automates the first two and increasingly drives toward the latter two. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#1-augmented-analytics-agentic-analytics-the-gartner-arc) ## 2. Conversational BI / NLQ - Natural-Language Query (NLQ) turns a plain-English question into a query against governed data, returns a result, a chart, and (via NLG) a written explanation. The non-negotiable lesson of 2024-2026: accuracy depends on grounding the LLM in a governed semantic model, not the raw schema. The semantic layer (see da-18) supplies business term → table/column/metric mappings, relationships, synonyms, and metric definitions so "revenue" always means the same SQL. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#2-conversational-bi-nlq) - Snowflake Cortex Analyst is built around a YAML semantic model (now Semantic Views as the recommended form) plus a Verified Query Repository of approved question→SQL pairs that the model references at generation time. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#2-conversational-bi-nlq) - Guardrails: restrict to a curated model/views, prefer verified queries, validate generated SQL, constrain output (e.g., function-calling / JSON-schema-constrained SQL), enforce row/column security so the agent inherits the user's permissions. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#2-conversational-bi-nlq) ## 3. Text-to-SQL - The research workhorse of LLM-assisted analytics. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#3-text-to-sql) - Schema linking - selecting the relevant tables/columns for a question - is the dominant accuracy lever, especially on large/enterprise schemas. 2025 SOTA approaches use context-aware bidirectional retrieval and autonomous schema exploration (e.g., AutoLink reports ~97% strict linking recall on BIRD-dev, ~91% on Spider 2.0-Lite). Hybrid dense-vector + symbolic schema retrieval (Semantic-RAG, CSR-RAG) scales linking to enterprise schemas. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#3-text-to-sql) - Prompt patterns: provide schema (DDL), few-shot question→SQL exemplars, value/sample hints, and dialect notes; decompose complex questions; use RAG to retrieve schema fragments + similar verified queries. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#3-text-to-sql) - Correctness & self-correction: never trust SQL on syntax. Use execution-guided self-correction - run the SQL (or a dry-run/EXPLAIN), feed errors/empty-results back, and let the model repair (e.g., LitE-SQL: 72.1% EX on BIRD, 88.45% on Spider 1.0 via execution-guided correction without multi-candidate sampling). Majority-vote / consensus over candidates (ReFoRCE) filters unreliable outputs. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#3-text-to-sql) - Benchmarks & metrics: Spider (cross-domain) and BIRD (large, dirty, real-world DBs with efficiency scoring) are the classics; Spider 2.0 targets enterprise workflows (huge schemas, dialects, nested query plans) and is hard - frontier execution accuracy sits in the ~25-35% range vs. ~70%+ on BIRD-dev. Primary metrics: Execution Accuracy (EX) - does the result match the gold result - and the stricter, brittler Exact-Match (EM) on SQL text. Prefer EX; note that benchmark annotation errors are a known caveat (CIDR 2026 "Text-to-SQL Benchmarks are Broken"). — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#3-text-to-sql) ## 4. Analytics agents (plan → query → analyze → narrate) - An analytics agent decomposes a goal into steps, calls tools (SQL, search, a sandboxed Python code interpreter), executes, reflects, and synthesizes a narrative. Pattern: select → aggregate → rank → explain, presented in plain language. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#4-analytics-agents-plan-query-analyze-narrate) - Code interpreter / sandboxed Python: the agent writes and runs Python (pandas/plots) in an isolated sandbox to do analysis beyond SQL (stats, joins across sources, charts). Managed sandboxes (e.g., Amazon Bedrock AgentCore Code Interpreter) handle isolation/scaling; ReAct-style loops (LangGraph) drive write→execute→observe. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#4-analytics-agents-plan-query-analyze-narrate) - Multi-agent specialization: planner / builder / critic / reflector agents make the final narrative more reliable (e.g., CoDA for collaborative visualization). — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#4-analytics-agents-plan-query-analyze-narrate) - Engineering guardrails: bound the number of tool calls and self-correction iterations, sandbox all code, scope DB credentials to the requesting user, and log every step for replay. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#4-analytics-agents-plan-query-analyze-narrate) ## 5. Automated insight generation & NLG narratives - Diagnostic automation answers "what changed and why." Key-driver analysis decomposes a metric movement into contributing dimensions/segments (often shown as a waterfall), automatically ranking drivers. Anomaly detection surfaces unexpected movements proactively (Tableau Pulse's model). — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#5-automated-insight-generation-nlg-narratives) - Significance-aware mining: only narrate insights that are statistically meaningful - guard against spurious "drivers" from multiple comparisons / small segments (ties to da-12 multiple-comparison discipline). — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#5-automated-insight-generation-nlg-narratives) - NLG converts the result into a human-readable descriptive/diagnostic/prescriptive narrative attached to a chart or dashboard, making insight portable and actionable (heavy adoption in finance reporting). — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#5-automated-insight-generation-nlg-narratives) ## 6. RAG over structured + unstructured analytical context (hybrid retrieval) - Analytical questions often need both numbers (in tables/warehouse) and context (in docs/metric definitions). Hybrid retrieval combines vector + keyword + metadata filtering: — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#6-rag-over-structured-unstructured-analytical-context-hybrid-retrieval) - Route by question type: structured/aggregate questions → text-to-SQL against governed data (do not answer "what was Q3 revenue" from a vector store); definitional/context questions → vector retrieval over docs. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#6-rag-over-structured-unstructured-analytical-context-hybrid-retrieval) - RAG-to-SQL: retrieve schema fragments, FK relationships, column descriptions, and similar verified queries to improve schema linking and grounding (Semantic-RAG, CSR-RAG - ~80%+ recall at ~30ms on commodity hardware). — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#6-rag-over-structured-unstructured-analytical-context-hybrid-retrieval) - Agentic RAG over long text in SQL tables handles documents stored alongside structured columns. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#6-rag-over-structured-unstructured-analytical-context-hybrid-retrieval) ## 7. Evaluation, trust & governance - LLM-produced numbers are a correctness problem, not just a fluency one. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#7-evaluation-trust-governance) - Hallucination types: faithfulness (output not grounded in the retrieved context/query result) vs factuality (wrong vs the real world). For analytics, faithfulness to the executed query result is the key bar - every number should trace to a query. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#7-evaluation-trust-governance) - Controls: execution-guided validation, verified-query repositories, self-consistency / consensus decoding, RAG grounding, span-level attribution (claim → source/query), and SelfCheckGPT-style inter-sample contradiction checks. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#7-evaluation-trust-governance) - Metric consistency: route metrics through the governed semantic layer so the same business term yields the same SQL every time (the anti-"metric sprawl" argument from da-18). — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#7-evaluation-trust-governance) - Human-in-the-loop & auditability: keep humans verifying high-stakes answers; expose the generated SQL/query and lineage so analysts can audit how a number was produced; integrate agent observability (distributed tracing, span-level evaluators) to run quality checks on live traffic. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#7-evaluation-trust-governance) ## Tooling landscape (2025-2026) - Choosing: if your data already lives in Snowflake → Cortex Analyst; Databricks lakehouse → Genie; Power BI/Microsoft → Copilot; want warehouse-independent NLQ → ThoughtSpot; building it yourself / embedding → Vanna or a LangChain/LlamaIndex SQL agent. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#tooling-landscape-2025-2026) ## Practical patterns - Ground before you generate. Put a governed semantic model / verified queries between the LLM and the warehouse. Raw-schema text-to-SQL is a demo, not a product. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#practical-patterns) - Execute to validate. Run (or EXPLAIN/dry-run) generated SQL, feed errors back, and self-correct. Prefer execution accuracy over trusting the text. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#practical-patterns) - Seed a verified-query repository. Curated question→SQL pairs are the single highest-ROI accuracy lever and double as regression tests. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#practical-patterns) - Route by question type. Aggregate/metric questions → SQL; definitional/context → vector RAG; complex/multi-source → agent with code interpreter. Don't answer numeric questions from a vector store. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#practical-patterns) - Bound the agent. Cap tool calls + self-correction loops, sandbox all code, scope credentials to the user, and log every step for replay/audit. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#practical-patterns) - Narrate only significant insights. Significance-test before NLG; rank drivers; show the waterfall and the supporting query. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#practical-patterns) - Make every number auditable. Surface the generated query + lineage; keep humans in the loop for high-stakes answers. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#practical-patterns) - Build an eval set. Golden NL→SQL→result triples; track EX (not just EM), faithfulness, and metric-consistency over time. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#practical-patterns) ## Anti-patterns - Pointing text-to-SQL at a raw, ungoverned schema and expecting reliable answers - accuracy collapses without semantic grounding. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#anti-patterns) - Trusting SQL because it parses - syntactically valid SQL can return wrong or empty results; always execute-validate. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#anti-patterns) - Answering numeric questions via vector RAG - vector similarity does not aggregate; route to SQL. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#anti-patterns) - Narrating "drivers" without significance testing - manufactures spurious explanations from noise / multiple comparisons. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#anti-patterns) - Unbounded agent loops or unsandboxed code execution - cost blowups and security holes. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#anti-patterns) - Numbers with no audit trail - if you can't show the query and lineage behind a figure, it isn't trustworthy for decisions. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#anti-patterns) - Optimizing for Exact-Match - EM is brittle (many correct SQLs differ textually); optimize Execution Accuracy. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#anti-patterns) - One metric, many definitions - bypassing the semantic layer reintroduces metric sprawl. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#anti-patterns) ## Troubleshooting - Wrong/empty results despite "good" SQL → schema-linking failure; add column descriptions, synonyms, sample values, and verified-query exemplars; check FK/join paths. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#troubleshooting) - Inconsistent numbers for the same question → route through the governed semantic layer; pin metric definitions; add the pair to the verified repository. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#troubleshooting) - Agent loops or runs up cost → cap iterations/tool calls; add a termination check; cache schema retrieval. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#troubleshooting) - Plausible-but-wrong narrative → faithfulness failure; require claims to cite the executed query result; add self-consistency / SelfCheckGPT-style checks; insert HITL for high stakes. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#troubleshooting) - Great on BIRD, fails in prod → enterprise schemas (Spider 2.0 regime) are far harder; invest in schema linking, dialect handling, and retrieval over the real catalog. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#troubleshooting) ## References - Gartner, Augmented Analytics glossary & Market Guide for Agentic Analytics (2025) - https://www.gartner.com/en/information-technology/glossary/augmented-analytics ; https://www.gartner.com/en/newsroom/press-releases/2025-06-18-gartner-predicts-75-percent-of-analytics-content-to-use-genai-for-enhanced-contextual-intelligence-by-2027 — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - Spider 2.0: Evaluating Language Models on Enterprise Text-to-SQL - https://openreview.net/pdf/a580c1b9fa846501c4bbf06e874bca1e2f3bc1d0.pdf — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - AutoLink: Autonomous Schema Exploration for Scalable Schema Linking (2025) - https://arxiv.org/pdf/2511.17190 — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - RSL-SQL: Robust Schema Linking in Text-to-SQL - https://arxiv.org/pdf/2411.00073 — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - LitE-SQL: Lightweight Text-to-SQL with Execution-Guided Self-Correction - https://arxiv.org/pdf/2510.09014 — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - ReFoRCE: A Text-to-SQL Agent - https://arxiv.org/pdf/2502.00675 — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - Text-to-SQL Benchmarks are Broken (CIDR 2026) - https://www.vldb.org/cidrdb/papers/2026/p5-jin.pdf — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - Semantic-RAG for Text-to-SQL - https://medium.com/@lbirjega/semantic-rag-for-text-to-sql-ed57fcdb0a45 ; CSR-RAG - https://arxiv.org/pdf/2601.06564 — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - RAGFlow, From RAG to Context - 2025 year-end review - https://ragflow.io/blog/rag-review-2025-from-rag-to-context — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - A review of faithfulness metrics for hallucination assessment in LLMs - https://arxiv.org/pdf/2501.00269 ; Faithfulness metric fusion - https://arxiv.org/pdf/2512.05700 — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - Snowflake, Cortex Analyst docs + Agentic Semantic Model Improvement - https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-analyst ; https://www.snowflake.com/en/blog/engineering/agentic-semantic-model-text-to-sql/ — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - Databricks AI/BI Genie - https://zenlytic.com/blog/databricks-ai-bi-genie — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - Tellius, Best AI Data Analysis Agents 2026 - https://www.tellius.com/resources/blog/best-ai-data-analysis-agents-in-2026-12-platforms-compared-for-nl-to-sql-autonomous-investigation-and-governance — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - Simon Willison, Coding agents for data analysis (NICAR 2026) - https://simonw.github.io/nicar-2026-coding-agents/coding-agents.html — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - AWS, Amazon Bedrock AgentCore Code Interpreter - https://aws.amazon.com/blogs/machine-learning/introducing-the-amazon-bedrock-agentcore-code-interpreter/ — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - Vanna AI (RAG-powered text-to-SQL) - https://medium.com/mitb-for-all/text-to-sql-just-got-easier-meet-vanna-ai-your-rag-powered-sql-sidekick-e781c3ffb2c5 — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) - Diagnostic Analytics / key-driver - https://www.lumi-ai.com/analytics-101/diagnostic-analytics ; NLG for BI - https://automatedinsights.com/business-intelligence/ — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#references) ## Related skills - da-18-semantic-layer-headless-bi - the governed semantic/metrics layer LLMs are grounded in. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#related-skills) - da-36-text-analytics-nlp - unstructured-text NLP (topics, sentiment, NER, embeddings). — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#related-skills) - da-7-machine-learning - LLM landscape, training, and eval theory. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#related-skills) - rag-architecture - generic RAG pipeline design. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#related-skills) - da-8-data-visualization / da-9-reporting-communication - viz recommendation and reporting. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#related-skills) - da-12-ab-testing-causal-inference - significance discipline behind driver/diagnostic claims. — [source](https://llms-explorer.com/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted/#related-skills) ## Where this helps - Building a natural-language-query feature over a governed data warehouse, where accuracy depends on grounding the model in a curated semantic model rather than pointing it at a raw schema. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Deciding when a question should route to text-to-SQL (aggregate/metric questions against governed data) versus vector RAG (definitional/context questions from docs) versus a full analytics agent with a code interpreter. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Choosing a platform-native NLQ tool based on where the data already lives — Snowflake Cortex Analyst, Databricks Genie, Power BI Copilot, or a warehouse-independent option like ThoughtSpot or Vanna. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Building trust into an LLM-generated numeric answer by exposing the executed query and its lineage, so an analyst can audit exactly how a figure was produced. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* ## Project ideas - Build a verified-query repository — curated question-to-SQL pairs — as the accuracy backbone of a text-to-SQL feature, since this pack calls it the single highest-ROI accuracy lever and it doubles as a regression test suite. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Implement execution-guided self-correction for a text-to-SQL pipeline: run the generated SQL (or an EXPLAIN/dry-run), feed any error or empty result back to the model, and let it repair before returning an answer. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Build an analytics agent that decomposes a goal into select → aggregate → rank → explain steps, executes each via SQL or a sandboxed Python code interpreter, and narrates only statistically significant findings. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Route analytical questions by type at the application layer — SQL for aggregate/metric questions, vector RAG for definitional/context questions, and an agent with a code interpreter for complex multi-source questions — rather than sending everything through one path. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* ## Common mistakes - Pointing text-to-SQL directly at a raw, ungoverned schema and expecting reliable answers — accuracy collapses without a semantic layer between the model and the warehouse. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Answering a numeric/aggregate question with vector RAG instead of SQL — vector similarity search does not aggregate, and this pattern reliably produces wrong numbers. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Narrating a 'driver' or trend without significance testing first, manufacturing a spurious explanation out of noise or a multiple-comparisons artifact. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Optimizing a text-to-SQL system for Exact Match instead of Execution Accuracy — EM is brittle because many textually different SQL statements are equally correct. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* ## Known issues - Great performance on BIRD or Spider doesn't transfer to production — enterprise schemas (the Spider 2.0 regime) are far harder, and schema-linking failures are the dominant cause of wrong or empty results in real deployments. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Faithfulness (is the output grounded in the retrieved context/query result) and factuality (is it correct against the real world) are separate failure modes, and for analytics specifically, faithfulness to the executed query result is the bar that matters most. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - Bypassing the governed semantic layer reintroduces metric sprawl — the same business term can silently generate different SQL and different numbers depending on which path answered the question. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* - An unbounded agent loop or unsandboxed code-interpreter execution is both a cost-blowup risk and a security hole, so tool-call and self-correction iteration caps aren't optional in a production analytics agent. — [source](https://llms-explorer.com/tree/augmented-analytics-and-llm-assisted-analysis/) *(AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.)* ## Context files - [Augmented Analytics and LLM-Assisted Analysis](https://llms-explorer.com/downloads/sources/mdb-context-hub/da-39-augmented-analytics-llm-assisted.md)