Declarative and Programmatic LLM Frameworks

Parent: AI Agent Ecosystems · Published reference · snapshot 2026-09-08 · skill ai-agent-engineering

Also known as: Declarative & Programmatic LLM Frameworks ("Prompt-as-Program", 2024-2026)

A class of frameworks that emerged in 2023-2024 and matured through 2026 reframes prompting as a software-engineering discipline rather than string authoring. The unifying thesis: don't write prompt s

These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.

Overview

A class of frameworks that emerged in 2023-2024 and matured through 2026 reframes prompting as a software-engineering discipline rather than string authoring. The unifying thesis: don't write prompt strings by hand - declare the interface (inputs, outputs, types, intent) and let a compiler, type system, or constrained decoder produce and enforce the actual prompt and output. The slogan that captures the paradigm is DSPy's "programming, not prompting" (DSPy), and the academic framing is "compiling declarative language model calls into self-improving pipelines" (DSPy paper, arXiv 2310.03714). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#overview]
The motivation is consistent across the literature: hard-coded prompt templates are "long strings of instructions crafted through manual trial and error - an approach that can be brittle and unscalable," and changing the model or task forces manual rewrites of fragile strings that "break in ways that are hard to debug" (Towards Data Science, DSPy paper). Treating prompts as compiled or generated artifacts unlocks four properties hand-written strings lack: automatic optimization, portability across models, testability, and maintainability. Academic work confirms the practice is real and immature: developers genuinely "build software containing prompts" and treat "prompts as programs," but lack the tooling, testing, and debugging norms of traditional code (Prompts Are Programs Too!, arXiv 2409.12447). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#overview]
The space splits into three loosely-coupled families that are often combined: [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#overview]
- Compiled / optimized prompting - DSPy. You declare signatures + modules; an optimizer compiles them into prompts using training data + a metric. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#overview]
- Schema-first / typed LLM functions - BAML, Instructor, Pydantic AI, Marvin, Mirascope, Ell. A prompt is a typed function: declare the return type, get validated structured output (plus retries). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#overview]
- Constrained generation / query languages - Outlines, Guidance, LMQL. Enforce output structure at the token / decoding level via regex, grammar (CFG), or FSM, so invalid output is impossible rather than retried. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#overview]
The boundary with app-level "structured output" (JSON mode, function-calling-as-extraction) is porous: native provider Structured Outputs are essentially constrained decoding exposed as an API, and most typed-function libraries route to it when available (Agenta, BuildMVPFast). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#overview]

1. Prompt-as-program / "compile, don't write"

The central abstraction is the separation of program logic from prompt strings. You specify what a transformation does, not how to phrase it; a compiler/optimizer derives the phrasing. This is explicitly analogized to the move from assembly to higher-level languages with compilers: "you write source code describing what you want and the compiler figures out how to express it" (MyEngineeringPath, TesterStories). Because the prompt is a compiled artifact, it can be regenerated when the model or task changes instead of hand-patched. Related research streams include PDL (a declarative prompt-programming YAML language from IBM, arXiv 2410.19135) and Prompt Decorators (a declarative, composable syntax for reasoning/formatting control, arXiv 2510.19850) - evidence the paradigm is broader than any single tool. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#1-prompt-as-program-compile-dont-write]

2. Signatures (declarative I/O specs)

A signature is a "natural-language typed function declaration" - a concise spec of a text transformation (e.g. question -> answer, or a typed class with fields), describing the task rather than the prompt for any specific model (DSPy paper, Medium / Jules Damji). Every LM call in a DSPy program uses a signature in place of a hand-written prompt; signatures are then compiled into "self-improving and pipeline-adaptive prompts." [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#2-signatures-declarative-io-specs]

3. Typed LLM functions (schema-first)

The schema-first family generalizes signatures to every language: "every prompt is a function that takes in parameters and returns a type" (BAML GitHub). Declaring the return type yields type safety, IDE autocomplete, compile-time checking, streaming of partial types, and reliable structured output even from models without native tool-calling (Starlog). This reframes "prompt engineering" as schema engineering. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#3-typed-llm-functions-schema-first]

4. Modules / prompting strategies as composable units

Modules wrap a signature with a concrete prompting strategy and compose into pipelines: Predict (bare signature + boilerplate), ChainOfThought, ProgramOfThought, ReAct, MultiChainComparison (dbreunig, DSPy paper). Modules "replace existing hand-prompting techniques" so the reasoning strategy is a swappable code object rather than baked-into-string boilerplate. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#4-modules-prompting-strategies-as-composable-units]

5. Compilation + optimizers/teleprompters

Compilation takes a program (signatures + modules), training examples, and a metric function, then runs an optimizer (formerly "teleprompter") that generates and evaluates many prompt variants - selecting few-shot demonstrations, rewriting instructions, and tuning structure to maximize the metric (DSPy optimizers docs, search synthesis). DSPy can also fine-tune model weights, and can use a larger model to generate prompts for a smaller execution model to avoid overfitting (dbreunig). (The optimization algorithms themselves - MIPROv2, BootstrapFewShot, COPRO, GEPA - are detailed under Tools below and are owned by the prompt-optimizer skills; see cross-reference note.) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#5-compilation-optimizersteleprompters]

6. Constrained generation / decoding

Rather than asking nicely and validating, constrained generation makes invalid output impossible by masking disallowed tokens during decoding. Autoregressive generation is reformulated as transitions between states of a finite-state machine (FSM) compiled from a regex/JSON-schema/grammar; invalid next-token logits are set to -inf (Outlines / FSM origin, mbrenndoerfer). Supported constraint forms are typically regex, context-free grammars (CFG), and JSON Schema (arXiv 2403.06988). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#6-constrained-generation-decoding]

7. Token healing

A decoding refinement (popularized by Guidance) that prevents tokenization boundaries from corrupting prompts: when a grammar makes some tokens known in advance, the framework inserts them rather than forcing a forward pass - e.g. after </, Guidance fills h1> directly - saving forward passes and GPU time (Microsoft Research, Guidance GitHub). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#7-token-healing]

8. Validation + auto-retry/reask

The app-level pattern: validate output against a schema; on failure, re-prompt the model with the validation error attached, looping until valid or a retry cap (Modelmetry, search synthesis). This is the core loop Instructor wraps; Anthropic's SDK even strips schema constraints (min/max/pattern) into the description and validates+retries post-generation (Collin Wilkins). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#8-validation-auto-retryreask]

DSPy (Stanford NLP → Databricks) — the compiled-prompting reference

Python framework released late 2023; ~28k+ GitHub stars and 160k+ monthly pip downloads by mid-2025 (search synthesis). Three abstractions: signatures, modules, optimizers (DSPy paper). Reported gains: 10-40% quality improvement over manual prompting on structured tasks (MyEngineeringPath); a concrete categorization example moved 51.9% → 63.0% accuracy with MIPROv2 (dbreunig). Optimizers (cross-ref the algorithm skills): [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#dspy-stanford-nlp-databricks-the-compiled-prompting-reference]
- BootstrapFewShot / BootstrapFewShotWithRandomSearch - generate and select few-shot demonstrations from the training set. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#dspy-stanford-nlp-databricks-the-compiled-prompting-reference]
- MIPROv2 - jointly optimizes instructions + demonstrations via an LLM proposing variants scored against the metric; recommends 200+ examples, 40+ trials (search synthesis). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#dspy-stanford-nlp-databricks-the-compiled-prompting-reference]
- COPRO - coordinate-ascent instruction optimization. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#dspy-stanford-nlp-databricks-the-compiled-prompting-reference]
- GEPA (Genetic-Pareto, Agrawal et al. 2025, ICLR 2026 oral) - reflective prompt evolution: the LM reflects on execution traces (inputs/outputs/failures/feedback), proposes new instructions, and maintains a Pareto frontier of candidates. Reported to beat MIPROv2 by ~10-13% and GRPO by 20% with 35x fewer rollouts, working from as few as 10 examples (arXiv 2507.19457, DSPy GEPA docs). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#dspy-stanford-nlp-databricks-the-compiled-prompting-reference]
- Assertions - DSPy Assert/Suggest constraints let you enforce/soft-enforce output properties and trigger self-correction (DSPy framework feature). Best fit: structured tasks (QA, classification, extraction, multi-hop reasoning) where you have evaluation data; not for one-shot tasks or pure orchestration (phData). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#dspy-stanford-nlp-databricks-the-compiled-prompting-reference]

BAML (BoundaryML) — schema-first typed LLM functions in a DSL

Open-source, Rust-built DSL: you write .baml files declaring typed functions, schemas, prompts, model choice, and retry policy; the compiler generates type-safe clients for Python, TypeScript, Ruby, Go, Java, C#, Rust (+ REST) (BAML GitHub). Tagline: "adds the engineering to prompt engineering." Key innovation: Schema-Aligned Parsing (SAP) - tolerantly parses real LLM output (markdown-in-JSON, chain-of-thought before the answer, trailing commas) into the declared type without requiring native tool-calling, so "structured outputs work on Day 1 of a model release" (Starlog, Medium / Manav Israni). Test-driven prompts: test blocks with check/assert run against live APIs via the VSCode/Cursor/JetBrains playground or baml-cli test, previewing the full rendered prompt and parsed result (BAML testing docs, DEV / hellovai). Distinction from Pydantic/Instructor: prompts + model selection + retries live declaratively in .baml files (centralized, version-controlled) rather than as library post-processing in app code (BAML GitHub, Medium / Raj Kundalia). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#baml-boundaryml-schema-first-typed-llm-functions-in-a-dsl]

LMQL (ETH Zürich) — constrained query language

A declarative, SQL-like programming language for LM interaction. Constraints (regex, type, stopping conditions) are "evaluated eagerly on each generated token" and compiled into token masks during generation, so the model is guided or validation fails early; supports interleaving control flow with generation (LMQL site, LMQL constraints docs, eth-sri/lmql). Excels at customization and offers workarounds for API-hosted models, but does not accelerate local constrained decoding the way Guidance does (arXiv 2403.06988). (Note: project momentum has slowed relative to Outlines/Guidance - see contested areas.) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#lmql-eth-zürich-constrained-query-language]

Guidance (Microsoft) — constrained generation + token healing

Python library that steers any LM token-by-token at the inference layer, constraining output via regex and CFGs and interleaving control (conditionals, loops, tool use) with generation (Microsoft Research, Guidance GitHub). Its guidance acceleration / token healing skips forward passes for grammar-determined tokens, cutting GPU cost; benchmarks show highest empirical coverage on 6 of 8 datasets (search synthesis). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#guidance-microsoft-constrained-generation-token-healing]

Outlines (.txt) — FSM-based structured generation

Originated the FSM reformulation of constrained decoding: JSON Schema → regex → FSM determining valid next tokens, compiled into index structures enabling O(1) valid-token lookup per step (LMSYS compressed FSM, zenvanriel). Noted as the fastest option for simple, high-volume tasks like ticket classification (Paul Simmering); supports regex, code, and generative constructs; integrated into serving stacks (LoRAX, AWS) (AWS ML blog). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#outlines-txt-fsm-based-structured-generation]

Instructor — typed structured output, minimal abstraction

"Safest default for most projects" - stays close to the OpenAI client, wraps the validate-retry loop, routes to provider-native structured output when available and falls back to tool calling otherwise; "works everywhere, almost no learning curve, covers the 80% case" (Paul Simmering, search synthesis). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#instructor-typed-structured-output-minimal-abstraction]

Pydantic AI — official Pydantic agent framework

Model-agnostic, production-grade agent framework from the Pydantic team, built to solve the "glue code" problem; recommended for new projects for its agent abstraction and clean code (Pydantic Docs, Paul Simmering). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#pydantic-ai-official-pydantic-agent-framework]

Marvin, Mirascope, Ell — function-based / decorator prompting

Marvin: simplest syntax, many built-in tasks, limited customization (Paul Simmering). Mirascope ("the LLM anti-framework"): turns plain Python functions into LLM calls via @llm.call + @prompt_template decorators, provider-agnostic across 20+ providers (switch with one parameter), explicitly avoiding DAG/"superfluous abstraction"; its Lilypad companion auto-versions every prompt-bearing function (Mirascope GitHub, Mirascope blog). Ell: treats prompts as versioned functions (same design lineage - prompts-as-functions with automatic versioning). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#marvin-mirascope-ell-function-based-decorator-prompting]

App-level structured output (the substrate)

Three mechanisms: (1) native provider Structured Outputs (OpenAI/Anthropic/Gemini/Bedrock/Mistral) with strict: true giving constrained-decoding guarantees; (2) constrained decoding (logit masking); (3) prompt + validate + retry (Agenta, BuildMVPFast). Plain JSON mode (valid JSON, no schema enforcement) is widely considered obsolete in production since mid-2025, superseded by schema-enforcing Structured Outputs (BuildMVPFast). Function-calling-as-extraction = providing a tool schema and using the model's filled arguments as your structured data. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#app-level-structured-output-the-substrate]

Practical Patterns

Pick the layer to the problem. "Are you mostly wiring an application, or mostly improving how a model behaves?" - DSPy optimizes task accuracy; orchestration frameworks optimize integration breadth (Leanware, zenvanriel). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#practical-patterns]
Hybrid is the mature default. Use an orchestrator (LangGraph etc.) for routing/memory/tools/retries, and drop DSPy inside a classifier/reranker/planner that benefits from optimization - "keep each framework in its lane" (Leanware). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#practical-patterns]
Declarative wins when you have eval data + repeated invocation. Optimization needs a training set + metric; the payoff scales with how often the prompt runs and how measurable quality is (phData). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#practical-patterns]
Portability is a first-class benefit. Typed-function libraries switch providers by changing one parameter without touching prompt logic (Mirascope); BAML/SAP makes new models work "Day 1" (Starlog). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#practical-patterns]
Constrained decoding for guarantees; retry-loops for portability. Use FSM/grammar constraints (Outlines/Guidance/native strict mode) when you need hard guarantees and control the decoder; use validate+retry (Instructor) when calling closed APIs without decoder access (mbrenndoerfer). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#practical-patterns]
Test prompts like code. BAML test blocks, Mirascope/Lilypad auto-versioning, and prompt-testing frameworks bring assertions + version control to prompts (BAML testing docs, Mirascope). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#practical-patterns]
High-volume simple extraction → Outlines; safe 80% default → Instructor; new agentic project → Pydantic AI; cross-language typed prompts → BAML (Paul Simmering, techsy). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#practical-patterns]

Anti-Patterns

Opaque / hard-to-debug compiled prompts. Optimized prompts are auto-generated; understanding why a demonstration was chosen requires inspecting the compilation trace, and developers "want to see what's being sent to the model" (search synthesis, Morph). DSPy lacks native tool-call logs, making trace-debugging harder than explicit chains (Leanware). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#anti-patterns]
Hidden token/compilation cost. Compiled prompts carry higher token counts; a MIPRO run with 200 examples can cost $5-10, and one documented compile took ~6 min / 3,200 API calls / 2.7M input tokens / $3 (search synthesis). Counter-argument: developer time saved on manual "prompt fiddling" usually dominates (MyEngineeringPath). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#anti-patterns]
Over-abstraction / "magic." Senior engineers have "a well-earned allergy to magic"; abstractions can hide the exact levers teams need over prompt shape, tool wiring, latency, and reliability - the root of many "the framework slowed us down" stories (techtidesolutions). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#anti-patterns]
Framework lock-in & ecosystem immaturity. Smaller ecosystems, fewer tutorials/SO answers, docs referencing old API versions; declarative DSLs (BAML, LMQL) add a non-Python language to learn (Redwerk, search synthesis). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#anti-patterns]
Reaching for compilation on one-shot tasks. DSPy/optimization is overkill for simple one-shot prompts or pure chain orchestration - plain prompting wins (phData). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#anti-patterns]
Using obsolete plain JSON mode in production when schema-enforcing Structured Outputs / constrained decoding are available (BuildMVPFast). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#anti-patterns]

Major Sub-Concepts (candidate child concepts)

Compiled/optimized prompting & the compile step (DSPy signatures → modules → optimize loop) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#major-sub-concepts-candidate-child-concepts]
Signatures & DSPy modules (Predict / ChainOfThought / ReAct / ProgramOfThought as composable strategies) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#major-sub-concepts-candidate-child-concepts]
Schema-first typed LLM functions & DSLs (BAML .baml, prompts-as-typed-functions, SAP, codegen) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#major-sub-concepts-candidate-child-concepts]
Constrained generation / decoding (FSM, regex, CFG, logit masking - Outlines / Guidance / LMQL) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#major-sub-concepts-candidate-child-concepts]
Token healing & decoding-time acceleration (Guidance) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#major-sub-concepts-candidate-child-concepts]
Type/validation structured-output libraries (Instructor, Pydantic AI, Marvin, Mirascope, Ell - function-based prompting + validate/retry) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#major-sub-concepts-candidate-child-concepts]
App-level structured output (native Structured Outputs, JSON mode, function-calling-as-extraction, reask loops) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#major-sub-concepts-candidate-child-concepts]
Test-driven & versioned prompting (BAML test blocks, Mirascope/Lilypad auto-versioning, prompt CI) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#major-sub-concepts-candidate-child-concepts]
When-declarative-wins decision framework & anti-patterns (portability/testability/optimizability vs over-abstraction, opacity, hidden cost, lock-in) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#major-sub-concepts-candidate-child-concepts]
Adjacent declarative prompt languages (PDL, Prompt Decorators - the broader research frontier) [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#major-sub-concepts-candidate-child-concepts]

Knowledge Gaps / Contested Areas

Ell has thin recent coverage in these sources; its design (prompts-as-versioned-functions) is inferred from the shared "prompts as functions" lineage with Mirascope rather than primary docs read in full. Low confidence on Ell specifics. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#knowledge-gaps-contested-areas]
LMQL's current maintenance status is ambiguous: sources describe it as a leading constrained query language, but several comparisons in 2025-2026 favor Outlines/Guidance, and community signal suggests LMQL momentum has slowed. The home-page fetch timed out, so the "slowed momentum" read is Low-to-Medium confidence (inferred from comparative articles, not a primary deprecation notice). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#knowledge-gaps-contested-areas]
DSPy quality-gain figures (10-40%; 51.9%→63.0%; GEPA's +10-13% over MIPROv2) are task- and benchmark-specific and should not be read as universal - Medium confidence; the direction (optimization helps on structured, eval-backed tasks) is High confidence. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#knowledge-gaps-contested-areas]
The DSPy paper PDF and several arXiv/official PDFs timed out on fetch; primary-source claims rely on the structured search synthesis plus successfully-fetched secondary sources (dbreunig, BAML GitHub) - direction is consistent across 3+ sources but exact figures carry the above caveats. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#knowledge-gaps-contested-areas]

Sources

DSPy (official site) - "programming, not prompting"; framework home. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
DSPy: Compiling Declarative LM Calls into Self-Improving Pipelines (arXiv 2310.03714) - foundational paper; signatures/modules/teleprompters. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
DSPy Optimizers docs - optimizer/teleprompter catalog. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
An Exploratory Tour of DSPy (Jules Damji, Medium) - signature definition, abstractions. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Pipelines & Prompt Optimization with DSPy (dbreunig) - modules, MIPROv2 compile workflow, 51.9%→63.0%. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Intro to DSPy: Goodbye Prompting (Towards Data Science) - brittleness of string prompts. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
DSPy Framework guide 2026 (MyEngineeringPath) - compiled-artifact thesis; token/compile cost. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
DSPy: Declaring Instead of Prompting (TesterStories) - compiler analogy. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
GEPA: Reflective Prompt Evolution (arXiv 2507.19457) - GEPA optimizer; Pareto frontier; vs MIPROv2/GRPO. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
DSPy GEPA overview - GEPA in DSPy. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Prompts Are Programs Too! (arXiv 2409.12447) - empirical study of prompt-bearing software. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
PDL: A Declarative Prompt Programming Language (arXiv 2410.19135) - IBM declarative prompt language. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Prompt Decorators (arXiv 2510.19850) - declarative composable prompt syntax. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
BAML GitHub (BoundaryML) - prompts-as-typed-functions, SAP, multi-language codegen. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
BAML: Schema-First Language (Starlog) - schema-first, Day-1 model support. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
BAML Testing Functions docs - test blocks, checks/asserts, CLI/playground. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
BAML VSCode Playground (DEV / hellovai) - DSL + playground. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
How BAML brings engineering discipline (Medium / Raj Kundalia) - Pydantic vs Instructor vs BAML. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
LMQL (official site) - SQL-like LLM query language. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
LMQL Constraints docs - eager per-token constraint evaluation, token masks. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
eth-sri/lmql GitHub - constraint-guided LLM programming. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Guidance (Microsoft Research) - token-by-token control, token healing. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Guidance GitHub - regex/CFG constraints, interleaved control, acceleration. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Guiding LLMs The Right Way (arXiv 2403.06988) - non-invasive constrained generation; CFG/regex/template. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Generating Structured Outputs: Benchmark (arXiv 2501.10868) - coverage benchmarks (Guidance 6/8 datasets). [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Compressed FSM for JSON decoding (LMSYS) - FSM/regex structured generation internals. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Constrained Decoding: Grammar-Guided Generation (mbrenndoerfer) - FSM, logit masking, decode-vs-retry tradeoff. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Outlines for Structured Generation (zenvanriel) - Outlines usage, JSON-schema constraints. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Generate structured output with Outlines in AWS (AWS ML blog) - Outlines in serving. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
The best library for structured LLM output (Paul Simmering) - Instructor vs Outlines vs Pydantic AI comparison. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Comparing Python libraries for structured extraction (McGinnis) - library comparison. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
llm-structured-output-benchmarks (GitHub / stephenleo) - Instructor/Mirascope/Marvin/Outlines benchmarks. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Pydantic AI (Pydantic Docs) - official agent framework. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Mirascope GitHub - "anti-framework", decorator prompts, 20+ providers. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Mirascope: Prompt Versioning - Lilypad auto-versioning of prompt functions. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
8 LLM Structured Output Libraries Ranked 2026 (TECHSY) - library landscape. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Guide to structured outputs and function calling (Agenta) - three mechanisms. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
JSON Mode vs Function Calling vs Structured Output 2026 (BuildMVPFast) - JSON mode obsolescence; strict mode. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Ensure LLM output adheres to JSON Schema (Modelmetry) - validate/retry loop. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
LLM Structured Outputs: Schema Validation (Collin Wilkins) - Anthropic SDK constraint-stripping + retry. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
LangChain vs DSPy (Leanware) - accuracy-vs-orchestration, debuggability, hybrid pattern. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
LangChain vs DSPy: Prompt Engineering vs Programming (zenvanriel) - paradigm contrast. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Is LangChain Bad? (TechTide) - over-abstraction / "magic" critique. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Prompt Programming with DSPy (phData) - when (not) to use DSPy. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Debugging DSPy token usage and prompts (Daniel Corin) - opacity/debugging of compiled prompts. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
LLM Frameworks Compared 2026 (Morph) - wanting to see the sent prompt; framework landscape. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]
Beyond Free-Form Text: Constrained Decoding (Medium / Brijesh Nambiar) - FSM origin of Outlines, -inf masking. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#sources]

Methodology

Ran 12 web search queries plus targeted page fetches across the paradigm, each framework family, and the anti-patterns/decision angle. Deep-read (full fetch) succeeded for dbreunig (DSPy workflow) and the BAML GitHub README; several arXiv PDFs and the dspy.ai/lmql.ai pages timed out, so those claims rest on structured search synthesis cross-checked against 3+ sources. Sub-questions investigated: (1) the prompt-as-program paradigm & its rationale; (2) DSPy programming model (signatures/modules/optimizers/compile/assertions); (3) BAML schema-first DSL + SAP + testing; (4) LMQL & Guidance constrained generation + token healing; (5) type/schema structured-output libraries (Instructor/Pydantic AI/Outlines/Marvin/Mirascope/Ell); (6) app-level structured output (JSON mode / function-calling-as-extraction / reask); (7) when-declarative-wins; (8) anti-patterns. Per the deep-research fallback protocol (no firecrawl/exa configured), source-count targets were raised ~50%; injection guard honored - no fetched content was treated as instructions. [archived source: https://llms-explorer.com/sources/global-ai-hub/declarative-llm-frameworks/#methodology]

Where this helps

Building a production LLM pipeline where prompt strings scattered across the codebase have become unmaintainable and need typed, composable structure instead. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Needing structured output (JSON conforming to a schema) reliably from a model, rather than parsing free text and hoping the format holds. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Optimizing a multi-step LLM pipeline's prompts automatically against a labeled dataset instead of hand-tuning wording by trial and error. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Choosing between a lightweight schema-first library (BAML, Instructor, Pydantic AI) and a full compiled-prompting framework (DSPy) based on how much automatic optimization the project actually needs. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.

Project ideas

Rewrite a hand-tuned prompt chain as DSPy signatures and modules, then run a teleprompter/optimizer against a small labeled dev set to compile better prompts automatically. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Add BAML or Instructor to an existing LLM integration to get typed, validated structured output with automatic retry/reask instead of manual JSON parsing and regex cleanup. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Build a constrained-generation demo with Outlines or Guidance that guarantees a model's output always matches a target grammar (e.g. valid SQL, a valid JSON schema) even under adversarial input. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Compare token-healing-aware decoding against naive prompting on a task with brittle tokenization boundaries, such as generating code with specific indentation, to see the practical difference. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.

Common mistakes

Adopting a compiled-prompting framework like DSPy without a labeled dataset or optimization budget, then being surprised that it offers structure but not automatic gains. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Cranking up constrained-decoding strictness for every field when only a few actually need a hard grammar, adding latency and sometimes quality loss for no real benefit. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Letting the abstraction layer (signatures, modules, compilation) hide exactly what prompt text is actually being sent to the model, which makes debugging a bad output much harder. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Pinning a project to framework defaults and version behavior from 2024 without re-checking them, in an ecosystem where APIs, defaults, and even project viability have shifted materially through 2026. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.

Known issues

Compiled-prompting frameworks like DSPy require a labeled dataset and an optimization budget of extra LLM calls to actually improve on a hand-written prompt; without that data, the framework offers structure but not automatic gains. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
Constrained decoding (grammar or schema enforcement) can materially slow generation and, depending on the library and model, sometimes degrades output quality by forcing token choices the model wasn't confident about. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
The abstraction layer these frameworks add — signatures, modules, compilation — has its own learning curve and can obscure exactly what prompt text is being sent to the model, complicating debugging. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.
This ecosystem moved fast between 2024 and 2026; framework APIs, defaults, and even project viability (forks, deprecations) have shifted enough that version-specific guidance ages quickly. [source] — AI-suggested, synthesized from this pack's existing facts — not extracted from a source document.

Children

Compiled/optimized prompting (DSPy signatures/modules/compile) (frontier)
Schema-first typed LLM functions (BAML + Schema-Aligned Parsing, codegen) (frontier)
Constrained generation/decoding (Outlines FSM, Guidance token-healing, LMQL) (frontier)
Type/validation structured-output libs (Instructor, Pydantic AI, Mirascope) (frontier)
App-level structured output (strict mode, function-calling-as-extraction, validate+reask) (frontier)
Test-driven & versioned prompting (frontier)
When-declarative-wins decision & anti-patterns (frontier)

Frontier under this node: App-level structured output (strict mode, function-calling-as-extraction, validate+reask), Compiled/optimized prompting (DSPy signatures/modules/compile), Constrained generation/decoding (Outlines FSM, Guidance token-healing, LMQL), Schema-first typed LLM functions (BAML + Schema-Aligned Parsing, codegen), Test-driven & versioned prompting, Type/validation structured-output libs (Instructor, Pydantic AI, Mirascope), When-declarative-wins decision & anti-patterns

← the whole tree · 3D view· how to read this page