Microsoft Foundry Local

Parent: Ollama alternatives and model sourcing · Published reference · snapshot 2026-10-14 · skill ollama-expert/references/ollama-alternatives-and-model-sourcing.md

Also known as: Microsoft Foundry Local relative to Ollama

↓ Facts as markdown all context files

Everything the scanned sources say about Microsoft Foundry Local relative to Ollama: 107 source-anchored units across 7 sources, grouped by facet.

These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.

Definitions

This is the single most important distinction to get right about Foundry Local: Foundry Local is not the same product as Azure AI Foundry, Microsoft's cloud-based platform for building and deploying AI at scale. Per Microsoft's own positioning, Foundry Local is "the high-performance local AI runtime stack that brings Azure AI Foundry's power to client devices" — language that signals a deliberate relationship between the two, not identical products. [source]
Foundry Local is a separate, on-device product from Azure AI Foundry, Microsoft's cloud platform for building and deploying AI at scale. Foundry Local runs entirely on a user's device and does not require an Azure subscription for local-only use, even though the two are designed to interoperate in hybrid workflows. [source]
No. Foundry Local is a separate, on-device product that runs entirely on a user's device without requiring an Azure subscription for local-only use. Azure AI Foundry is Microsoft's cloud-based platform for building and deploying AI at scale. The two are designed to interoperate in hybrid workflows but are not the same product. [source]

Structure and components

Part of the Local AI Directory View Foundry Local in the directory→ [source]
Model and component downloads—The first time a model runs, Foundry Local downloads the model files and might also download execution providers for the user's hardware. [source]

How it works

When you install the SDK, the Foundry Local Core and ONNX Runtime binaries are automatically downloaded and bundled into your application during build as a dependency. The Foundry Local SDKs (Python, JavaScript, C#, Rust) are thin wrappers that call into the Foundry Local Core native library, which is the main runtime managing the model lifecycle (download, load into device memory, inference management, and unload). Foundry Local integrates with Foundry Catalog to download models on first run that are optimized for the device hardware, which is done intelligently so that end users will get the [source]
No. Foundry Local is an **end-to-end local AI solution** that your application ships with. It handles model acquisition, hardware acceleration, and inference inside your app process through the SDK. The optional web server and CLI are available for development workflows, but the core product is the local AI runtime and SDK that you integrate directly into your application. [source]
Foundry Local's catalog is intentionally curated: chat models such as GPT OSS, Qwen, DeepSeek, Mistral and Phi, plus Whisper for audio transcription. Microsoft says it does not support every model because it targets shipping production apps, not general model experimentation. [source]
Foundry Local runs as an in-process native library. [source]
If you need to serve models to multiple concurrent users, use a dedicated server inference framework. Use Foundry Local when the model runs on the end user's own device. [source]
The two products are designed to interoperate in hybrid workflows — a developer might prototype and test locally with Foundry Local, then scale a workload to Azure AI Foundry's cloud endpoints when it needs more capacity than a single device can provide — but they remain separate products, and Foundry Local does not require an Azure subscription for local-only use. Do not assume the two names refer to the same thing, or that installing Foundry Local creates any dependency on an Azure account. [source]
Foundry Local runs inference entirely on the device. When your application sends prompts to a Foundry Local endpoint, prompts and model outputs are processed locally. [source]

How-to and procedures

The Foundry Local inference APIs support the OpenAI request/response format for chat completions, audio transcription, and the Open Responses API format allowing you to seamless switch between cloud and on-device inference without the overhead and complexity of spinning up a local HTTP webserver. However, if your scenario demands an OpenAI compliant HTTP webserver to make REST calls, Foundry Local has you covered; you can configure an optional webserver on initialization. [source]
Foundry Local is optimized for hardware-constrained devices where a single user accesses the model at a time. While you can technically install and run it on server hardware, it isn't designed as a server inference stack. [source]
Yes, Foundry Local itself is free to install and use for local-only inference, with no subscription required. Individual downloaded models carry their own separate license terms, and any use of Azure AI Foundry alongside it in a hybrid workflow carries Azure's own separate billing. [source]
Use of Foundry Local is governed by the product terms and licenses that apply to the software and the models in use. If the terms allow Microsoft to collect diagnostic information, the details are described in those terms and the Microsoft Privacy Statement. [source]
If you want to run custom models like I did on my EVO-X2, Foundry Local isn’t the right tool. [source]

Measurements and reference values

Foundry Local (github.com/microsoft/Foundry-Local) is Microsoft's on-device AI inference tool — an end-to-end local AI solution for building applications that run entirely on a user's device, with native SDKs for C#, JavaScript, Rust, and Python, automatic hardware-acceleration detection (it selects the NPU, GPU, or CPU per device), and a curated model catalog, in a lightweight (roughly 20 MB) package. It is distinct from Azure AI Foundry, Microsoft's cloud-based platform for building and deploying AI at scale: Foundry Local runs entirely on-device and does not require an Azure subscription fo [source]
Foundry Local is an **end-to-end local AI solution** for building applications that run entirely on the user's device. It provides native SDKs (C#, JavaScript, Python, and Rust), a curated catalog of optimized models, and automatic hardware acceleration — all in a lightweight package (~20 MB). The compact size makes it easy to integrate into your application and distribute to end users. [source]
Microsoft Ships Foundry Local GA: A ~20MB Native Library That Bundles Local AI Into Any App [source]
Microsoft claims “ONNX Runtime is on average 3.9x faster than llama.cpp, and up to 13.4x faster for long sequences.” [source]
Foundry Local is an embeddable SDK plus native runtime (C#, JavaScript, Rust, Python) that runs inference in-process and adds about 20 MB to an app package; the web server and CLI are optional development tools, not the core product. [source]
“What a wonderful project: parakeet.cpp. A ggml based parakeet inference pipeline that is 2x faster than my ONNX parakeet pipeline on Apple Silicon.” [source]
Foundry Local's repository (github.com/microsoft/Foundry-Local) showed 2,541 stars as of September 5, 2026, per this site's own directory verification. Check the repository directly for a current count, since it changes over time. [source]

Problems, failure modes and limitations

Foundry Local is Microsoft's on-device AI inference tool for building applications that run entirely on a user's device, with native SDKs for C#, JavaScript, Rust, and Python and automatic hardware-acceleration detection. It is distinct from Azure AI Foundry, Microsoft's cloud platform for building and deploying AI at scale — Foundry Local runs offline-capable and does not need an Azure subscription for local use, while Azure AI Foundry is a separate cloud product the two can interoperate with in hybrid workflows. [source]
Distinct from Azure AI Foundry: Foundry Local runs on-device and does not require an Azure subscription for local-only use, while Azure AI Foundry is Microsoft's separate cloud platform for building and deploying AI at scale; the two are designed to interoperate in hybrid workflows [source]
Foundry Local runs entirely on-device with no cloud dependency and requires no Azure subscription. [source]
Individual models made available for use with Foundry Local are subject to the each model's license terms, notices, and use restrictions. Refer to the model's documentation or download/listing page for the applicable terms before using or redistributing a model. [source]
Mistake 2: Assuming Foundry Local is fully open source [source]
Mistake 3: Assuming the model catalog covers every model you might want [source]

Comparisons and alternatives

Foundry Local (github.com/microsoft/Foundry-Local, foundrylocal.ai) is Microsoft's on-device AI inference tool: an end-to-end local AI solution for building applications that run entirely on a user's device, with native SDKs for C#, JavaScript, Rust, and Python, automatic hardware-acceleration detection, and a curated model catalog, packaged in a lightweight (roughly 20 MB) install. This review covers what Foundry Local actually is, how it differs from Microsoft's cloud-based Azure AI Foundry, its dual-license structure, how to install it, and where it fits next to other local-inference tools. [source]
Foundry Local is designed for shipping production applications, not for general-purpose model experimentation. The model catalog is intentionally curated to include models that are optimized for specific application scenarios, tested across a range of consumer hardware, and small enough to distribute to end users. This approach ensures that every model in the catalog delivers reliable performance when embedded in your application — rather than offering a broad selection of models with unpredictable on-device behavior. [source]
A third-party reviewer reported Foundry Local 20-30% faster than Ollama on the same models on their own laptop. This is a single anecdote, not a benchmark. Microsoft's ONNX-versus-llama.cpp speed claims compare ONNX-optimized models, not matched GGUF quantizations. PromptQuorum's review quotes no benchmark numbers. [source]
Curated model catalog — model families such as Qwen, DeepSeek, Mistral, Phi, and GPT-OSS, plus Whisper for audio transcription; the catalog is curated rather than an open upload system, and each model carries its own separate license terms [source]
Foundry Local and Ollama get compared because both let a developer run models locally through a CLI and an API — the real difference for a reader deciding between them is the inference engine underneath and which ecosystem (Microsoft's SDKs vs. a hardware-agnostic llama.cpp-based runtime) fits the target application. [source]
Why it matters: Building on ONNX Runtime, rather than a project-specific inference engine, lets Foundry Local benefit from ONNX Runtime's own hardware-acceleration and execution-provider ecosystem. [source]
Model catalog: a curated set of models, rather than an open, unrestricted upload catalog [source]
Microsoft released Foundry Local as an on-device counterpart to its broader Azure AI Foundry platform, aimed at developers who need inference to run on client hardware rather than in the cloud. [source]
Ollama — a general-purpose local model runtime and API built on llama.cpp, fully MIT-licensed, with a larger and more open model library and no Microsoft-specific SDK requirement. The more direct fit if you want the broadest hardware compatibility and model choice rather than native C#/JavaScript/Rust/Python SDKs. See the full Ollama review. [source]
Why it matters: Foundry Local launched positioned specifically as the local, client-device counterpart to Azure AI Foundry, rather than as a standalone product unrelated to Microsoft's cloud AI strategy. [source]
Foundry Local is a CLI-and-SDK developer tool, not a single consumer app with per-OS download buttons, so installation happens through GitHub Releases and language package managers rather than a signed installer from a marketing page. [source]
Foundry Local is free to install and use, with no subscription required for local-only inference. Licensing is more nuanced than a single blanket term, however: the SDK is MIT-licensed, while the CLI is distributed under Microsoft's own proprietary Microsoft Software License Terms — confirmed via the project's own LICENSE file. It is not accurate to describe the entire product as "open source" or as fully "MIT-licensed." [source]
Whether Foundry Local is worth adopting depends mainly on whether you are building an application (rather than just wanting a chat interface) and whether Microsoft's SDK languages and hardware-acceleration approach match your project. [source]
You want a polished graphical chat app for end users rather than a developer-facing CLI and SDK — Foundry Local has no consumer-facing chat interface of its own [source]
You need Foundry Local for something other than the platforms in its official README (Windows, macOS Apple Silicon, Linux) — confirm exact platform and device support against the official repository rather than assuming coverage [source]
The key difference: in the Foundry Local approach, the user’s workflow has zero AI-related steps. [source]

Changes and history

Foundry Local v2.1.0 added bring-your-own-model: it registers ONNX Runtime GenAI models in the local catalog without copying the files, from C++, C#, Python, JavaScript/TypeScript and Rust. Registered models use the same discovery, load and session APIs as catalog models. The release notes name no GGUF support. [source]
Today we’re thrilled to announce the General Availability (GA) of Foundry Local — Microsoft’s cross-platform local AI solution that lets developers bring AI directly into their applications across modalities like chat and audio, with no cloud dependency, no network latency, and no per-token costs. [source]
We're excited to announce Foundry Local v2.1.0. This release makes it easier to bring your own models to local applications, expands tool calling for agentic workflows, and delivers important security, reliability, and inference improvements across the SDKs. [source]
Foundry Local reached general availability on 2026-04-09, announced on the Microsoft Foundry devblog. [source]
Microsoft’s newly GA’d “Foundry Local” fills this gap. [source]
Download the asset for your platform from the [`cli-preview-0.10.0`](https://github.com/microsoft/Foundry-Local/releases/tag/cli-preview-0.10.0) GitHub release. [source]

Facts and statements

Foundry Local is Microsoft's on-device AI inference tool — an end-to-end local AI solution for building applications that run entirely on a user's device. Its GitHub repository describes it as bringing native SDKs, automatic hardware acceleration, and a curated model catalog together in a lightweight package aimed at developers who need inference to happen on the device itself, not on a remote server. [source]
Model formatGGUF (flexible quantization)GGUF (flexible quantization)ONNX (from Foundry Catalog) [source]
Foundry Local is Microsoft's on-device AI inference tool — native SDKs for C#, JavaScript, Rust, and Python, automatic hardware-acceleration detection, and a curated model catalog — and it is a separate product from the cloud-based Azure AI Foundry. [source]
Foundry Local (github.com/microsoft/Foundry-Local) is Microsoft's on-device AI inference tool — an end-to-end local AI solution for building applications that run entirely on a user's device, with native SDKs for C#, JavaScript, Rust, and Python. [source]
Supported platforms are Windows, macOS (Apple Silicon) and Linux. An Azure subscription is not required. Prompts and outputs stay on-device; the network is used only for model/execution-provider downloads and optional diagnostics. [source]
→Curated model catalog includes model families such as Qwen, DeepSeek, Mistral, Phi, and GPT-OSS, plus Whisper for audio transcription; each downloaded model carries its own license [source]
Curated model catalog includes model families such as Qwen, DeepSeek, Mistral, Phi, and GPT-OSS, plus Whisper for audio transcription — each downloaded model carries its own separate license terms [source]
Foundry Local's catalog is curated (model families such as Qwen, DeepSeek, Mistral, Phi, GPT-OSS, plus Whisper), not an open upload system. Check the catalog for a specific model before assuming it is available, and remember each model carries its own separate license terms. [source]
- **OpenAI-compatible API** — Supports OpenAI request and response formats including the [OpenAI Responses API format](https://developers.openai.com/api/reference/resources/responses). If your application already uses the OpenAI SDK, point it to a Foundry Local endpoint with minimal code changes. [source]
Model freedom mattersThe GGUF ecosystem (Ollama / llama.cpp) offers far more choices [source]
If you’re an app developer who wants to embed local inference into your product, you’re stuck with two options: tell users “please install Ollama first” in your README, or link ONNX Runtime or llama.cpp directly and implement model management yourself. [source]
On first launch, the optimal variant for the machine’s hardware is automatically downloaded from the Foundry Catalog and cached locally. [source]
Foundry Local may collect usage data and send it to Microsoft to help improve our products and services. See the [privacy statement](sdk_v2/cpp/docs/Privacy.md) for more details. [source]
- **Automatic hardware acceleration** — Foundry Local detects the available hardware on the user's device and selects the best execution provider and device (NPU, GPU or CPU). [source]
- **Smart model management** — Foundry Local handles the full lifecycle of models on end-user devices. Models download automatically on first use, are cached locally for instant subsequent launches, and the best-performing variant is selected for the user's specific hardware. [source]
Bring your own ONNX Runtime GenAI models into Foundry Local without copying, downloading, or transferring ownership of the model files. [source]
4 Ongoing — curated model catalog expansion [source]
Foundry Local uses ONNX Runtime for inference, with automatic hardware-acceleration detection that selects the optimal execution provider — NPU, GPU, or CPU — per device. [source]
For C#, running dotnet add package pulls in the NuGet package, and at build time, the Foundry Local Core native libraries (.dll / .so / .dylib) and ONNX Runtime binaries are copied into the output directory. [source]
graph TD A[Application] --> B[Foundry Local SDK Python / C# / JS / Rust] B --> C[Foundry Local Core Native Library .dll / .so / .dylib] C --> D[ONNX Runtime] C --> E[Model Manager Download, Cache, Load] D --> F{Hardware Detection} F -->|Windows| G[Windows ML GPU / NPU / CPU auto-select] F -->|macOS| H[Metal Apple Silicon GPU] F -->|Linux| I[CPU fallback] [source]
Foundry Local exposes OpenAI request/response formats, including the Responses API, via an optional OpenAI-compatible local web server. Existing OpenAI-SDK code can point at the endpoint with minimal changes. [source]
For development and debugging, there’s also a mode that starts Foundry Local Core as an OpenAI-compatible HTTP server. [source]
Foundry Local provides an OpenAI-compatible API with chat completions and transcription endpoints. [source]
Foundry Local powered by Azure Local brings models and agentic AI — including RAG, chat — to customer-owned distributed infrastructure. This is in preview now, with more coming soon. [source]
Home/Power Local LLM/Foundry Local Review: Microsoft's On-Device AI Runtime [source]
Overview & Reference Foundry Local Review: Microsoft's On-Device AI Runtime [source]
→On-device AI inference tool from Microsoft; source at github.com/microsoft/Foundry-Local, project site foundrylocal.ai [source]
→Distinct from Azure AI Foundry (Microsoft's cloud AI platform) — Foundry Local runs on-device with no Azure subscription required for local-only use [source]
Foundry Local is Microsoft's on-device AI inference tool; source at github.com/microsoft/Foundry-Local, project site foundrylocal.ai [source]
Foundry Local lets a developer build an app that runs AI models directly on a user's Windows, Mac, or Linux machine, picking whichever chip (NPU, GPU, or CPU) does the job fastest on that specific device, without needing a cloud account for local use. It is not the same thing as Azure AI Foundry, which is Microsoft's cloud platform — the two can work together, but Foundry Local's whole point is running offline, on-device. [source]
Foundry Local's feature set centers on making on-device inference straightforward to integrate for a developer, per its GitHub README. [source]
Licensing is split: the Foundry Local SDK is MIT, the CLI is under Microsoft proprietary terms, and each catalog model carries its own license. Ollama is fully MIT. [source]
> For the full CLI reference and advanced usage, see the [CLI documentation on Microsoft Learn](https://learn.microsoft.com/en-us/azure/foundry-local/reference/reference-cli). [source]
- [Foundry Local Documentation](https://learn.microsoft.com/en-us/azure/foundry-local/) on Microsoft Learn - [Foundry Local Lab](https://github.com/Microsoft-foundry/foundry-local-lab) — Hands-on exercises and step-by-step instructions [source]
If a developer specifically wants automatic hardware-acceleration selection across NPU, GPU, and CPU with native C#, JavaScript, Rust, or Python SDKs — for example, building a Windows or cross-platform client application — Foundry Local's design fits that use case directly. If a developer wants the broadest model library and a fully open-source runtime, Ollama's workflow is the more direct fit. See the full Ollama review for details. [source]
The catalog is curated and includes model families such as Qwen, DeepSeek, Mistral, Phi, and GPT-OSS, plus Whisper for audio transcription. Each model carries its own separate license terms. [source]
graph TD subgraph "Ollama approach" A1[User installs Ollama] --> A2[Starts ollama serve] A2 --> A3[App connects via HTTP] end subgraph "Foundry Local approach" B1[Developer adds SDK package] --> B2[Build bundles native library] B2 --> B3[User installs the app] B3 --> B4[AI inference ready immediately] end [source]
- **Lightweight runtime** — The runtime handles model acquisition, hardware acceleration, model management, and inference (via [ONNX Runtime](https://onnxruntime.ai/)). [source]
Technical Comparison With Ollama and llama.cpp [source]
Exposes an OpenAI-compatible API (including OpenAI Responses API request/response formats) through a local web server, and uses ONNX Runtime for inference [source]
3 Ongoing — ONNX Runtime as the inference engine [source]
Foundry Local detects GPU, NPU and CPU and picks an execution provider automatically, falling back to CPU. On Windows it uses WinML; on macOS it uses the Apple Silicon GPU via Metal. [source]
Whether you’re building a desktop assistant, a healthcare decision-support tool, a private coding companion, or an offline-capable edge application, Foundry Local gives you production-grade AI that runs entirely on the user’s machine. [source]
Microsoft Foundry spans cloud to edge — from Microsoft Foundry in the cloud for frontier models, agents, and fine-tuning, to Foundry Local for on-premises and distributed deployments validated on Azure Local, to Foundry Local running natively across Windows, MacOS, Android, and other devices including phones, laptops, and desktops. [source]
Foundry Local is cross platform: Windows, Linux and macOS. On Windows it integrates with Windows ML (WinML) for inferencing on Windows and to acquire hardware matched execution provider plugins from the OS / Windows update, which ensures driver compatibility and version negotiation. Your end users do not need to be concerned with installing device drivers to get the most optimal performance. On macOS Foundry Local runs natively on the Apple Silicon GPU through Metal. [source]
Samples for each language are provided in the Foundry Local GitHub repository. [source]
Foundry Local supports Windows, macOS (Apple silicon), and Linux. [source]
Canonical repository: github.com/microsoft/Foundry-Local; project site: foundrylocal.ai [source]
6 On supported hardware, Foundry Local's automatic hardware-acceleration detection selects the NPU, GPU, or CPU for you; no manual execution-provider configuration is required for the default path. [source]
Most confusion about Foundry Local comes from conflating it with Azure AI Foundry, or assuming the whole product is open source. [source]
Foundry Local runs on your users' own devices, so your app works offline and you don't need an Azure subscription. If you need enterprise-scale AI inference on your own infrastructure with Kubernetes-native operations and Azure Arc management instead, see Foundry Local on Azure Local. [source]
No. Foundry Local runs entirely on local hardware. No Azure subscription is required. [source]
Foundry Local delegates this choice to Windows ML (on Windows) or Metal (on macOS), shielding developers from hardware differences. [source]
Foundry Local delegates hardware selection to Windows ML, so manual tuning isn’t available. [source]
These numbers come from ONNX-optimized models, though — not a direct comparison with GGUF quantized models. Take them with appropriate context. [source]

Related concepts

Foundry Local — is a hyponym of Microsoft Foundry Local relative to Ollama
on-device — is a related of Microsoft Foundry Local relative to Ollama
ONNX — is a part of Microsoft Foundry Local relative to Ollama
curated catalog — is a part of Microsoft Foundry Local relative to Ollama
LocalAI — is a hyponym of Microsoft Foundry Local relative to Ollama
Ollama — is a related of Microsoft Foundry Local relative to Ollama
inference engine — is a part of Microsoft Foundry Local relative to Ollama
pull counts — is a measure of Microsoft Foundry Local relative to Ollama
Llama — is a instance of Microsoft Foundry Local relative to Ollama
GGUF — is a part of Microsoft Foundry Local relative to Ollama
OpenAI-compatible — is a part of Microsoft Foundry Local relative to Ollama
general availability — is a problem of Microsoft Foundry Local relative to Ollama
Foundry Catalog — is a part of Microsoft Foundry Local relative to Ollama
quantization — is a part of Microsoft Foundry Local relative to Ollama
BYOM — is a part of Microsoft Foundry Local relative to Ollama
Microsoft Learn — is a related of Microsoft Foundry Local relative to Ollama
GGML — is a part of Microsoft Foundry Local relative to Ollama

Children

No children recorded.

← the whole tree · 3D view· how to read this page