Ollama REST API: generate and chat with streaming NDJSON and errors and usage

Parent: Ollama core platform · Published reference · snapshot 2026-10-14 · skill ollama-expert/references/ollama-core-platform.md

↓ Facts as markdown all context files

Everything the scanned sources say about Ollama REST API: generate and chat with streaming NDJSON and errors and usage: 111 source-anchored units across 9 sources, grouped by facet.

These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.

Definitions

num_ctx is the bug fix. Config discipline is the solution. Fix the setting today, then go delete half your soul.md this week. [source]

Structure and components

- `total_duration`: time spent generating the response - `load_duration`: time spent in nanoseconds loading the model - `prompt_eval_count`: number of tokens in the prompt - `prompt_eval_cached_count`: number of prompt tokens read from the cache - `prompt_eval_duration`: time spent in nanoseconds evaluating uncached prompt tokens - `eval_count`: number of tokens in the response - `eval_duration`: time in nanoseconds spent generating the response - `context`: an encoding of the conversation used in this response, this can be sent in the next request to keep a conversational memory - `response`: [source]

How it works

The same /v1 path has a second cost that looks unrelated: several users report the same local model returning corrupted streamed tool_calls over /v1 and correct ones over native /api/chat. If your agent forgets its rules and its tool calls come back mangled, that's one root cause, not two. And if tool calls never fire at all, that's a different Ollama issue: see Ollama does not support tools. [source]
Your agent ignores soul.md or drops skills on Ollama. The cause is the context window default, plus one endpoint that ignores num_ctx entirely. Hermes included. [source]
Why the num_ctx you already set did nothingWhat Ollama actually does when the prompt is too longThe token math that makes this clickCheck what your model actually receivedThree ways to set it, and which to useDon't trust any published "default" for your model, including this pageHow much headroom, and what it costsHermes, OpenClaw, and the framework layerThe fix you should make even after fixing num_ctxFrequently Asked Questions [source]
would it be possible to add an env option so that num_ctx from the request is ignored and always set to the value of OLLAMA_CONTEXT_LENGTH? [source]
The rationale behind this is that for a shared Ollama instance, it's unfortunate if users cause a runner restart by setting an incorrect num_ctx value (even one smaller than the current value). [source]

How-to and procedures

To calculate how fast the response is generated in tokens per second (token/s), divide `eval_count` / `eval_duration` \* `10^9`. [source]

Examples and snippets

```shell — `curl http://localhost:11434/api/generate -d '{` … [source]
```json5 — `"model": "codellama:code",` … [source]
```json — `"model": "llama3.1:8b",` … [source]
```json — `"model": "llama3.2",` … [source]
```shell — `curl -X POST http://localhost:11434/api/generate -H "Content-Type: application/json" -d '{` … [source]
```json — `"model": "llava",` … [source]
```json — `"model": "mistral",` … [source]
```shell — `curl http://localhost:11434/api/copy -d '{` … [source]
```shell — `curl -X DELETE http://localhost:11434/api/delete -d '{` … [source]
```shell — `curl http://localhost:11434/api/push -d '{` … [source]
```shell — `curl http://localhost:11434/api/ps` … [source]
```shell — `curl http://localhost:11434/api/version` … [source]
```shell — `curl http://localhost:11434/api/tags` … [source]
```shell — `curl http://localhost:11434/api/show -d '{` … [source]
```shell — `curl http://localhost:11434/api/pull -d '{` … [source]
```shell — `curl http://localhost:11434/api/embed -d '{` … [source]
```json — `"model": "all-minilm",` … [source]
```shell — `curl http://localhost:11434/api/embed -d '{` … [source]
```shell — `curl http://localhost:11434/api/embed -d '{` … [source]
```shell — `curl http://localhost:11434/api/embed -d '{` … [source]
```shell — `curl http://localhost:11434/api/embeddings -d '{` … [source]

Measurements and reference values

/api/tags is much faster on large model libraries (3.1 s → 294 ms cold in testing), and model capabilities are now reported consistently. [source]

Problems, failure modes and limitations

If an error occurs after streaming has started, the server emits a line with an `error` property in the NDJSON stream and the HTTP status stays 200. Docs example: {"error":"an error was encountered while running the model"}. [source]
Robust client rule: parse each line separately, check for an `error` key before reading content, and treat a stream that ends without done:true as incomplete. A reader that only joins `response` fields can return partial text and silently drop the error line. [source]
Clients that expect a single JSON object from a streaming-default endpoint fail with parse errors (e.g. 'JSON Parse error: Unexpected EOF'); the workaround is stream:false. [source]
If an error occurs mid-stream, the error will be returned as an object in the `application/x-ndjson` format with an `error` property. Since the response has already started, the status code of the response will not be changed. [source]
Ollama documents that errors during streaming arrive as NDJSON objects with an error property. Once streaming starts, the response status is not changed. That means an application which checks only HTTP 200 can miss a later failure. Official error reference [source]
Someone built a reproducible demonstration of this: place a secret at the very start of a 160,000-token prompt, then ask the model to repeat it back. Ollama returned HTTP 200, reported prompt_eval_count: 16387 on a 160,000-token input, and the model confidently invented an answer. No error, no warning, nothing in the logs. [source]

Comparisons and alternatives

1. Open Roo Code in VS Code and click the **gear icon** on the top right corner of the Roo Code window to open **Provider Settings** 2. Set `API Provider` to `Ollama` 3. (Optional) Update `Base URL` if your Ollama instance is running remotely. The default is `http://localhost:11434` 4. Enter a valid `Model ID` (for example `qwen3` or `qwen3-coder:480b-cloud`. See all supported models [here](https://ollama.com/search)) 5. Adjust the `Context Window` to at least 32K tokens for coding tasks [source]
Ollama streams several API responses as newline-delimited JSON so callers can display tokens as they arrive. Streaming lowers perceived latency for long generations, but the client must parse multiple JSON objects instead of one response body. [source]
openai is not always desired, as some tools such as Continue from VS Code do not work correctly via openai API while via ollama api they do. [source]
3. Provide your **Ollama API URL** and select your models. <Note>If you're running Onyx in Docker, to access your computer's local network use `http://host.docker.internal` instead of `http://127.0.0.1`.</Note> [source]

Facts and statements

/api/generate and /api/chat stream by default; the stream is NDJSON served as application/x-ndjson, one JSON object per line. Setting "stream": false returns a single application/json object with the full text and done:true. [source]
Final-chunk usage fields: total_duration, load_duration, prompt_eval_count, prompt_eval_cached_count, prompt_eval_duration, eval_count, eval_duration. All durations are nanoseconds; counts are tokens. Usage appears only in the done:true chunk. [source]
Output tokens per second = eval_count / eval_duration * 10^9 (eval_duration is nanoseconds). The docs example 18 tokens in 52,479,709 ns is about 343 tokens/s. [source]
Certain API endpoints stream responses by default, such as `/api/generate`. These responses are provided in the newline-delimited JSON format (i.e. the `application/x-ndjson` content type). For example: [source]
client = anthropic.Anthropic( base_url='http://localhost:11434', api_key='ollama', ) [source]
const anthropic = new Anthropic({ baseURL: "http://localhost:11434", apiKey: "ollama", }); [source]
If you are using the API you can preload a model by sending the Ollama server an empty request. This works with both the `/api/generate` and `/api/chat` API endpoints. [source]
POST http://localhost:11434/api/chat [source]
* `total_duration`: How long the response took to generate * `load_duration`: How long the model took to load * `prompt_eval_count`: How many input tokens were in the prompt * `prompt_eval_cached_count`: How many prompt tokens were read from the cache * `prompt_eval_duration`: How long it took to evaluate the uncached prompt tokens * `eval_count`: How many output tokens were processes * `eval_duration`: How long it took to generate the output tokens [source]
Streaming is enabled by default through the REST API, but disabled by default in the SDKs. [source]
num_ctx (int): Context window size [source]
This loop streams the assistant response, accumulates partial fields, passes them back together, and appends the tool results so the model can complete its answer. [source]
Related: How to generate text with the Ollama API [source]
Related: How to run a chat request with the Ollama API [source]
Provide an `images` array. SDKs accept file paths, URLs or raw bytes while the REST API expects base64-encoded image data. [source]
1. Create an [API key](https://ollama.com/settings/keys) from ollama.com 2. Click on `Use custom base URL` and set it to `https://ollama.com` 3. Enter your **Ollama API Key** 4. Select a model from the list [source]
1. Create an [API key](https://ollama.com/settings/keys) from ollama.com 2. Select **Internet Hosted** and enter URL as `https://ollama.com` 3. Enter your **Ollama API Key** and click **Add** [source]
Ollama binds 127.0.0.1 port 11434 by default. Change the bind address with the `OLLAMA_HOST` environment variable. [source]
num_ctx 次のトークン生成に使うコンテキストウィンドウのサイズ int 2048 [source]
repeat_last_n 繰り返し防止のためにどこまで遡るか（0=無効、-1=num_ctx） int 64 [source]
Verify the client waits until the final done chunk arrives. $ python3 - <<'PY' last = {"done": True, "done_reason": "stop"} print(last["done"]) PY True [source]
export ANTHROPIC_AUTH_TOKEN=ollama export ANTHROPIC_BASE_URL=http://localhost:11434 [source]
config_list = [ { "model": "codellama", "base_url": "http://localhost:11434/v1", "api_key": "ollama", } ] [source]
For endpoints that return usage metrics, the response body will include the usage fields. For example, a non-streaming call to `/api/generate` may return the following response: [source]
> Note: this endpoint has been superseded by `/api/embed` [source]
repeat_last_nSets how far back for the model to look back to prevent repetition. (Default: 64, 0 = disabled, -1 = num_ctx)intrepeat_last_n 64 [source]
Why the num_ctx you already set did nothing [source]
prompt_eval_count in the API response. If your prompt is clearly 9,000 tokens and this comes back near your window size, the rest was discarded. [source]
The fix you should make even after fixing num_ctx [source]
I set num_ctx and nothing changed. Why? [source]
Tags:ollama num_ctxollama context windowollama truncatesagent ignores soul.mdollama context lengthOLLAMA_CONTEXT_LENGTHollama num_ctx not workingollama v1 endpoint num_ctxhermes ollama context [source]
curl -X POST http://localhost:11434/v1/responses \ -H "Content-Type: application/json" \ -d '{ "model": "qwen3:8b", "input": "Write a short poem about the color blue" }' [source]
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chat/ollama-chat.adoc [source]
@JsonProperty("keep_alive") String keepAlive, [source]
private static final List<String> NON_SUPPORTED_FIELDS = List.of("model", "format", "keep_alive", "truncate"); [source]
private static final List<String> NON_SUPPORTED_FIELDS = List.of("model", "format", "keep_alive", "truncate", [source]
/api/generate Generates a response for the provided prompt [source]
As a work-around, you could have the clients use the OpenAI API endpoint, or use a proxy to remove num_ctx from the request. [source]
indeed, I patched ollama sources to ignore num_ctx, but it would be nice, if supported directly by ollama. [source]
/openapi.yaml get /api/version Retrieve the version of the Ollama [source]
The local API at `http://localhost:11434` does not require authentication. [source]
/openapi.yaml post /api/embed Creates vector embeddings representing the input text [source]
/openapi.yaml post /api/generate Generates a response for the provided prompt [source]
/openapi.yaml get /api/ps Retrieve a list of models that are currently running [source]
/openapi.yaml get /api/tags Fetch a list of models and their details [source]
<Note> The `/api/embed` endpoint returns L2‑normalized (unit‑length) vectors. </Note> [source]
Select Ollama as the provider, use `http://localhost:11434` as the base URL if prompted, and choose a model such as `qwen3.5` or `kimi-k2.6:cloud`. [source]
2. Find **Ollama**, click **Configure** 3. Confirm **API Host** is `http://localhost:11434` and click Submit [source]
1. **Install** — If Hermes isn't installed, Ollama prompts to install the Hermes command-line agent 2. **Model** — Pick a model from the selector (local or cloud). See currently-supported models [here](https://ollama.com/search). 3. **Onboarding** — Ollama configures the Ollama provider, points Hermes at `http://127.0.0.1:11434/v1`, and sets your model as the primary 4. **Gateway** — Optionally connects a messaging platform (Telegram, Discord, Slack, WhatsApp, Signal, Email) and launches the Hermes chat [source]
``` API base URL [e.g. https://api.example.com/v1]: http://127.0.0.1:11434/v1 ``` [source]
``` Verified endpoint via http://127.0.0.1:11434/v1/models (1 model(s) visible) Detected model: kimi-k2.5:cloud Use this model? [Y/n]: ``` [source]
3. Under **Third Party AI Providers**, choose **Ollama** 4. Confirm the **Host URL** is `http://localhost:11434`, then click **Ok** 5. Once connected, select a model under **Local models by Ollama** [source]
1. In marimo, go to the user settings and go to the AI tab. From here you can find and configure Ollama as an AI provider. For local use you would typically point the base url to `http://localhost:11434/v1`. [source]
3. Confirm Base URL is set to `http://localhost:11434` if running locally or `http://host.docker.internal:11434` if running through docker and click **Save** [source]
The extension discovers models from `http://127.0.0.1:11434` by default. [source]
2. Select **Locally Hosted**, enter port **11434** and click **Add** [source]
2. Under **LLM Providers**, choose **Ollama** 3. Confirm the **Host URL** is `http://localhost:11434`, then click **Connect** 4. Once connected, select a model under **Ollama** [source]
Ollama runs as a native Windows application, including NVIDIA and AMD Radeon GPU support. After installing Ollama for Windows, Ollama will run in the background and the `ollama` command line is available in `cmd`, `powershell` or your favorite terminal application. As usual the Ollama [API](/api) will be served on `http://localhost:11434`. [source]

Related concepts

localhost:11434 — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
stream — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
eval_count — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
/api/generate — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
num_ctx — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
prompt_eval_duration — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
NDJSON — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
/api/chat — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
keep_alive — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
done_reason — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
tool_calls — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
/api/embed — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
context window — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
/api/tags — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
/api/ps — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
/api/version — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
/api/delete — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
/api/copy — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage
/api/push — is a part of Ollama REST API: generate and chat with streaming NDJSON and errors and usage

Children

No children recorded.

← the whole tree · 3D view· how to read this page