llama-server router mode child instance exposure and auth

Parent: Mac local LLMs: Serving ops and multi-model · Published reference · snapshot 2026-10-05

The router forwards each request to the child that serves the `model` named in the body or query. The router also makes two internal calls to children (`POST /v1/streams/lookup` and `DELETE /v1/stream`) that carry no key.

These notes link each claim to its source. A source may be a research report hosted on this site rather than the primary document. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.

Facts

The router forwards each request to the child that serves the `model` named in the body or query. The router also makes two internal calls to children (`POST /v1/streams/lookup` and `DELETE /v1/stream`) that carry no key. [source]
Children inherit the router's command-line arguments and environment; the router removes or overwrites host, port, API key, HF repo and model alias when it spawns them. [source]
15 Sep 2026: a contributor ran a before/after matrix on PR 28938 and found the fix removes the child 401s, with the trade that a child port is reachable without a key. [source]
Before the fix, with `--api-key-file` only, a direct call to a child's `/v1/streams/lookup` returned 401 and `DELETE /v1/stream` returned 401, so the router's own internal stream calls could 401. [source]
A contributor proposed rejecting `--api-key` and `--api-key-file` when both are passed, since he saw no case needing both. [source]
Whether children bind loopback regardless of the router's `--host`; no cached source states the child bind address. [source]
RESOLVES the open question in service-accounts-and-api-key-hardening-for-lan-e.md ("whether the LAN-exposed router still accepts unauthenticated direct requests on child ports"): after PR 28938 a child port is reachable without a key, marked "by design" in a tested matrix. [source]
With `--api-key` alone (no key file), a direct `GET /v1/models` on a child returned 200 without a key both before and after the change, because the CLI key never reached the child; the matrix author labels this "pre-existing". [source]
With `--api-key-file` alone, a direct child `GET /v1/models` returned 401 before the change and 200 after it. [source]
Before the change, with `--api-key-file`, direct child calls to `POST /v1/streams/lookup` and `DELETE /v1/stream` returned 401; after it they returned 200 and 204. [source]
A key set in a preset INI model section was forwarded to the child before the change and stripped after it. [source]
After the change the router logs a CORS warning for children; the matrix author calls it cosmetic. [source]
The server README says host, port, API key, HF repo and model alias are controlled by the router and are removed or overwritten when a preset model loads. [source]
The server README says router model instances inherit both command-line arguments and environment variables from the router. [source]
Practical consequence: with a router bound to a LAN address, authentication protects only the router port; anyone who can reach a child port can call it with no key (inferred from the matrix; child bind address untested). [source]

Children

No children recorded.

← the whole tree · 3D view· how to read this page