LangChain and LlamaIndex integrations with Ollama

Parent: Ollama integrations and ecosystem · Published reference · snapshot 2026-10-14 · skill ollama-expert/references/ollama-integrations-ecosystem.md

↓ Facts as markdown all context files

Everything the scanned sources say about LangChain and LlamaIndex integrations with Ollama: 37 source-anchored units across 5 sources, grouped by facet.

These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.

Definitions

LangChain's Python Ollama chat integration is the langchain-ollama package; ChatOllama supports bind_tools tool calling, image input, token streaming, native async and logprobs, and the docs example uses validate_model_on_init=True. [source]

How it works

In LlamaIndex's Ollama LLM, structured_predict sets the response format to the Pydantic class's JSON schema (overriding json_mode); tool calls are limited to one per response unless allow_parallel_tool_calls is set; tool IDs equal the tool name because Ollama supplies none; additional_kwargs merge after defaults and override temperature/num_ctx. [source]

How-to and procedures

Install langchain-ollama and download any models you want to use from ollama. [source]
Use the same model tag in `ollama pull` and `ChatOllama(model=...)`. The image, log probability, and custom-role examples below include separate pull commands for their models. [source]
To access Granite 3.2's thinking features, pass a message with a `"control"` role with content set to `"thinking"`. Because `"control"` is a non-standard message role, use a [ChatMessage](https://reference.langchain.com/python/langchain-core/messages/chat/ChatMessage) object to implement it: [source]

Examples and snippets

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} — `from typing import List` … [source]
```python — `from llama_index.llms.ollama import Ollama` … [source]
```bash theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} — `pip install -qU langchain-ollama` … [source]
```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} — `from langchain_ollama import ChatOllama` … [source]
```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} — `from langchain.messages import HumanMessage` … [source]
```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} — `from langchain_ollama import ChatOllama` … [source]
```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} — `llm = ChatOllama(model="llama3.1", logprobs=True, top_logprobs=3)` … [source]
```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}} — `from langchain.messages import HumanMessage` … [source]

Problems, failure modes and limitations

Third-party guides report Ollama truncates over-long prompts silently, dropping early turns or the prompt prefix, which can remove retrieved RAG chunks or system instructions without any error. [source]

Facts and statements

LlamaIndex's Ollama LLM class (llama-index-llms-ollama) defaults base_url to http://localhost:11434, takes context_window=-1 (read from model info via the client, else a DEFAULT_CONTEXT_WINDOW constant), and has a thinking option (bool or low/medium/high) and a json_mode flag. [source]
Context-length trap: Ollama's server default context depends on VRAM (under 24 GiB: 4k; 24-48 GiB: 32k; 48 GiB or more: 256k) and its docs recommend at least 64000 tokens for agents, web search and coding tools. Set num_ctx in ChatOllama or context_window/additional_kwargs in LlamaIndex, or OLLAMA_CONTEXT_LENGTH on the server, and check with ollama ps. [source]
ChatOllama has a reasoning parameter: True returns reasoning separately in additional_kwargs reasoning_content, False disables it, None (default) leaves model behavior so think tags may appear in the main content. It can be set per call. [source]
LlamaIndex embeds via a separate package, llama-index-embeddings-ollama (OllamaEmbedding, subclass of BaseEmbedding); LangChain uses OllamaEmbeddings inside langchain-ollama. The LlamaIndex docs example uses embeddinggemma (768-dim vectors). [source]
The LangChain Ollama integration lives in the `langchain-ollama` package: [source]
def __init__( self, model: str, base_url: str = "http://localhost:11434", temperature: Optional[float] = None, context_window: int = -1, request_timeout: Optional[float] = DEFAULT_REQUEST_TIMEOUT, prompt_key: str = "prompt", json_mode: bool = False, additional_kwargs: Optional[Dict[str, Any]] = None, client: Optional[Client] = None, async_client: Optional[AsyncClient] = None, is_function_calling_model: bool = True, keep_alive: Optional[Union[float, str]] = None, thinking: Optional[Union[bool, Literal["low", "medium", "high"]]] = None, headers: Optional[Dict[str, str]] = None, **kwargs: Any, ) [source]
from llama_index.llms.ollama import Ollama [source]
Source code in llama-index-integrations/llms/llama-index-llms-ollama/llama_index/llms/ollama/base.py [source]
base_url: str = Field( default="http://localhost:11434", description="Base url the model is hosted under.", ) model: str = Field(description="The Ollama model to use.") temperature: Optional[float] = Field( default=None, description="The temperature to use for sampling.", ) context_window: int = Field( default=-1, description="The maximum number of context tokens for the model.", ) request_timeout: float = Field( default=DEFAULT_REQUEST_TIMEOUT, description="The timeout for making http request to Ollama API server", ) prompt_key: str = Field( default="prompt", description="The key to use for t [source]
ollama pull gpt-oss:20b pip install -U langchain-ollama [source]
from llama_index.embeddings.ollama import OllamaEmbedding [source]
Examples: `pip install llama-index-llms-ollama` [source]
_client: Optional[Client] = PrivateAttr() _async_client: Optional[AsyncClient] = PrivateAttr() [source]
@property def async_client(self) -> AsyncClient: if self._async_client is None: self._async_client = AsyncClient( host=self.base_url, timeout=self.request_timeout, headers=self.headers or {}, ) return self._async_client [source]
Trace LangChain applications (Python and JS/TS) [source]
model = ChatOllama( model="gpt-oss:20b", validate_model_on_init=True, temperature=0.8, num_predict=256, [source]
json_model = ChatOllama(format="json") json_model.invoke( "Return a query for the weather in a random location and time of day with two keys: location and time_of_day. " "Respond using JSON only." ).content [source]
from langchain_ollama import ChatOllama from pydantic import BaseModel, Field [source]
model = ChatOllama( model="deepseek-r1:8b", validate_model_on_init=True, reasoning=True, ) [source]
> Integrate with the ChatOllama chat model using LangChain Python. [source]
For detailed documentation of all features and configuration options, head to the [`ChatOllama`](https://reference.langchain.com/python/langchain-ollama/chat_models/ChatOllama) API reference. </Tip> [source]
`ChatOllama` supports token-level log probabilities via the `logprobs` and `top_logprobs` parameters. Log probabilities indicate how likely each token was at each generation step. [source]
For detailed documentation of all `ChatOllama` features and configuration options, see the [API reference](https://reference.langchain.com/python/langchain-ollama/chat_models/ChatOllama). [source]

Related concepts

LangChain — is a instance of LangChain and LlamaIndex integrations with Ollama
ollama — is a related of LangChain and LlamaIndex integrations with Ollama
LlamaIndex — is a instance of LangChain and LlamaIndex integrations with Ollama
tool calling — is a dependent of LangChain and LlamaIndex integrations with Ollama
ollama-python — is a instance of LangChain and LlamaIndex integrations with Ollama
localhost — is a problem of LangChain and LlamaIndex integrations with Ollama
num_ctx — is a problem of LangChain and LlamaIndex integrations with Ollama

Children

No children recorded.

← the whole tree · 3D view· how to read this page