Skip to content

Ollama

Ollama exposes both its native API and an OpenAI-compatible shim. The shim was added late in the project's history and has caught up gradually.

Surface (current)

Endpoint Status Notes
/v1/models ✅ Maps to ollama list
/v1/models/{id} ✅ Returns the model object
/v1/chat/completions ✅ Tools, JSON mode
/v1/chat/completions (stream) ✅ [DONE] sentinel emitted
/v1/completions ⚠️ Routes through chat under the hood; finish_reason mapping is approximate
/v1/embeddings ✅ /api/embed is preferred
/v1/audio/* ❌ Not implemented
/v1/images/* ❌ Not implemented (Ollama is text/vision only)

Common deviations

  • Tool-call argument JSON drift. For models without strong tool-use training, Ollama's shim sometimes returns tool_calls[].function.arguments as a parsed object instead of a JSON-encoded string. WARN.
  • Inference parameters. Ollama accepts many OpenAI params (temperature, top_p, seed) but silently caps max_tokens against its own num_predict. Probes that ask for more than the model's default num_predict may get fewer tokens than requested.
  • response_format: json_schema is not honored — Ollama ignores the schema and falls back to a generic JSON-mode prompt.

Quirks worth knowing

  • Native API is richer. If you can use /api/chat and /api/embed, you get features (model pulling, parameter passing via Modelfile) that don't have OpenAI analogs.
  • Concurrency. Ollama serializes requests per loaded model. The prober makes one shared discovery call and at most two requests per selected endpoint, but concurrent pytest runs against the same Ollama instance will queue.

See also