Run a Local Agent Model with Ollama

Ollama offers a local model service with an HTTP chat interface and optional tool calls. Choose a checkpoint tag the Ollama library actually publishes; an OMW source-repository entry is not proof that Ollama serves it.

Four steps to a reproducible deployment

Install and verify
Follow Ollama's installation instructions for your OS and confirm its local service responds. Do not expose its API to public networks.
Select the exact model
Use the Ollama library to confirm tag, quantization, license and template. A Hugging Face checkpoint cannot be assumed to have a compatible Ollama build.
Test the interface
Send a harmless chat request to the local API. For tools, provide a JSON schema, validate the response, and execute only allowlisted functions.
Hardening
Bind locally, apply authorization at any gateway, cap context and request size, and disable unwanted outbound dependencies.

Minimal acceptance tests

Record the repository revision, artifact checksum, runtime version and command line in a deployment manifest. Verify ordinary chat, a negative permission test, malformed tool arguments and model recovery. Measure actual memory and latency on the target hardware rather than extrapolating from a weight-only estimate.

No specific one of the 28 curated checkpoints is certified to run in this stack by this page. See the agent model explorer for source-linked candidates.

COMMON QUESTIONS

Frequently asked questions

Can any listed checkpoint be deployed with Ollama?

Not necessarily. Runtime compatibility, weight format and chat templates must be checked for the exact checkpoint.

Does local inference make tool execution safe?

No. Tool permissions, sandboxing, network policy, logging and human approval remain application responsibilities.

READ THE ORIGINALS

Documentation and primary sources

COPYABLE LOCAL RECIPE · NOT YET REPRODUCED BY OMW

Example setup and acceptance check

Ollama's official catalog currently lists gpt-oss:120b as an MXFP4 120B artifact. The runtime artifact is not identical to the original Hugging Face repository files; check its revision and license. This is a documented recipe, not an OMW run.

1. Start the documented local service

ollama pull gpt-oss:120b
ollama show gpt-oss:120b
ollama list

2. Request one harmless tool call

This request declares a deterministic integer-sum tool; it does not execute that tool. Review the model response for the correct name and arguments.

curl -fsS -X POST 'http://127.0.0.1:11434/api/chat' \
  -H 'Content-Type: application/json' \
  --data-binary '{"model":"gpt-oss:120b","stream":false,"messages":[{"role":"user","content":"Please call sum_numbers with a=13 and b=29. Do not calculate directly."}],"tools":[{"type":"function","function":{"name":"sum_numbers","description":"Return the sum of two integers.","parameters":{"type":"object","properties":{"a":{"type":"integer"},"b":{"type":"integer"}},"required":["a","b"],"additionalProperties":false}}}]}'

3. Run a bounded, local-only evaluator

From a checkout of our repository, the following CLI verifies tool-call structure and records raw outcomes from a local endpoint. Its self-test validates the parser only. A live run requires the model server above; none was performed during publication.

python3 scripts/agent_tool_call_eval.py --self-test
python3 scripts/agent_tool_call_eval.py --endpoint http://127.0.0.1:11434/api/chat --model gpt-oss:120b --output /tmp/agent-tool-calls.json

The Ollama listing reports roughly 65 GB of model download. This is an artifact size, not a measured RAM/VRAM or throughput guarantee. Tool execution remains entirely in your application.

Check the provider's exact runtime instructions ↗

Minimum acceptance checklist

Record model artifact digest and license, runtime release, template/parser, sampled output, rejection of unexpected tool names/arguments, elapsed time, actual peak RAM/VRAM and all failures. Keep tool execution sandboxed and require human approval for side effects.

Evaluate before deployment.

Model metadata is source-linked; agent task success, tool-call reliability and production readiness require separate tests.

Review the methodology → Return to the explorer →