Requests to local Ollama models hang indefinitely with full CLI toolset (works via raw ollama run and cloud providers)

[oleg-koval]

Requests to local Ollama models hang indefinitely with the full CLI toolset enabled (works fine via raw ollama run and via cloud providers)

Environment

• Hermes Agent: v0.17.0 (2026.6.19), upstream 190e1ffa
• Python: 3.11.15
• OpenAI SDK: 2.24.0
• OS: macOS 26.4.1 (build 25E253), Apple M1 Max, 32GB RAM
• Ollama: 0.30.10, server running locally on http://localhost:11434
• cli platform toolset: 28 entries configured (browser, clarify, code_execution, computer_use, context_engine, cronjob, delegation, file, google_meet, homeassistant, image_gen, kanban, memory, messaging, moa, rl, session_search, skills, spotify, terminal, todo, tts, video, video_gen, vision, web, x_search, yuanbao) — note messaging, moa, rl log as "Unknown toolset" at startup and are presumably no-ops, so effectively ~25 toolsets load.

Summary

Any local Ollama model, run through hermes chat with the standard cli toolset, hangs indefinitely (no output, no error, no timeout) once the agent loop begins. The exact same model responds in seconds via raw ollama run, and the exact same toolset works fine against a cloud model (gpt-5.4-mini via openai-codex). This reproduces across three different local models spanning different vendors/sizes, so it doesn't look like a model-quality/tool-calling-format issue — it looks like something in the local/Ollama transport path itself.

Steps to reproduce

# 1. Any locally-pulled Ollama model registered under providers.ollama in config.yaml, e.g.:
ollama pull gemma4:e4b        # or llama3.1:8b, or mistral-small

# 2. Confirm the model works fine directly via Ollama (fast, single-digit seconds)
time ollama run gemma4:e4b "reply with exactly: pong"

# 3. Run the identical prompt through hermes chat with the default full `cli` toolset
time hermes chat -Q -m "gemma4:e4b" --provider ollama -q "reply with exactly the word: pong"
# hangs indefinitely — no output, no error, doesn't return even after 5 minutes

Relevant ~/.hermes/config.yaml fragment (providers: and top-level model:):

model:
  default: gpt-5.4-mini
  provider: openai-codex
  base_url: https://chatgpt.com/backend-api/codex
  ollama_num_ctx: 65536
  context_length: 65536
providers:
  ollama:
    base_url: http://localhost:11434/v1
    default_model: "gemma4:e4b"
    models:
      "gemma4:e4b": {}
      "llama3.1:8b": {}

Expected behavior

hermes chat -m <local-ollama-model> --provider ollama -q "..." returns a response within roughly the same order of magnitude as raw ollama run (single-digit to low-double-digit seconds for a trivial prompt), or fails with a clear error.

Actual behavior

The process hangs with zero output until manually killed. Verified across three different local models:

Model	Vendor/notes	Raw ollama run <model> "reply with exactly: pong"	hermes chat -m <model> --provider ollama -q "..." (full cli toolset)
gemma4:e4b	Google, 9.6GB	~14s ✅	Hung, no output after 5 min (killed) ❌
llama3.1:8b	Meta, 4.9GB	~6s ✅	Hung, no output after 90s+ (killed) ❌
mistral-small	Mistral, 14GB, marketed for reliable function-calling	~18s ✅	Hung, no output after 120s (killed) ❌
gpt-5.4-mini (control)	OpenAI via openai-codex provider, same cli toolset	—	~13.5s ✅

Diagnostics collected

• ollama ps sampled ~15–20s into a hung request shows the model fully resident and idle-ready, not computing:

  NAME                    ID              SIZE     PROCESSOR    CONTEXT    UNTIL
  mistral-small:latest    8039dd90c113    19 GB    100% GPU     32768      4 minutes from now
  

  No CPU/swap thrashing, no sign of active generation — the model is loaded and waiting, but Hermes never appears to receive/process a completion.
• hermes chat --verbose shows the agent initializes fully (all ~25 available toolsets enabled, memory provider activated, context compressor initialized) and then produces no further log lines before the hang — the stall happens after initialization, seemingly during or after the first request to Ollama's /v1/chat/completions.
• Ran into two related-but-distinct config foot-guns while isolating this, both worth calling out separately since they may confuse other users hitting the same hang:
  1. Hermes auto-requests num_ctx equal to the model's full advertised context (e.g. 131072 for a 128K model) on every call unless model.ollama_num_ctx is set — expensive locally, though not itself the cause of the hang (still hangs at any num_ctx value tested: 32768, 65536).
  2. Hermes enforces a hard 64,000-token minimum context for its default toolset size and will refuse to initialize below that (clear, fast error) — but the check compares against what Ollama's /api/show reports, not the model's true architecture max, so e.g. mistral-small (128K true context) gets rejected at its Ollama-reported 32,768 until model.context_length is explicitly overridden in config.

Hypothesis

Given the hang reproduces identically across three unrelated model families/vendors, and raw Ollama inference is fast for all three, the issue is very unlikely to be model tool-calling quality. It looks specific to something in Hermes's local/custom (Ollama) provider transport — possibly related to how the large tool-schema payload (~25 toolsets) is sent/streamed to a local OpenAI-compatible endpoint, since the same payload size works fine against openai-codex.

Happy to provide --verbose full logs, HERMES_DEBUG-style output, or test additional isolation steps (e.g. trimmed toolset counts, curl-level request capture against localhost:11434) if useful for triage.

[alt-glitch]

This was generated by AI during triage.

Duplicate of #25629 (bug 1: local Ollama hangs with stream=true + N tool definitions). The hang here reproduces after full init during the first /v1/chat/completions when the ~25-toolset cli schema payload is sent to the local OpenAI-compatible endpoint; the same toolset works against a cloud provider and the same model is fast via raw ollama run -- the payload-size mechanism matches #25629. Also related to #26339 (76KB payload -> Ollama TCP reset, same root cause). Workarounds: trim toolsets, use a cloud provider, or a non-streaming/direct endpoint. The num_ctx auto-request and the 64K-min context check vs /api/show-reported context are the same context-detection facet tracked in that cluster.