[Bug]: Hermes Agent + Ollama Local — Hangs indefinitely with tool definitions

[t1t4nium]

Bug Description

This report was generated with Claude.ai's help.

Date: May 13–14, 2026
Environment: GEEKOM A5 Mini PC, Debian 13 Trixie, Linux 6.12, Python 3.11.15
Objective: Run Hermes Agent with a local LLM via Ollama, accessible through Telegram gateway

---

Environment

Component	Version
Hermes Agent	v0.13.0 (2026.5.7)
Ollama	v0.23.2
LiteLLM (proxy)	v1.84.0
Model	mistral-small:22b (Q4_0, 13 GB, 128K native context)
OpenAI SDK	2.24.0
Python	3.11.15
Hardware	AMD Ryzen 5 7430U, 64 GB DDR4, CPU-only inference

---

Summary

Hermes Agent never produces a response when using a local Ollama backend, regardless of configuration. The root cause is a combination of two distinct bugs:

1. Ollama bug: stream=true + N tool definitions causes the Ollama API to hang indefinitely (N somewhere between 3 and 25 tools)
2. Hermes bug: pre-flight detection of Ollama-native routes is performed regardless of api_mode, blocking any intermediate proxy that doesn't expose those routes

---

Symptom

When a message is sent to Hermes via Telegram or TUI with a local Ollama custom endpoint configured:

1. Hermes correctly loads the model (confirmed via ollama ps)
2. No response ever arrives — the agent hangs indefinitely
3. The Ollama API becomes completely unresponsive until restarted

---

Investigation

Step 1: Confirming Ollama receives the request

Monitoring journalctl -u ollama while sending a message revealed:

[GIN] 2026/05/14 - 10:52:08 | 500 | 1m17s | POST "/v1/chat/completions"

Ollama received the request but returned HTTP 500 after 1 minute 17 seconds.

Step 2: Isolating the bug via curl

Systematic testing of the /v1/chat/completions endpoint:

Test	Parameters	Result
/api/generate	simple prompt	✅ OK
/v1/chat/completions	stream=false, 0 tools	✅ OK
/v1/chat/completions	stream=true, 0 tools	✅ OK
/v1/chat/completions	stream=false, 3 tools	✅ OK
/v1/chat/completions	stream=true, 3 tools	❌ Hangs, 0 bytes
/v1/chat/completions	stream=false, 25 tools	✅ OK (~12s)
/v1/chat/completions	stream=true, 25 tools	❌ HTTP 500 after 1m17s

Conclusion: Ollama hangs on stream=true + tools. The threshold is somewhere between 3 and 25 tools.

The entire Ollama API becomes unresponsive after the hang — even /api/generate stops responding until the service is restarted.

Step 3: Confirming Hermes always uses stream=true

Hermes Agent sends stream=True for all LLM calls (hardcoded in run_agent.py). The display.streaming: false config option only disables token display, not HTTP streaming. With ~25 native tools sent by Hermes, the Ollama bug is triggered on every single request.

Step 4: LiteLLM proxy as workaround

LiteLLM was configured as a proxy between Hermes and Ollama, with stream: false forced server-side:

model_list:
  - model_name: mistral-small:22b-64k
    litellm_params:
      model: ollama_chat/mistral-small:22b-64k
      api_base: http://127.0.0.1:11434
      stream: false

curl tests via LiteLLM (port 4000):

Test	Result
stream=true, 2 tools	✅ SSE chunks received
stream=true, 3 tools	✅ SSE chunks received
stream=true, 25 tools	✅ SSE chunks received
stream=false, 25 tools	✅ Full JSON response

LiteLLM successfully works around the Ollama bug by converting requests to stream=false and simulating SSE streaming back to the client (fake streaming).

However, Hermes still does not respond when pointed at LiteLLM.

Step 5: Identifying the second bug — Hermes pre-flight detection

Monitoring LiteLLM logs while Hermes sends a message revealed the following sequence:

GET /api/v1/models → 404 Not Found
GET /api/tags      → 404 Not Found
POST /api/show     → 404 Not Found
GET /v1/props      → 404 Not Found
GET /props         → 404 Not Found
GET /version       → 404 Not Found
GET /v1/models     → 200 OK
[no POST /v1/chat/completions follows]

Hermes performs a pre-flight detection of Ollama-native routes before sending any generation request. When these routes return 404 (LiteLLM doesn't expose them), Hermes aborts without ever sending the actual completion request.

This detection occurs regardless of api_mode: chat_completions being set in config.yaml.

When pointed directly at Ollama (which does expose these routes), the Ollama stream+tools bug is triggered instead.

---

Tested Configurations

Config 1: Hermes → Ollama direct

model:
  base_url: http://127.0.0.1:11434/v1
  provider: custom
  api_mode: chat_completions
  context_length: 65536

Result: ❌ HTTP 500 after ~1m17s (stream=true + 25 tools bug)

Config 2: Hermes → LiteLLM → Ollama

model:
  base_url: http://127.0.0.1:4000/v1
  provider: custom
  api_mode: chat_completions
  context_length: 65536

Result: ❌ Hermes aborts on 404s from pre-flight route detection

Config 3: Hermes → Ollama, provider "LM Studio"

Selected "LM Studio" as provider type in Hermes model picker, pointing to LiteLLM at port 4000.

Result: ❌ Same pre-flight detection behavior, same 404 aborts

Config 4: api_mode: chat_completions

Explicitly set api_mode: chat_completions in config.yaml.

Result: ❌ Pre-flight detection of Ollama-native routes persists regardless

---

Additional Findings

Ollama context window behavior

Ollama applies a default context depending on available RAM:

Available RAM	Default context
< 24 GB	4,096 tokens
24–48 GB	32,768 tokens
48+ GB	256,000 tokens

With 64 GB RAM, Ollama still defaulted to 4,096 tokens. Fix:

sudo systemctl edit ollama.service
# Add: Environment="OLLAMA_CONTEXT_LENGTH=65536"

Qwen2.5 models context issue

Qwen2.5 models packaged by Ollama have their context hard-capped at 32,768 tokens in the GGUF file itself (no YaRN config). Neither OLLAMA_CONTEXT_LENGTH nor Modelfile num_ctx overrides are respected at runtime — ollama ps consistently shows 32,768 regardless of configuration.

This affects: qwen2.5:7b, qwen2.5:14b, qwen2.5:32b.

Models confirmed to work with native 128K context: mistral-small:22b, llama3.3:70b, llama3.1:8b.

LiteLLM notes

• Must install with pip install 'litellm[proxy]' — base package missing websockets dependency
• master_key: null in config causes abnormal behavior (hanging with 2 tools in stream mode) — omit entirely
• Without explicit api_base in config, LiteLLM attempts to spawn its own Ollama process, creating zombie processes

---

Expected Behavior

When api_mode: chat_completions is configured, Hermes should:

1. Skip Ollama-native route detection entirely
2. Send requests directly to /v1/chat/completions
3. Work correctly with any OpenAI-compatible proxy (LiteLLM, vLLM, etc.)

Actual Behavior

Hermes performs Ollama-native route detection (/api/tags, /api/show, /props, /version) regardless of api_mode, and aborts when these routes return 404.

---

Referenced Issues

• Hermes Agent Optimize Ollama Integration: Native /api/chat vs OpenAI-Compatible Endpoint #4505: "Ollama /v1 breaks tool calling — use native /api/chat instead"
• Ollama docs: fix stale and incorrect documentation across 18 files #2805: "Ollama gets stuck in infinite loop and locks API"

---

Proposed Fix

Option 1 (Hermes side — recommended): When api_mode: chat_completions is explicitly set, skip all Ollama-native route pre-flight detection. This would make Hermes compatible with any OpenAI-compatible proxy without requiring those routes.

Option 2 (Hermes side): Add a skip_provider_detection: true config option to allow users to bypass pre-flight detection entirely.

Option 3 (Ollama side): Fix the stream=true + tools hang in /v1/chat/completions. This is tracked in Ollama #2805.

---

Workaround (Partial)

No complete workaround found. LiteLLM successfully bypasses the Ollama stream+tools bug, but Hermes's pre-flight detection prevents it from being used as a proxy.

The only currently working configuration is using a cloud provider (Nous Portal, OpenRouter, etc.).

Steps to Reproduce

1. Install Ollama v0.23.2 and pull any model with tool calling support
   (tested: mistral-small:22b, llama3.3:70b)

2. Set OLLAMA_CONTEXT_LENGTH=65536 in the Ollama systemd service

3. Configure Hermes with a custom endpoint pointing to Ollama:
   base_url: http://127.0.0.1:11434/v1
   provider: custom
   api_mode: chat_completions
   context_length: 65536

4. Start the Telegram gateway (or use --tui)

5. Send any message (e.g. "Hello")

6. Observe: no response ever arrives

Expected Behavior

Hermes sends a POST /v1/chat/completions request to Ollama and
receives a streamed response. The user receives a reply via
Telegram or TUI within a reasonable time.

Actual Behavior

Two distinct failure modes observed depending on configuration:

--- Scenario A: Hermes → Ollama direct ---

Ollama receives the POST /v1/chat/completions with stream=true
and ~25 tool definitions, then hangs for 1m17s before returning
HTTP 500. The entire Ollama API becomes unresponsive until the
service is restarted.

Ollama log:
[GIN] 2026/05/14 | 500 | 1m17s | POST "/v1/chat/completions"

Root cause confirmed via curl:
stream=false + 25 tools → HTTP 200 OK ✅
stream=true + 25 tools → hangs indefinitely, then HTTP 500 ❌

Hermes always sends stream=True (hardcoded). With ~25 native tools
in the payload, the Ollama bug is triggered on every request.

--- Scenario B: Hermes → LiteLLM proxy → Ollama ---

LiteLLM successfully works around the Ollama stream+tools bug
(confirmed via curl: stream=true + 25 tools → SSE chunks received).

However, before sending any POST /v1/chat/completions, Hermes
performs a pre-flight detection of Ollama-native routes:

GET /api/v1/models → 404
GET /api/tags → 404
POST /api/show → 404
GET /v1/props → 404
GET /props → 404
GET /version → 404
GET /v1/models → 200

Hermes aborts after the 404s — no POST /v1/chat/completions
is ever sent. This behavior persists even with api_mode:
chat_completions explicitly set in config.yaml.

Affected Component

Agent Core (conversation loop, context compression, memory)

Messaging Platform (if gateway-related)

Telegram

Debug Report

Report       https://paste.rs/5DfXu
agent.log    https://paste.rs/p50Rg
gateway.log  https://paste.rs/F2l9Z

Operating System

Debian 13 Trixie, Linux 6.12

Python Version

Python 3.11.15

Hermes Version

Hermes Agent v0.13.0 (2026.5.7) Project: /home/hermes/.hermes/hermes-agent Python: 3.11.15 OpenAI SDK: 2.24.0 Up to date

Additional Logs / Traceback (optional)

Root Cause Analysis (optional)

No response

Proposed Fix (optional)

No response

Are you willing to submit a PR for this?

• I'd like to fix this myself and submit a PR

[usesben]

Reproduced this independently with a different setup. Adding data points that may help narrow the root cause:

• Hangs even with just 1 tool definition (not just the 3-25 range reported above) — model: qwen2.5 (64k context variant), served via Ollama, Hermes 0.16.0, provider: custom.

• Confirmed via direct curl: non-streaming /v1/chat/completions returns a perfectly valid, spec-conformant tool_calls response. Streaming mode sends the complete tool call in one chunk (full id+name+arguments at once, finish_reason: null), followed by a separate final chunk with finish_reason: "tool_calls" but no tool_calls data — i.e. Ollama doesn't incrementally split arguments across chunks the way OpenAI's real API does.

• Tested both api_mode: chat_completions (explicit) and skip_provider_detection: true in 0.16.0 — neither resolves the hang when pointed directly at Ollama (no proxy in between). Matches the issue's own conclusion that no complete workaround currently exists.

• GPU utilization confirms the model is actively generating during the hang (not stalled) — periodic ListToolsRequest reprobing every ~180s (matches gateway_notify_interval) while waiting, no CallToolRequest ever issued.

[konsisumer]

These issues look like facets of one underlying problem: #25629, #49297, #49983.

They were grouped by a semantic pass as different symptoms of one underlying problem in comp/agent. A single coordinated fix may resolve them together — flagging for visibility; no action has been taken on the individual issues.

Automated root-cause grouping. Reply to correct if these are unrelated.