LiteLLM enabling in-depth analysis of agentic behaviours

[Cletus-Mccoy]

On a fresh chat with no history, three agents sent my LiteLLM gateway between 63K and 191K characters of prompt before I typed a word. Most of it was tool schemas the agent had never called.

I run several self-hosted agents (OpenClaw, Hermes, Agent Zero) against local models, all through one LiteLLM gateway that also fronts a set of MCP servers. Prefill on local hardware is the expensive part: a big static prefix means slow first tokens, a thrashed KV cache, and a hot CPU. So I stopped guessing what was in the prompt and measured it.

How I measured

1. Run tcpdump on the gateway host, filtered to the gateway port.
2. Send one trivial message to each agent in a fresh chat.
3. Parse the captured request bodies with a small script and count characters per part: system prompt, native tools, MCP tools.
4. Delete the capture afterwards. It contains your prompts.

For real usage I did not trust the agents' own accounting. LiteLLM's tool tables (LiteLLM_ToolTable, LiteLLM_DailyToolSpend) show what was called through the gateway, and Hermes keeps every tool call in its own session database.

What a fresh chat costs

Agent | First request (chars) | System prompt | Native tools | MCP tools -- | -- | -- | -- | -- OpenClaw | 164K | 31.5K | 44 tools, 84K | 45 tools, 52K Hermes | 191K | 22K (12K is a skills list) | 31 tools, 62K | 99 tools, 108K Agent Zero | 63K | 43K (tool docs as text) | none | 16K extras

In tokens, a fresh chat measured about 14K for Agent Zero, about 42K for OpenClaw, and 50–95K for Hermes. The top Hermes figure included stored history from earlier agent-to-agent calls.

Two MCP servers dominate: one note-taking server alone is 36–38K characters in both OpenClaw and Hermes, and a workflow server adds another 34K in Hermes.

A suspect that was innocent

I expected outputSchema to be the bloat. It is not. None of the three agents sends it, which I confirmed in real request bodies and in Hermes's code. Stripping it from your tool definitions saves nothing. Check the wire before you optimise.

What the agents actually use

This is the part that changes the design. Over October 7–31, Hermes made 3,463 tool calls:

• Native tools: 81% (the terminal alone was 48%)
• Task board: 15.6%
• Homelab status: 2.2%
• Memory: 0.8%
• Deploy history: 0.2%
• Zero calls to the note-taking, workflow, trading, graph, search or skills-catalog servers

Those zero-call servers make up about 90K characters, 47% of Hermes's request. Agent Zero uses roughly 12 of 91 MCP tools.

Why I did not just turn tools off

The obvious fix is a deny list. I rejected it. The point of these agents is capability, and the bloat is only the cost of paying for schemas on turns that don't need them. The design is:

1. A small eager core chosen from logged usage, not from my guess.
2. A cheap generated index: one line per remaining tool, so the agent knows what exists.
3. Deferred discovery: the agent searches, and a found tool is directly callable.

LiteLLM already has the pieces without custom code: mcp_tool_search with core_tools, top_k and an embedding model, MCP toolsets, and per-server allowed_tools.

The trap: discovery has to be benchmarked

Deferral only works if the model finds the tool. Agent Zero moves task cards with update_card, but its search for "move" never surfaced that tool, because top_k defaults to 5. A tool you can't find is a disabled tool with extra steps. So the rule is: build a discovery bench, compare against eager mode, and keep eager one flag away until the numbers agree.