[Feature] Support Anthropic Tool Search for MCP tools

[LK-Corporation]

Summary

Hermes Agent v0.11.0 sends the full schema of every registered MCP tool on every API call, regardless of relevance to the current turn. With multiple MCP servers connected, this becomes a significant fraction of every prompt.

Anthropic shipped Tool Search Tool (beta) precisely to solve this: tools marked defer_loading: true are not loaded into the model context until the model actively searches for them. The feature is supported across Anthropic API, Bedrock, Vertex AI, and Azure (i.e. all paths Hermes already uses).

This issue requests Hermes config support to opt MCP tools into Tool Search.

Why this matters (numbers)

• Anthropic reports ~85% reduction in tool-definition tokens on real catalogs (source).
• Accuracy improves, not degrades: Anthropic's internal MCP evals went from Opus 4 49% → 74% and Opus 4.5 79.5% → 88.1% with Tool Search enabled (large catalogs reduce false positives by removing decision paralysis).
• Independent confirmation: arXiv 2604.21816 "Tool Attention" measures the "MCP Tools Tax" at 15-60K tokens/turn for typical multi-server deployments.
• Related: existing issue Token overhead analysis: 73% of each API call is fixed overhead (~13.9K tokens) — data + suggestions #4379 ("Token overhead analysis: 73% of each API call is fixed overhead") quantifies the problem from the Hermes side.

Concrete sample

A real Hermes deployment (5 MCP servers, 34 tools total, mostly Haiku 4.5 via Bedrock) shows average prompt size 45K tokens/turn, of which ~22K is tool schema overhead (≈50%). Cache miss generations (start of session) cost $0.07-$0.10; cache hits cost $0.007. Tool Search would shrink the schema portion of the cache miss without affecting the rest of the prompt.

Proposed config surface

In ~/.hermes/config.yaml:

mcp_servers:
  vault-bridge:
    transport: streamable-http
    url: http://...:3142/mcp
    defer_loading: true   # NEW: marks all tools from this server as defer_loading=true

  cve-lookup:
    transport: stdio
    command: /usr/bin/node
    args: [...]
    defer_loading: true
    # OR per-tool override:
    tool_overrides:
      cve_lookup: { defer_loading: false }   # always-loaded for hot tools

agent:
  tool_search:
    enabled: true
    variant: bm25   # bm25 | regex
    # When enabled, Hermes adds the tool_search_tool definition to the tools array
    # and sends the `anthropic-beta: advanced-tool-use-2025-11-20` header on Anthropic-family providers

Implementation notes

1. Header propagation: pass anthropic-beta: advanced-tool-use-2025-11-20 on Anthropic, Bedrock, Vertex, Azure paths. Skip on non-Anthropic providers (graceful degradation — Hermes already supports fallback model logic).
2. tool_search_tool injection: when agent.tool_search.enabled: true, prepend the synthetic {"type": "tool_search_tool_bm25_20251119"} (or regex variant) to the tools array.
3. Defer flag: when serializing each MCP-discovered tool to the API request, include defer_loading: <bool> as configured.
4. Beta MCP header: also requires mcp-client-2025-11-20 per Anthropic docs.
5. Backward compat: default tool_search.enabled: false, defer_loading: false — opt-in only. Existing deployments unchanged.

Related issues

• Token overhead analysis: 73% of each API call is fixed overhead (~13.9K tokens) — data + suggestions #4379 — Token overhead analysis (73% fixed overhead). Tool Search directly attacks this.
• MCP utility stubs (list_prompts / get_prompt / list_resources / read_resource) are registered even when the server doesn't advertise the corresponding capabilities #18051 — MCP utility stubs registered even when server doesn't advertise capabilities. Tool Search reduces the cost of those stubs being present.

References

• Anthropic Tool Search docs: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool
• Anthropic Advanced Tool Use blog: https://www.anthropic.com/engineering/advanced-tool-use
• arXiv "Tool Attention": https://arxiv.org/abs/2604.21816
• arXiv "Instruction-Tool Retrieval (ITR)": https://arxiv.org/abs/2602.17046
• LiteLLM proxy already supports forwarding the beta header transparently: https://docs.litellm.ai/docs/providers/anthropic_tool_search

Acceptance criteria

• mcp_servers.<name>.defer_loading (bool) accepted and propagated to all tools from that server
• mcp_servers.<name>.tool_overrides.<tool>.defer_loading (bool) optional per-tool override
• agent.tool_search.enabled: true injects tool_search_tool_bm25_20251119 and advanced-tool-use-2025-11-20 header on Anthropic-family providers
• No-op on non-Anthropic providers (graceful degradation)
• Documentation updated in docs/user-guide/features/tools/
• Telemetry: log when Tool Search is engaged + how many tools were deferred

[aliasocracy]

Built a Hermes plugin that addresses this: alias8818/hermes-tool-slimmer

It reduces repeated tool-schema overhead by selecting the smallest useful tool subset per turn. Local BM25 ranking with explicit boosts, four selection modes (eager, keyword, hybrid placeholder, anthropic_tool_search). In keyword mode with top_k=8, a 57-tool catalog (~18K tokens) drops to ~3.7K tokens — roughly 80% reduction, consistent with the benchmarks cited in this issue.

Safety model: always_include for critical tools, never_defer to keep tools in prompt, fail_open on selector errors, dry-run mode for diagnostics without altering behavior. No monkeypatching if the Hermes core selector hook does not exist yet — degrades to benchmarking/diagnostics mode. Includes a minimal upstreamable Hermes core patch for the selector hook contract.

MIT, v0.1.0. Feedback welcome.

[LK-Corporation]

thanks, will evaluate when Hermes core merges the selector hook

[teknium1]

Closing as not planned. Anthropic's Tool Search Tool is a provider-specific beta that only covers the Anthropic API path (Bedrock, Vertex, Azure as variants); Hermes runs across 10+ providers and we don't want to bake an Anthropic-beta flag into our MCP config.

The underlying concern — tool-schema payload size on every API call as MCP servers multiply — is real. A generic tool-subset / lazy-load mechanism that works across providers would be the right shape; happy to consider a design for that separately. But a defer_loading: true flag that only works when the active provider is Anthropic adds a provider-specific surface that other users pay for without benefit.

If this becomes a non-beta Anthropic feature with durable semantics, we can revisit.

[xuli500177]

👍 Strongly support this feature request.

Use Case

Running Hermes in WSL but driving Windows GUI via MCP server (cua-driver). The Windows cua-driver exposes 39 tools (list_apps, get_window_state, click, type_text, hotkey, etc.), all of which get eagerly loaded into context at session startup.

Impact

• Token overhead: ~3,500-5,000 tokens per API call for tools that are rarely used (Windows GUI automation is niche)
• Context pollution: Model sees 39 tool definitions even for unrelated tasks (web search, coding, etc.)
• Latency: Full schema injection on every turn

Desired Config

Workaround Currently Used

Setting in config and manually toggling via Usage: hermes config set

Examples:
hermes config set model anthropic/claude-sonnet-4
hermes config set terminal.backend docker
hermes config set OPENROUTER_API_KEY sk-or-... + session restart - clunky and breaks workflow.

Related

• Issue Feature: Lazy Tool Schema Loading — Two-Pass Tool Injection to Reduce Token Overhead #6839 (Lazy Tool Schema Loading - two-pass) would complement this
• Anthropic's Tool Search Tool (beta) with is exactly what we need

Priority: High for WSL→Windows GUI automation workflows. Any MCP server with many niche tools benefits from this.