Skip to content

feat(tools): web_search MCP tool + mcp_servers harness passthrough#543

Closed
AbirAbbas wants to merge 4 commits into
mainfrom
feat/web-search-tool-registry
Closed

feat(tools): web_search MCP tool + mcp_servers harness passthrough#543
AbirAbbas wants to merge 4 commits into
mainfrom
feat/web-search-tool-registry

Conversation

@AbirAbbas

Copy link
Copy Markdown
Contributor

Summary

Adds a way for harnessed Claude Code agents to invoke web search during a run, so reasoners can look up library docs, error messages, deprecation status, and other context that doesn't live in the codebase.

Three layers, three commits:

  1. agentfield.tools.search — vendored multi-provider search package (Jina, Tavily, Firecrawl, Serper) with a unified SearchProvider ABC, env-driven auto-detection, and a priority order overridable via SEARCH_PROVIDER. Ported from Agent-Field/af-deep-research's skills/search from aiohttp to httpx so it lines up with this repo's existing pytest-httpx test infra.

  2. mcp_servers parameter on Agent.harness — new keyword arg (also reachable via the router.harness delegation path) that forwards an MCP-server mapping straight to ClaudeAgentOptions.mcp_servers. Honored only by the claude-code provider; opencode/codex/gemini ignore it the same way they already ignore tools=[...].

  3. agentfield.tools.web_search.get_web_search_server() — returns (McpSdkServerConfig, [tool_names]) for an in-process MCP server (built via claude_agent_sdk.create_sdk_mcp_server + @tool) that wraps the search package. Reasoners drop the server under a key in mcp_servers={...} and merge the namespaced mcp__af_search__web_search name into their tools=[...] allow-list. Tool description tells the model to reach for it only when the answer can't come from the codebase, mitigating loop risk in coding agents.

Why this shape

Considered building it as a per-call Python pre-step (search before the prompt, inject results) and as a custom-tool registry across providers. Pre-step loses the agent's ability to decide when to search; cross-provider registries would have required modifying three external CLI binaries. The in-process MCP server is the canonical extension point in claude-agent-sdk and matches the way Claude Code itself ships built-in tools — minimal new surface area, no per-call wrapper code in reasoners.

Test plan

  • 39 unit/integration tests pass (tests/tools/, tests/test_harness_mcp.py)
    • Per-provider HTTP mocking for Jina, Tavily, Firecrawl, Serper
    • Registry/auto-detect: priority order, SEARCH_PROVIDER override, fallback when preferred is unavailable, custom provider registration
    • web_search wrapper: input validation, no-provider error path, exception-in-search handler, markdown formatting (snippet truncation, missing fields, empty results)
    • Harness plumbing: passthrough to ClaudeAgentOptions, omission when not set, real-server-config wiring, silent ignore on non-claude provider
  • 2 live tests in tests/tools/test_search_live_jina.py — skipped without JINA_API_KEY, otherwise hit the real Jina API and assert structured results
  • 1 end-to-end test in tests/test_harness_web_search_e2e.pyharness_live marker (deselected by default addopts), gated on both ANTHROPIC_API_KEY and JINA_API_KEY. Drives a real Claude harness call, asks the model to use web_search to look up pytest's docs URL, asserts the model actually invoked the tool

Existing harness suite (130 tests under -k harness): all green, no regressions.

Follow-up

Sister PR coming on Agent-Field/SWE-AF that wires get_web_search_server() into 6 reasoners — run_architect, run_ci_fixer, run_pr_resolver as Tier 1 (web access is high-leverage), and run_coder, run_retry_advisor, run_product_manager as Tier 2 (with prompt guardrails on the coder to scope use). Per-reasoner cost analysis already done; the deterministic / pure-synthesis / short-budget reasoners are deliberately excluded.

🤖 Generated with Claude Code

AbirAbbas and others added 3 commits May 6, 2026 15:09
Adds agentfield.tools.search — a unified async interface over Jina, Tavily,
Firecrawl, and Serper. Each provider implements the SearchProvider ABC and
returns the same SearchResponse shape. Auto-detection picks the first
provider whose API key is configured (priority: jina > tavily > firecrawl >
serper); SEARCH_PROVIDER env var overrides.

Vendored from github.com/Agent-Field/af-deep-research/skills/search; ported
from aiohttp to httpx so tests can use the existing pytest-httpx fixtures.

Tests: 28 unit tests covering happy path, missing-key, HTTP error, and
registry/auto-detect logic per provider, plus a live Jina smoke test gated
on JINA_API_KEY.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an mcp_servers parameter to Agent.harness() (and its router-delegated
form). When the claude-code provider is selected, the value is forwarded
verbatim to ClaudeAgentOptions so reasoners can attach in-process MCP
servers (e.g. those from claude_agent_sdk.create_sdk_mcp_server) for the
agent to invoke. Other providers silently ignore mcp_servers, matching the
existing behavior for the tools allow-list.

The runner's _resolve_options gains "mcp_servers" as a recognized field so
HarnessConfig defaults can carry it too.

Tests: 4 cases covering passthrough to ClaudeAgentOptions, omission when
absent, real web_search server-config wiring, and silent-ignore on a
non-claude provider.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… agents

Adds agentfield.tools.web_search.get_web_search_server() which returns an
McpSdkServerConfig backed by the multi-provider search package, plus the
fully-namespaced tool names to drop into the harness allow-list. Reasoners
running against the claude-code provider can now grant a Claude harness
the ability to look up external context (library docs, error messages,
deprecation status) by passing the returned server + tool names through
the new mcp_servers / tools parameters on router.harness().

Use sparingly — the tool description tells the model to reach for it only
when the answer cannot come from the codebase. This avoids the loop-risk
of agents searching instead of coding.

Tests: 11 unit tests covering input validation, no-provider fallback,
formatted-result shape, exception handling in the tool handler, and the
get_web_search_server contract. Plus an e2e test (harness_live marker;
gated on ANTHROPIC_API_KEY + JINA_API_KEY) that drives a real Claude
harness call and asserts the model actually invoked web_search.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@AbirAbbas AbirAbbas requested a review from a team as a code owner May 6, 2026 19:30
@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

Performance

SDK Memory Δ Latency Δ Tests Status
Python 9.4 KB +4% 0.31 µs -11%

✓ No regressions detected

… is absent

CI installs the SDK as ``.[dev]``, which doesn't pull claude-agent-sdk
(it lives under the ``harness-claude`` / ``harness`` extras). Tests that
exercise ``@tool`` and ``create_sdk_mcp_server`` need the real SDK and
can't run in that env.

Adds ``pytest.importorskip("claude_agent_sdk")`` to the call sites that
require it:
- _get_handler() in tests/tools/test_web_search.py — covers the @tool
  handler tests
- test_get_web_search_server_returns_config_and_namespaced_tool_names
- test_harness_with_real_web_search_server_config in test_harness_mcp.py

Pure markdown-formatting tests (no SDK call) and the harness-passthrough
tests (which use a fake claude_agent_sdk module via monkeypatch) continue
to run in every env.

Verified: with the SDK blocked from import, the gated tests skip cleanly
and the rest still pass; with the SDK installed locally, all 41 tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

📊 Coverage gate

Thresholds from .coverage-gate.toml: per-surface ≥ 86%, aggregate ≥ 88%, max per-surface regression ≤ 1.0 pp, max aggregate regression ≤ 0.50 pp.

Surface Current Baseline Δ
control-plane 87.40% 87.30% ↑ +0.10 pp 🟡
sdk-go 91.90% 90.70% ↑ +1.20 pp 🟢
sdk-python 93.66% 93.63% ↑ +0.03 pp 🟢
sdk-typescript 92.68% 92.56% ↑ +0.12 pp 🟢
web-ui 89.81% 90.01% ↓ -0.20 pp 🟡
aggregate 88.92% 89.01% ↓ -0.09 pp 🟡

✅ Gate passed

No surface regressed past the allowed threshold and the aggregate stayed above the floor.

@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

📐 Patch coverage gate

Threshold: 80% on lines this PR touches vs origin/main (from .coverage-gate.toml:thresholds.min_patch).

Surface Touched lines Patch coverage Status
control-plane 0 ➖ no changes
sdk-go 0 ➖ no changes
sdk-python 0 ➖ no changes
sdk-typescript 0 ➖ no changes
web-ui 0 ➖ no changes

✅ Patch gate passed

Every surface whose lines were touched by this PR has patch coverage at or above the threshold.

@AbirAbbas

Copy link
Copy Markdown
Contributor Author

Closing in favor of a simpler approach.

After research, both providers have native first-party web search built in:

  • claude-code: WebSearch / WebFetch are standard Claude Code tool names; just need to be in the harness tools allow-list.
  • opencode: websearch / webfetch are built-ins, gated behind OPENCODE_ENABLE_EXA=1 + EXA_API_KEY env vars (Exa backend). The opencode subprocess inherits env naturally via _cli.py's merged_env = {**os.environ}.

So the MCP wrapper, the multi-provider search package, and the mcp_servers harness plumbing were all over-engineered — none of it is needed when the providers already ship their own web search tools. The wiring lives entirely in SWE-AF (and other consumers) by adding tool names to the allow-list and setting the right env vars at deploy time.

Filed as a separate concern: the opencode provider in agentfield silently drops tools=[...] (no permission-config translation). Not blocking web search since opencode enables tools by default, but worth fixing for proper allow-list semantics. Will track separately if it becomes a need.

No production code from this branch was running anywhere — safe to close without follow-up.

@AbirAbbas AbirAbbas closed this May 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant