feat(search): keyless web research on Claude Code — claude-code SearchProvider#187
Merged
Conversation
…hProvider
Web research no longer needs an Exa/Tavily key when running on Claude Code. A new
`ClaudeCodeSearch` SearchProvider drives the Agent SDK's WebSearch tool and returns
{url,title,snippet} hits that the existing _external_search rung grounds like any
other provider — so a full report (Reddit demand + web) runs with zero keys.
Keyless search FLOOR: resolve_search() falls back to ClaudeCodeSearch when no
external search key is set and the SDK + CLI are present (mirrors the chat floor);
any search key still wins.
Grounding (no-cite-no-claim): native web-search citations are unreachable through
the SDK (it drops the Messages-API citations array — confirmed by probe + SDK issue
anthropics/claude-agent-sdk-typescript#254), so attribution is reconstructed and
URL-VALIDATED — the real hit URLs are parsed from the WebSearch tool-result, the
model returns structured {url,title,snippet}, and any result whose URL is not a real
hit is DROPPED. An invented source can never enter the corpus. Binding is claim→URL,
not native span-citation; set EXA_API_KEY for the stronger path.
- search/adapters/claude_code.py: ClaudeCodeSearch (WebSearch + output_format
json_schema, URL-validate-against-real-hits, bare-URL fallback).
- llm/adapters/_claude_code_runtime.py (new): shared async→sync bridge
(background_loop + sdk_model) — chat adapter refactored to use it, search adapter
shares the one daemon loop. No behavior change to the chat adapter.
- config.py: keyless search floor in resolve_search().
- tests: offline (fake SDK) — keep-real/drop-invented URL validation, max_results,
bare-URL fallback, MissingExtraError, floor engages/absent.
- docs + CHANGELOG. Verified live: keyless WebSearch → 3 validated real hits + floor.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
Testing the keyless path as a real user would surfaced a stale hint: with no
provider key, `doctor`/`preflight` said "No provider key found → set one to run
the pipeline" and marked it an ERROR — but the claude-code chat floor means the
pipeline DOES run keyless now. The hint contradicted the resolved chat provider
(claude-code/sonnet) shown two lines above.
- preflight.py: when no provider key is set, branch on claude_code_available() —
if present, a non-error hint ("running keyless on your Claude Code login; set a
key for a faster path"); else the original hint, now also pointing at
metalworks[claude-code] as the keyless option. The keyless hint doesn't start
with "No provider key found", so _hint_severity keeps it a warning, not error.
- config.py: _claude_code_available → claude_code_available (public) — it's now
used cross-module (preflight + config); pyright reportPrivateUsage forbids the
private cross-module import. Callers + tests updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pgsharma
pushed a commit
that referenced
this pull request
Jun 26, 2026
Cut 0.4.0: the keyless Claude Code path (chat floor #186, web-search floor #187, caveat fix #188) plus the 0.3.1–0.3.3 reliability run, with a full docs/metadata pass and a positioning refresh. - Version → 0.4.0 (pyproject + __init__ + plugin.json) + CHANGELOG [Unreleased] → [0.4.0]. - Docs audit: surface the keyless `claude-code` path across README, docs/ (installation, index, quickstart, cli, claude-code, configuration, extending, demand-research, ai-agents, custom-chatmodel, internals), plugin docs + skills, CONTRIBUTING; add the `claude-code` extra everywhere it's listed; fix the stale "submissions come from the HF Parquet mirror" claim (the live Arctic Shift API has been the default since 0.1.1). - Positioning: drop "marketing research" (markety) and de-narrow the whole-product framing from Reddit-only to "real conversations across the web (Reddit, HN, forums, …)"; refresh keywords (out: marketing; in: validation, startups, founders). Reddit kept where it's a genuine feature. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pgsharma
added a commit
that referenced
this pull request
Jun 26, 2026
Cut 0.4.0: the keyless Claude Code path (chat floor #186, web-search floor #187, caveat fix #188) plus the 0.3.1–0.3.3 reliability run, with a full docs/metadata pass and a positioning refresh. - Version → 0.4.0 (pyproject + __init__ + plugin.json) + CHANGELOG [Unreleased] → [0.4.0]. - Docs audit: surface the keyless `claude-code` path across README, docs/ (installation, index, quickstart, cli, claude-code, configuration, extending, demand-research, ai-agents, custom-chatmodel, internals), plugin docs + skills, CONTRIBUTING; add the `claude-code` extra everywhere it's listed; fix the stale "submissions come from the HF Parquet mirror" claim (the live Arctic Shift API has been the default since 0.1.1). - Positioning: drop "marketing research" (markety) and de-narrow the whole-product framing from Reddit-only to "real conversations across the web (Reddit, HN, forums, …)"; refresh keywords (out: marketing; in: validation, startups, founders). Reddit kept where it's a genuine feature. Co-authored-by: Zpoof <praguns0726@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Follow-on to the keyless chat floor (#186). Web research no longer needs an Exa/Tavily key when running on Claude Code. A new
ClaudeCodeSearchSearchProviderdrives the Agent SDK'sWebSearchtool and returns{url, title, snippet}hits that the existing_external_searchrung inresearch/web.pygrounds like any other provider — so a full report (Reddit demand + web) runs with zero keys.Keyless search floor:
resolve_search()falls back toClaudeCodeSearchwhen no external search key is set and the SDK + CLI are present (mirrors the chat floor). Any search key still wins.Verified live (no key):
search("latest Python 3.14 release")→ 3 validated real python.org/peps hits with snippets;resolve_search()→ClaudeCodeSearch.Why this design (not
complete_grounded)I dug into whether native web citations are reachable keyless — they're not: the SDK drops the Messages-API
citationsarray (confirmed by probe and SDK issue #254). SoGroundedResultspan-citations are impossible without an API key. Instead I used theSearchProviderseam —WebSearchreturns real URLs, which the proven external-search grounding path already consumes. This is exactly how research agents (GPT-Researcher, Tavily/Exa pipelines) do it.Grounding — no-cite-no-claim holds (URL-validated)
Attribution is reconstructed and validated, never trusted blind:
WebSearchtool-result (Links:[{title,url}]).{url,title,snippet}(SDKoutput_formatjson_schema).Binding is claim→URL (model-asserted, URL-verified), not native span-citation. Honest and labeled as such;
EXA_API_KEYremains the stronger path.Changes
search/adapters/claude_code.py(new) —ClaudeCodeSearch: agenticWebSearch+output_format, capture-real-URLs, validate, bare-URL fallback.llm/adapters/_claude_code_runtime.py(new) — the shared async→sync bridge (background_loop+sdk_model) factored out of the chat adapter; the chat adapter now imports it (so both share one daemon loop, and the cross-module use isn't a private-access violation). No behavior change to the chat adapter — its 12 tests stay green.config.py— keyless search floor inresolve_search().test_claude_code_search.py, new) — offline (fake SDK): keep-real/drop-invented URL validation,max_results, bare-URL fallback,MissingExtraError, floor engages/absent.Gate
ruff check·ruff format --check·pyright(0 errors) ·gen_ts_types.py --checkPASSED ·import metalworksstays free. 145 passed clean-room (search + web-research + config + chat + adapters). No version bump — tagging left to you.🤖 Generated with Claude Code