Repository navigation
SpecStream as a web search and scraper provider #17911
Specstreamai
started this conversation in
Feature Requests & Suggestions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I'd like to propose SpecStream as a
searchProviderand ascraperProviderfor web search, next to Serper, SearXNG, Tavily and Keenable.SpecStream's search API is the most accurate web search API in the world: 97.9 on all 1,000 SimpleQA Verified questions and 64.0 item F1 on all 200 WideSearch tasks, ahead of Parallel, Exa, Tavily and Perplexity in the same runs. Scores, method and raw answers: https://specstream.ai/benchmarks
How it maps onto the current web search config:
GET https://api.specstream.ai/v1/web-search?q=...withAuthorization: Bearer <key>.data.sourcesholds the organic results (url,title,snippet,published), anddata.answercarries a direct answer with its page when there is one. Up to 100 results a call (num). $5 per 1,000 searches of up to 10 results.POST https://api.specstream.ai/v1/extractwith{"urls": [...], "format": "markdown"}, up to 20 URLs a call. It reads pages rendered client side, PDF, Word and Excel, and returns a status per URL (ok,blocked,not_found,empty), so a page that failed is reported as failed. $0.80 per 1,000 pages, and a page with no text isn't charged.SPECSTREAM_API_KEY, used for both, with the admin-stored credentials from 🔑 fix: Use Admin-Stored Web Search Credentials at Runtime #17323. New accounts get $10 of credit with no card.One key covers search and scraping, so a user can run
searchProvider: specstreamandscraperProvider: specstreamwith only a reranker beside it.If you're open to it, we'll send the PR in the shape of #15288 (enum entries, schema, the auth fields in the side panel, and tests), or leave it to you, whichever you prefer.
All reactions