Repository navigation
Add SerpKite as a web search (and optional scraper) provider #16944
Nolikzero
started this conversation in
Feature Requests & Suggestions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Opening this here first because CONTRIBUTING asks for an approved proposal before work starts on a feature that isn't on the roadmap.
Disclosure: I work on SerpKite, so I'm the vendor, not a user reporting a need. I haven't written any LibreChat code for this and won't unless you say it's wanted. "No" or "not now" is a perfectly good answer.
Proposal
Add SerpKite as an opt-in entry in
SearchProviders(currentlyserper,searxng,tavily,keenable) and, optionally, inScraperProviders. SerpKite is a Google SERP API with a URL-to-Markdown endpoint, so it can fill either role or both, the way Tavily does. Operators who never setserpkiteApiKeywould see no change.API contract
results[]:{ position, title, link, domain, displayed_link, snippet, date }. Top-levelanswer_box,knowledge_graph,people_also_ask,related_searchesandtop_storiesmap onto the existing serper-shaped fields (organic,answerBox,knowledgeGraph,peopleAlsoAsk,relatedSearches,topStories) without loss.GET /v1/newsuses the same parameters and returns{ title, link, snippet, source, date, image_url }.POST /v1/webpagewith{"url": "<public URL>"}returns the page as Markdown, with an SSRF guard on our side. LibreChat's ownallowedAddressespreflight would still apply.{"error":{"code","message","request_id"}}, with 400 (unknown or invalid parameter, since params are strict), 401, 402 (out of credits), 429 and 503. The adapter can surfaceerror.messagedirectly.numis capped at 100.timetakes the five named windows, not custom ranges (those go in separatestart_date/end_dateparams).include_domains/exclude_domainsare native parameters, sosite:rewriting isn't needed. Failed and empty searches aren't billed.Docs: https://serpkite.com/docs. OpenAPI: https://serpkite.com/openapi/serp-api.yaml.
Affected areas
The closest template is the Keenable work (#15288) and Tavily (#12581):
packages/data-provider/src/config.tsandconfig.spec.ts(enum entries,serpkiteApiKeyand an optionalserpkiteApiUrl)packages/data-provider/src/types/web.ts,packages/data-schemas/src/app/web.tsand typespackages/api/src/web/web.ts(auth and credential resolution) plus specsclient/src/components/SidePanel/Agents/Search/ApiKeyDialog.tsxandentranslation keyslibrechat.example.yaml,.env.example@librechat/agents(search and scrape functions next to the serper and tavily ones), as a separate PR thereEverything is additive. I'd follow the existing SSRF, fail-closed credential and "system key never forwarded to a user-controlled URL" rules from #15288.
What I'd ask
Is a SerpKite provider something you'd accept? If so, should it be search only, or search plus scraper? I can open the
@librechat/agentsadapter PR first so you can review the mapping on its own. Happy to provide a test key for CI or review.All reactions