cse-bridge 1.2.0

Google's item.pagemap carried the structured data it had scraped from each result page: metatags, schema.org objects, cse_image, cse_thumbnail. SearXNG returns none of that, so code that read item["pagemap"]["metatags"][0]["og:image"] broke the moment it pointed at a bridge.

Set CSE_BRIDGE_PAGEMAP=on, or pagemap: true on a profile (which wins over the environment, in both directions), and the bridge fetches the result pages themselves and rebuilds the parts that genuinely came off the page:

• metatags with the key exactly as the page wrote it. og:title keeps its case, a bare name="description" is lowercased, which is what the live JSON API did.
• DataObjects named by lowercased schema.org type (product, recipe, qapage, newsarticle, ...) from application/ld+json, @graph nodes included, and from microdata itemtype/itemprop.
• cse_image and cse_thumbnail as [{"src": ...}], from the first of og:image, og:image:secure_url, twitter:image, twitter:image:src that resolves to an http(s) URL.
• Any literal <PageMap> block the site publishes. That was Google's own markup for site owners, so it wins over a DataObject of the same name.

What it will not do is invent the rest. Objects that existed only because Google's index built them cannot be recovered from the page, and cse_thumbnail width and height described Google's own crop, so only src is emitted. When nothing parses, the pagemap key is absent rather than an empty object pretending to be data. Same posture as totalResults and image.thumbnailWidth: an honest gap beats a synthesized number.

It is off by default, and byte-identical to 1.1.0 when off. Not an argument, a measurement: the same seven request shapes ran against the same fixed backend in a worktree at v1.1.0 and on this tree, and the 816 lines of output diffed clean.

Fetching pages on behalf of a search is the risky part, so the guardrails are the bulk of src/pagemap.ts. Ten pages per request, four at a time, three seconds per URL under a nine second budget for the whole pass, at most two redirects, non-HTML refused, a 512 KB cap with the read stopping at </head>, and an in-memory LRU with a one hour TTL. Results that share a page share one fetch, so ten images from one gallery cost one request. Every failure is swallowed: a page that times out or 404s leaves that item bare and the search still returns 200. Because these are URLs a third party chose, loopback, RFC1918, link-local, CGNAT, .internal/.local and the IPv6 equivalents are refused before any request goes out, on the original URL and again on every redirect. It matches hostnames rather than resolving them, so it is not proof against DNS rebinding, and the README says so rather than implying a sandbox.

Parsing is regex plus JSON.parse. No HTML parser, so the runtime dependency count is still zero.

196 offline tests, 200 with CSE_BRIDGE_LIVE=1 against a real SearXNG, all green. The README has a new "Structured data" section with real captured output and an honest list of where this will not match Google, and the migration guide splits what a migrator can and cannot rely on.

• npm: npm i -g cse-bridge
• image: docker pull ghcr.io/booyaka101/cse-bridge:1.2.0
• full changelog: CHANGELOG.md

cse-bridge 1.1.0

Image clients now migrate with the same one-line base-URL change web clients already get.

searchType=image switches the backend query to SearXNG's images category and maps its image results into Google's exact item shape. link is the image file itself, because that is what Google promises and what clients hotlink into <img> tags; the page the image was found on becomes image.contextLink. image.thumbnailLink comes from thumbnail_src, image.width/image.height are parsed from SearXNG's human-readable resolution ("1920 x 1080"), image.byteSize from filesize ("412 KB", 1 KB = 1024), and mime/fileFormat from img_format (jpg → image/jpeg).

Anything an engine does not report is omitted, never guessed — including image.thumbnailWidth/thumbnailHeight, which SearXNG does not report at all. That is the same posture this project takes on totalResults: an honest gap beats a synthesized number. A result with no image URL is dropped entirely rather than emitted with a page URL as link, so no item ever claims to be an image while pointing at an HTML page.

searchType round-trips through queries.request[0], as on Google, and image is the only accepted value — anything else returns Google's INVALID_ARGUMENT envelope.

imgSize, imgType, imgColorType and imgDominantColor are validated against Google's exact enums (an out-of-enum value gets Google's 400, because Google rejects it too) and then accepted for compatibility. SearXNG has no size/type/color parameters to map them onto, so they do not filter — the same documented posture as sort expressions beyond date.

Bug fix: result de-duplication keyed on the result's url, but for image results url is the page the image sits on — so ten images from one gallery collapsed into a single item. De-duplication now keys on the image URL when present, falling back to url for web results.

searchType=image supersedes the profile's categories rather than merging with them: a cx pinned to categories: [news] cannot also be an image engine. The profile's site: restriction, language and engines still apply. Image and web result sets for the same query occupy separate cache entries, so alternating between them never leaks results across.

The Limitations note claiming there is no image search is gone from the README, and the migration guide's searchType=image row now reads Supported.

138 offline tests (up from 109), 142 with CSE_BRIDGE_LIVE=1 against a real SearXNG — all green.

• npm: npm i -g cse-bridge
• image: docker pull ghcr.io/booyaka101/cse-bridge:1.1.0
• full changelog: CHANGELOG.md

cse-bridge 1.0.2

Documentation-only release. No functional change from 1.0.1 — the bridge behaves identically.

The npm package page renders the README from the published tarball, so 1.0.1 still displayed a maintainer-facing "Distribution" section that pointed readers at the Hacker News shutdown thread. That thread is archived and cannot be replied to. It has been replaced with a reader-facing "Further reading and feedback" section linking the long-form write-up and discussion, plus an explicit ask: report any client library that will not accept an endpoint override.

Also syncs package-lock.json, which was left at 1.0.0 during the 1.0.1 bump.

• npm: npm i -g cse-bridge
• image: docker pull ghcr.io/booyaka101/cse-bridge:1.0.2

cse-bridge 1.0.1

First public release of cse-bridge: a self-hosted drop-in for Google's Custom Search JSON API (retiring 2027-01-01), backed by your own SearXNG instance.

• GET /customsearch/v1 returns a Google-exact customsearch#search body
• Unmodified @googleapis/customsearch, google-api-python-client and LangChain GoogleSearchAPIWrapper clients work by changing only the endpoint
• Honest lower-bound totalResults (SearXNG's JSON has no count field)
• Stable per-query result sets so start=1/11/21 pages are disjoint
• Google-exact error envelopes (400/403/404/429/503)
• cx -> engine/category/site profiles via profiles.yml
• docker compose up -d brings up SearXNG (json format pre-enabled) + the bridge

Install: npm i -g cse-bridge, or docker pull ghcr.io/booyaka101/cse-bridge:1.0.1

1.0.1 fixes the GHCR image name casing in docker-compose.yml (1.0.0's compose file failed on clean clones).