feat(scraper): fetch targets directly, delete the API-side scrape proxy

[Makisuo]

Why

Tonight's PlanetScale metrics blip (metrics.psdb.cloud 502/504s and 29s timeouts from 21:54 UTC) surfaced as maple-api 5xxs on GET /api/internal/prometheus-scrape and paged the org-wide "High error rate" rule for maple-api as well as scraper. Every branch scrape also parked a Worker invocation for up to 29s waiting on a third-party target. The proxy predates PlanetScale OAuth and existed to keep credentials server-side; both services already read secrets from the same Infisical project, and PlanetScale's data plane never used the Authorization header (it authenticates with the signed URL minted by discovery), so the proxy no longer bought anything.

What changed

• lib/safe-fetch (@maple/safe-fetch): the SSRF-safe fetch and URL validator moved out of apps/api unchanged so both apps share one guard. The API's five import sites now point at the lib. validateExternalUrlSync uses URL.canParse because lib/** enforces maple/no-try-catch.
• Target list carries what the scraper needs. InternalScrapeTarget gains targetType, scrapeUrl (the signed PlanetScale branch URL, otherwise the row url) and authHeaders. The API decrypts each row's credential while building the list, so the master key and the OAuth grant never leave the API. A row whose credential fails to decrypt is skipped for that round with a warning rather than failing the whole list.
• apps/scraper fetches directly through a new TargetFetcher service: same timeout formula, Retry-After parsing and span attributes the proxy had. The client span is scraper.fetch_target; it records server.address and url.path but never the signed query.
• Rotation without restarts. scrapeUrl and authHeaders are deliberately outside the scheduler's loop fingerprint. Each scrape reads the latest reconciled copy, so a re-signed PlanetScale URL or a rotated credential is picked up mid-loop without re-jittering the cadence.
• Failure mapping matches the old behaviour: timeouts and connection failures back off like the proxy's 502 did (target_error); an SSRF-rejected URL reports at interval without backoff (scrape_failed).
• Deleted from the API: the proxy route, scrapeForCollector, fetchUpstream, the per-isolate row memo and its invalidation hooks. ScrapeTargetsService exposes authHeaders(row) instead.

Verification

Check	Result
Typecheck: api, scraper, domain, lib	clean
Scraper tests (10 new: fetcher, SSRF, redirect credential strip, timeout abort, span redaction, rotation, backoff)	104 passed
API scrape-target tests	29 passed
lib/safe-fetch tests	69 passed
oxlint (repo Effect config), knip on touched workspaces	clean

Reviewer notes

• Repoint the "Scraper Planetscale Timeout" alert rule after merge. It watches the deleted ScrapeTargetsService.fetchUpstream span and will go silent. New where clause: service.name = "scraper" AND span.name = "scraper.fetch_target" AND attr.maple.scrape.target_type = "planetscale".
• Rollout window. An old scraper against the new API gets 404s on the proxy and self-heals on its next reconcile once the scraper ships. A new scraper against the old API fails to decode the target list and keeps its running loops (none on a cold boot) until the API ships. Both are transient; ship together.
• The scraper's own error rate now reflects target failures honestly, so a third-party outage pages for scraper rather than maple-api. Excluding it from the critical rule is a separate decision (the rule's excludeServiceNames field).
• lib/safe-fetch is a new workspace, registered with bun install --minimum-release-age=0; the lockfile diff is 17 added lines with no version moves.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[github-actions]

🍁 Maple PR preview

Warning

Preview cleanup could not be confirmed. The Alchemy teardown outcome was skipped.

Final commit e7c9608 · View workflow run