Add direct fetch fallback for URL loading with race condition

[simonw]

Modify llm-cliche-highlighter - when it loads a URL have it fire off two requests in parallel, one to the jina thing and one to the URL directly (on the off-chance that it supports open CORS headers)

If both return favor the Jina one. If the Jina one returns an HTTP status error of any sort and the CORS one does not, use the CORS one. If both fail then show an error. Don't make a big deal out of this, just quietly use the CORS fetch as backup for if Jina fails.

Summary

Enhanced the URL loading mechanism to implement a race condition between Jina Reader proxy and direct fetch, allowing the page to load content even if Jina fails but the target site has open CORS headers.

Key Changes

• Refactored fetch logic into a reusable fetchText() helper function that handles HTTP error checking
• Implemented dual-fetch strategy: Jina Reader proxy as primary method with direct fetch as silent fallback
• Added try-catch nesting to attempt Jina first, then fall back to direct fetch if Jina fails
• Direct fetch runs in parallel (via Promise race semantics) but silently catches errors to avoid noise
• Updated code comments to document the new fallback behavior

Implementation Details

• Both fetch requests are initiated simultaneously to minimize latency
• The direct fetch promise has its rejection explicitly caught and suppressed (.catch(() => {})) to prevent unhandled rejection warnings
• Error handling prioritizes Jina errors but will throw them only if both methods fail
• The fallback gracefully handles CORS restrictions on the target site by catching and ignoring those errors

https://claude.ai/code/session_014q3Z2gAqr6XZxMkQBTaoem

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[cloudflare-workers-and-pages]

Deploying tools with    Cloudflare Pages

Latest commit:	7b3bcf9
Status:	✅  Deploy successful!
Preview URL:	https://e5a8635d.tools-b1q.pages.dev
Branch Preview URL:	https://claude-cliche-highlighter-pa.tools-b1q.pages.dev

View logs