An URL reader like jina-ai/reader for AI Agents which can be deployed very easily as a Cloudflare Worker.
IMPORTANT: You need Worker Paid plan to use this worker due to the limitation of Browser ability.
npm run deploy[YOUR_WORKER_URL]?target={targetUrl}&mode={mode}&auth_key={authKey}
-
targetUrl: URL of the target page, read image / audio / video directly is not supported. -
mode(optional): Could be "markdown", "body". As formarkdownmode, if the original content istext/html, it will be formatted into a human-readable markdown format text. As forbodymode, it will return the body part of the page. Otherwise, the worker will return the original rendered HTML text. -
authKey(optional): IfAUTH_KEYset, you need to pass the key in the query. We recommend to use Cloudflare's abilities to protect your worker.
CACHE_CONTROL: Response cache control which follows Cloudflare's rules.BROWSER_TIMEOUT: General timeout for puppeteer.FETCH_CACHE_TTL: General ttl for caching HEAD and GET requests to original content.BROWSER_USER_AGENT: User-Agent string for puppeteer.SALVAGE_USER_AGENT: User-Agent string for salvage fetching.AUTH_KEY: A key to access the worker.
Please using WAF / Rules on Cloudflare to secure your worker because it can access any url from any source by default, THIS WORKER CAN BE ABUSED POTENTIALLY.
If you want to add some extra logic to block some evil requests, please fork it and modify the code.
Apache-2.0