Open-source Cloudflare Worker that makes any website agent-readable. No code changes to your origin.
When you put this Worker in front of your site, it transparently:
- Serves
/llms.txt— auto-generated from your site, or override via KV - Serves
/ai.txt— agent permission directives - Returns Markdown when an agent asks for it —
Accept: text/markdowntriggers HTML→MD conversion - Injects
<link rel="alternate" type="text/markdown">into your<head>so agents discover the Markdown version - Injects
schema.orgJSON-LD if your page doesn't already have any - Sets
Vary: Acceptand canonicalLinkheaders so CDNs and agents handle the responses correctly
Your origin server is unmodified. Roll back by removing the route — that's it.
These are the same checks the Agent Ready certification scanner verifies. Deploy this Worker and your site passes all of them automatically.
# 1. Clone and install
git clone https://github.com/dman1313/agent-ready-worker.git
cd agent-ready-worker
npm install
# 2. Authenticate with Cloudflare
npx wrangler login
# 3. Create the KV namespace for per-site config
npx wrangler kv:namespace create AGENT_READY_CONFIG
# Copy the returned id into wrangler.toml under [[kv_namespaces]]
# 4. Edit wrangler.toml to set your routes
# routes = [{ pattern = "yourdomain.com/*", zone_name = "yourdomain.com" }]
# 5. Deploy
npx wrangler deployThat's it. Your site is now agent-ready. Verify with the Agent Ready scanner.
All config is optional — the Worker has sensible defaults out of the box. To customize, write keys to your AGENT_READY_CONFIG KV namespace:
| KV key | Purpose | Default |
|---|---|---|
config:llms_txt |
Body served at /llms.txt |
Auto-generated from site name + URL |
config:ai_txt |
Body served at /ai.txt |
Permissive defaults (Allow-RAG: yes) |
config:jsonld |
JSON-LD object string injected into <head> |
WebSite schema with site name + URL |
config:site_name |
Display name used in defaults | Hostname |
config:site_description |
Description used in defaults | Generic placeholder |
Set values:
npx wrangler kv:key put --binding=AGENT_READY_CONFIG "config:site_name" "My Company"
npx wrangler kv:key put --binding=AGENT_READY_CONFIG "config:site_description" "We make widgets."
npx wrangler kv:key put --binding=AGENT_READY_CONFIG "config:llms_txt" --path=./llms.txtThe Worker is conservative — it never overrides files you already have.
- If your origin already serves
/llms.txt, the Worker passes it through unchanged. - If your HTML already has
<link rel="alternate" type="text/markdown">, the Worker doesn't add a duplicate. - If your
<head>already has schema.org JSON-LD, the Worker doesn't inject another one.
You only get the Worker's defaults where your site has gaps.
Agent / Browser
│
▼
┌──────────────────────────────────────┐
│ Cloudflare Edge — Your Worker │
│ │
│ if /llms.txt or /ai.txt: │
│ try origin first; fall back to │
│ KV-configured or generated body │
│ │
│ if Accept: text/markdown: │
│ fetch HTML, convert via │
│ turndown, return as Markdown │
│ │
│ if HTML response: │
│ HTMLRewriter injects alternate │
│ link and JSON-LD as needed │
│ │
│ else: pass through unchanged │
└──────────────────────────────────────┘
│
▼
Your Origin (unmodified)
npm run dev # Local dev server (wrangler dev)
npm test # Run vitest test suite
npm run typecheck # Run TypeScript without emitting- Latency: HTML rewriting via
HTMLRewriteris streaming — no buffering. Markdown conversion is the slowest path (~10-50ms for typical pages); cache aggressively. - Cost: Cloudflare Workers free tier (100k requests/day) is sufficient for most sites. KV reads on the hot path: 1-3 per HTML request, all parallel. KV free tier (100k reads/day) is also typically enough.
- Workers Paid plan ($5/mo): removes request limits and adds CPU time headroom for the Markdown conversion path.
- It doesn't crawl or pre-render your site — it only transforms responses on demand.
- It doesn't change your sitemap.xml or robots.txt.
- It doesn't paywall the badge — the Agent Ready certification is a separate product covering manual review and listing.
- Agent Ready — certification scanner and managed install dashboard
- llms.txt spec — the standard this Worker implements
PRs welcome. Areas where help is most valuable:
- Markdown quality — edge cases in
turndownrules for cleaner agent output - HTMLRewriter coverage — handling unusual
<head>structures - Tests — adding fixtures from real-world sites that don't convert cleanly
- Multi-platform ports — Vercel Edge, Netlify Edge, Deno Deploy
MIT — see LICENSE.