Skip to content

v1.22.0 — .md-sibling duplicate check + fixer

Choose a tag to compare

@dkdev24 dkdev24 released this 11 Nov 02:20
· 9 commits to main since this release

New md-duplicate-signal check in the agent-headers scanner: fails when a .md mirror and its HTML twin both return 200 and the .md has neither a canonical Link header nor X-Robots-Tag: noindex. This is the pattern behind Search Console's "duplicate without user-selected canonical" alerts on sites that publish markdown siblings for agents.

enhance fixes it per platform: Cloudflare Pages and Netlify merge a /*.md noindex rule into _headers, Vercel merges one into vercel.json, CloudFront/OpenNext gets a warning with the exact response-headers-policy instruction, and GitHub/GitLab Pages stay report-only. Existing rules are never rewritten.

Also fixes agent-headers missing .md links in Starlight-format llms.txt files (bare root-relative paths), which made the new check skip on exactly the sites it targets.