docs: add AFDocs audit + fix skills for agent-friendly docs monitoring#25
Merged
rachaelrenk merged 7 commits intomainfrom May 5, 2026
Merged
docs: add AFDocs audit + fix skills for agent-friendly docs monitoring#25rachaelrenk merged 7 commits intomainfrom
rachaelrenk merged 7 commits intomainfrom
Conversation
Add a reusable skill (.agents/skills/afdocs-audit/) that runs the AFDocs scorecard against docs.warp.dev and produces a structured report. Follows the same pattern as the existing docs-seo-audit skill. Includes: - SKILL.md with instructions for running, reporting, and Slack notifications - afdocs_audit.mjs wrapper script that runs npx afdocs check --format json, parses the output, and produces a structured report with scores, issues, and fix guidance - references/known-exceptions.md documenting expected failures and false positives (content-start-position, markdown-content-parity, page-size) The skill is designed to be run by a scheduled Oz agent (weekly cadence) to monitor the docs site's agent-friendliness score over time. Co-Authored-By: Oz <oz-agent@warp.dev>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Add the companion skill that reads the afdocs-audit output and applies automated fixes for each failing check. Includes: - Fix procedures for each AFDocs check (llms.txt directive, content negotiation middleware, llms.txt coverage, MCP discovery, etc.) - Code snippets and file paths for each fix - Diagnostic steps for coverage mismatches - Clear separation of automatable vs. non-automatable checks - PR conventions (title prefix, labels, co-author) This is the 'fix layer' that a scheduled Oz agent runs after the audit skill to automatically remediate regressions. Co-Authored-By: Oz <oz-agent@warp.dev>
… skill Co-Authored-By: Oz <oz-agent@warp.dev>
…abel Co-Authored-By: Oz <oz-agent@warp.dev>
… counts - Fix stale /sse path in afdocs-fix MCP example (matches PR #23 fix) - Add URL validation in afdocs_audit.mjs to prevent shell injection - Remove hardcoded baseline score from SKILL.md reporting instructions - Soften hardcoded page counts in known-exceptions.md (varies by run) - Scope page-size exception to /changelog/ only, not all large pages Co-Authored-By: Oz <oz-agent@warp.dev>
rachaelrenk
changed the title
The Astro middleware in src/middleware.ts was not running at request time
for pre-rendered (static) pages. The default middlewareMode ('classic')
only executes middleware at build time for static pages, so Accept:
text/markdown requests were served HTML from Vercel's CDN cache.
Setting middlewareMode: 'edge' deploys the middleware as a separate
Vercel Edge Function that runs at request time for ALL requests,
including static pages. This enables content negotiation so agents
sending Accept: text/markdown get clean markdown automatically.
Co-Authored-By: Oz <oz-agent@warp.dev>
hongyi-chen
approved these changes
May 5, 2026
|
|
||
| ## Slack notification | ||
|
|
||
| After the audit completes, post a summary to the **#growth-docs** Slack channel (`C09BVK0PL3Y`). |
Collaborator
There was a problem hiding this comment.
Nit: should we make the Slack channel ID configurable via an env var too? It’s not really a secret, but hardcoding C09BVK0PL3Y makes this tied to one workspace/channel and harder to update later. Something like GROWTH_DOCS_SLACK_CHANNEL_ID could make this easier to reuse/configure.
|
|
||
| ## page-size-markdown / page-size-html | ||
|
|
||
| **Expected status**: warn — but only allowlist `/changelog/` |
Collaborator
There was a problem hiding this comment.
This makes me wonder if we should separate out our changelog based on the year, and have dedicated pages to 2020 --> 2026, which makes it easier to parse from an SEO / AEO perspective. The default docs.warp.dev/changelog link should just land the user on the latest changelog year page
Reverts the hardcoded #growth-docs channel ID (C09BVK0PL3Y) back to a <CHANNEL_ID> placeholder, matching the docs-seo-audit pattern. The channel is specified in the scheduled agent's prompt, not in the skill. Co-Authored-By: Oz <oz-agent@warp.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two paired skills for monitoring and remediating the agent-friendliness of docs.warp.dev, plus a fix for content negotiation middleware.
What's included
Part A:
afdocs-auditskill -- Runs the AFDocs scorecard and produces a structured report.afdocs_audit.mjswrapper script that runsnpx afdocs check --format json, parses the output, and produces a report with scores, grades, and fix guidancereferences/known-exceptions.mddocumenting expected flags (content-start-position, markdown-content-parity, changelog page size) so scheduled runs don't create noisePart B:
afdocs-fixskill -- Reads audit output and applies fixes.Content negotiation fix --
middlewareMode: 'edge'inastro.config.mjsmiddlewareMode: 'classic'only executes middleware at build time for static pagesmiddlewareMode: 'edge'deploys the middleware as a Vercel Edge Function that runs at request time for ALL requests, enablingAccept: text/markdowncontent negotiationcontent-negotiationcheck from FAIL to PASS (88 B+ toward 93+ A)How the skills work together
A scheduled Oz agent runs Part A (audit), then invokes Part B for any genuine failures, opens a PR with fixes, and posts a summary to #growth-docs.
Usage
Next step: Scheduled Oz agent
After this merges, configure a weekly scheduled Oz agent via the Oz web app or
oz schedule createto run both skills and post results to Slack. This is a configuration step, not a code change.References
.agents/skills/docs-seo-audit/Co-Authored-By: Oz oz-agent@warp.dev