docs: add AFDocs audit + fix skills for agent-friendly docs monitoring

.agents/skills/afdocs-audit/SKILL.md

@@ -0,0 +1,141 @@
+---
+name: afdocs-audit
+description: >-
+  Audit docs.warp.dev for agent-friendly documentation issues using the AFDocs
+  scorecard. Checks llms.txt, markdown availability, content negotiation, page
+  size, URL stability, and content structure. Use when asked to check agent
+  readiness, run an AFDocs audit, improve the docs score, or verify llms.txt
+  and markdown support.
+---
+
+# AFDocs Audit
+
+Run the [AFDocs scorecard](https://agentdocsspec.com/spec/) against docs.warp.dev and report results.
+
+## Running the audit
+
+From the docs repo root:
+
+```bash
+node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs \
+  --output /tmp/afdocs-report.json
+```
+
+The script runs `npx afdocs check https://docs.warp.dev --format json`, parses the output, and writes a structured report.
+
+### Options
+
+- `--output FILE` — Write the JSON report to a file (otherwise prints to stdout).
+- `--url URL` — Override the site URL (default: `https://docs.warp.dev`).
+
+## Reading the report
+
+The JSON report contains:
+- `score` — Overall score out of 100
+- `grade` — Letter grade (A+ through F)
+- `total_checks` — Number of checks run
+- `summary` — Counts by status (`pass`, `fail`, `warn`, `skip`)
+- `categories` — Per-category scores and grades
+- `issues` — Array of failing and warning checks with details and fix guidance
+
+Each issue includes:
+- `id` — Check identifier (e.g., `llms-txt-directive-html`)
+- `category` — Check category (e.g., `content-discoverability`)
+- `status` — `fail` or `warn`
+- `message` — Human-readable description
+- `fix` — Suggested fix from the AFDocs spec
+
+### Known exceptions
+
+Before reporting, cross-reference every issue against the known exceptions in `references/known-exceptions.md`. Classify each issue into exactly one bucket:
+- **Allowlisted** — known exceptions that are intentional (not problems)
+- **Remaining** — genuine issues that need attention
+
+Only include a section if its count is > 0. Never list allowlisted issues under "Remaining."
+
+## Reporting results
+
+After running the audit, ALWAYS report the results to the user before taking any action. Include:
+
+1. **Score**: Overall score and grade
+2. **Failures first**: List every fail-severity check with its message and fix guidance. These are the most impactful.
+3. **Warnings**: List warning-severity checks with context.
+4. **Allowlisted**: Briefly note any known exceptions that were flagged.
+5. **If all checks pass**: Explicitly tell the user everything looks clean.
+
+Example report format:
+```
+AFDocs audit complete: 23 checks run, score 82/100 (B).
+
+**Failures (5):**
+- llms-txt-directive-html: No llms.txt directive in HTML pages
+  Fix: Add a visually-hidden element near the top of each page with a link to /llms.txt
+- content-negotiation: Server ignores Accept: text/markdown
+  Fix: Add middleware to serve .md variants when Accept: text/markdown is requested
+
+**Warnings (1):**
+- llms-txt-coverage: 80% of sitemap pages covered (247/308)
+
+**Allowlisted (2):**
+- page-size-markdown: 1 page over 50K (changelog — intentionally long)
+- markdown-content-parity: 7 pages with minor diffs (Turndown escaping, not real content gaps)
+```
+
+After reporting, ask the user which issues they want to address.
+
+## Slack notification (optional)
+
+If instructed to send a report to Slack, post a summary after the audit completes.
+
+1. Check if `BUZZ_SLACK_TOKEN` environment variable exists.
+2. If the token exists, send a summary to the channel the user specified (or the channel configured in the agent's instructions).
+
+**Format:**
+
+```
+*AFDocs Audit — <date>*
+Score: <score>/100 (<grade>) | <total_checks> checks | <pass> pass, <fail> fail, <warn> warn
+
+*Failures (<count>):*
+• <check_id>: <message>
+
+*Warnings (<count>):*
+• <check_id>: <message>
+
+*Allowlisted (<count>):*
+• <check_id>: <reason>
+```
+
+Send using:
+
+```bash
+curl -X POST https://slack.com/api/chat.postMessage \
+  -H "Authorization: Bearer $BUZZ_SLACK_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "channel": "<CHANNEL_ID>",
+    "text": "<formatted_summary>",
+    "unfurl_links": false,
+    "unfurl_media": false
+  }'
+```
+
+If `BUZZ_SLACK_TOKEN` is not set, skip the notification and note that the token is required.
+
+## Dependencies
+
+Node.js 18+ with npm (for `npx afdocs`). No additional install required — `afdocs` is fetched on demand by npx.
+
+## Checks performed
+
+The AFDocs scorecard evaluates these categories:
+
+**Content Discoverability** — llms.txt existence, validity, size, link resolution, markdown links, and in-page directives
+**Markdown Availability** — .md URL support and Accept: text/markdown content negotiation
+**Page Size and Truncation Risk** — rendering strategy, page sizes (markdown and HTML), and content start position
+**Content Structure** — tabbed content serialization, section header quality, code fence validity
+**URL Stability and Redirects** — HTTP status codes and redirect behavior
+**Observability and Content Health** — llms.txt coverage, markdown/HTML parity, cache headers
+**Authentication and Access** — auth gate detection and alternative access paths
+
+Full spec: https://agentdocsspec.com/spec/

.agents/skills/afdocs-audit/references/known-exceptions.md

@@ -0,0 +1,42 @@
+# AFDocs Known Exceptions
+
+This file lists checks from the afdocs-audit skill that may flag as warnings or failures but are expected and intentional. When reporting audit results, classify these as "Allowlisted" rather than "Remaining."
+
+## content-start-position
+
+**Expected status**: fail or warn
+**Reason**: Sampled pages may have content starting past 50% of the HTML output. This is inherent to Starlight's layout — sidebar navigation, header markup, and JavaScript/CSS precede the `<main>` content area.
+**Mitigation**: The llms.txt directive, `<link rel="alternate" type="text/markdown">` in `<head>`, and `Accept: text/markdown` content negotiation middleware all steer agents to the clean markdown version, bypassing the HTML boilerplate entirely.
+**Action**: No fix needed. This is a structural property of Starlight sites.
+
+## markdown-content-parity
+
+**Expected status**: warn (several pages, ~2% average difference)
+**Reason**: False positive. The "missing" segments are numbered heading text like "2. Tabbed File Viewer" where Turndown correctly escapes the period (`### 2\. Tabbed File Viewer`) to prevent markdown parsers from interpreting it as a list item. The content IS present in the markdown — the AFDocs checker's text comparison doesn't account for markdown escaping.
+**Affected pages** (as of 2026-05-05):
+- `/agent-platform/cloud-agents/triggers/scheduled-agents-quickstart/` — step headings
+- `/agent-platform/cloud-agents/integrations/github-actions/` — numbered use case headings
+- `/support-and-community/troubleshooting-and-support/troubleshooting-login-issues/` — URLs with special chars
+- `/reference/cli/quickstart/` — optional step headings
+- `/guides/getting-started/welcome-to-warp/` — numbered section headings
+- `/terminal/editor/vim/` — "See Vim docs:" link text
+- `/guides/getting-started/10-coding-features-you-should-know/` — numbered feature headings
+**Action**: No fix needed. Content is intact.
+
+## page-size-markdown / page-size-html
+
+**Expected status**: warn — but only allowlist `/changelog/`
+**Reason**: The changelog page (`/changelog/`) is intentionally a single long page (~4,000 lines of MDX). It is excluded from `llms-full.txt` generation due to a `hast-util-to-text` stack overflow, but is still accessible at its URL and indexed by the sitemap.
+**Action**: If the only flagged page is `/changelog/`, classify as allowlisted. If other pages are flagged, treat those as genuine issues that may need splitting.
+
+## section-header-quality
+
+**Expected status**: skip
+**Reason**: Only evaluated when tab panels contain section headers. Most sampled pages with tabs don't have headers inside the tab panels, so the check is skipped.
+**Action**: None needed.
+
+## auth-alternative-access
+
+**Expected status**: skip
+**Reason**: All docs pages are publicly accessible, so no alternative access path is needed.
+**Action**: None needed.

.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs

@@ -0,0 +1,193 @@
+#!/usr/bin/env node
+/**
+ * AFDocs audit wrapper script.
+ *
+ * Runs `npx afdocs check` against docs.warp.dev, parses the JSON output,
+ * and produces a structured report with scores, issues, and fix guidance.
+ *
+ * Usage:
+ *   node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs
+ *   node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs --output /tmp/report.json
+ *   node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs --url https://preview.docs.warp.dev
+ */
+
+import { execSync } from 'node:child_process';
+import { writeFileSync } from 'node:fs';
+import { resolve } from 'node:path';
+
+const GRADE_THRESHOLDS = [
+	[97, 'A+'],
+	[93, 'A'],
+	[90, 'A-'],
+	[87, 'B+'],
+	[83, 'B'],
+	[80, 'B-'],
+	[77, 'C+'],
+	[73, 'C'],
+	[70, 'C-'],
+	[67, 'D+'],
+	[63, 'D'],
+	[60, 'D-'],
+	[0, 'F'],
+];
+
+function scoreToGrade(score) {
+	for (const [threshold, grade] of GRADE_THRESHOLDS) {
+		if (score >= threshold) return grade;
+	}
+	return 'F';
+}
+
+function parseArgs(argv) {
+	const args = { output: null, url: 'https://docs.warp.dev' };
+	for (let i = 0; i < argv.length; i++) {
+		if (argv[i] === '--output') args.output = argv[++i];
+		else if (argv[i] === '--url') args.url = argv[++i];
+		else if (argv[i] === '--help' || argv[i] === '-h') {
+			console.log('Usage: node afdocs_audit.mjs [--output FILE] [--url URL]');
+			process.exit(0);
+		}
+	}
+	return args;
+}
+
+function runAfdocsCheck(url) {
+	// Validate URL to prevent shell injection
+	try {
+		const parsed = new URL(url);
+		if (!['http:', 'https:'].includes(parsed.protocol)) {
+			throw new Error(`Invalid protocol: ${parsed.protocol}`);
+		}
+	} catch (e) {
+		throw new Error(`Invalid URL "${url}": ${e.message}`);
+	}
+
+	try {
+		const stdout = execSync(`npx afdocs check ${url} --format json`, {
+			encoding: 'utf8',
+			maxBuffer: 10 * 1024 * 1024, // 10 MB — the JSON output can be large
+			timeout: 300_000, // 5 minutes
+			stdio: ['pipe', 'pipe', 'pipe'],
+		});
+		return JSON.parse(stdout);
+	} catch (error) {
+		// npx afdocs exits with code 1 when there are failures, but still
+		// prints valid JSON to stdout. Try to parse it.
+		if (error.stdout) {
+			try {
+				return JSON.parse(error.stdout);
+			} catch {
+				// Fall through to error
+			}
+		}
+		throw new Error(`Failed to run afdocs check: ${error.message}`);
+	}
+}
+
+function buildReport(raw) {
+	const { summary, results } = raw;
+	const score = raw.summary?.score ?? estimateScore(results);
+	const grade = scoreToGrade(score);
+
+	// Group results by category
+	const categories = {};
+	for (const r of results) {
+		if (!categories[r.category]) {
+			categories[r.category] = { checks: [], pass: 0, fail: 0, warn: 0, skip: 0 };
+		}
+		categories[r.category].checks.push(r);
+		categories[r.category][r.status] = (categories[r.category][r.status] || 0) + 1;
+	}
+
+	// Extract issues (fail + warn)
+	const issues = results
+		.filter((r) => r.status === 'fail' || r.status === 'warn')
+		.map((r) => ({
+			id: r.id,
+			category: r.category,
+			status: r.status,
+			message: r.message,
+			fix: r.details?.fix || r.fix || null,
+		}));
+
+	return {
+		url: raw.url,
+		timestamp: raw.timestamp || new Date().toISOString(),
+		score,
+		grade,
+		total_checks: summary.total,
+		summary: {
+			pass: summary.pass,
+			fail: summary.fail,
+			warn: summary.warn,
+			skip: summary.skip,
+		},
+		categories: Object.fromEntries(
+			Object.entries(categories).map(([name, cat]) => [
+				name,
+				{ pass: cat.pass, fail: cat.fail, warn: cat.warn, skip: cat.skip },
+			])
+		),
+		issues,
+		all_results: results.map((r) => ({ id: r.id, category: r.category, status: r.status, message: r.message })),
+	};
+}
+
+/**
+ * Estimate score from results when the raw JSON doesn't include a score field.
+ * Uses a simple formula: (pass / (total - skip)) * 100.
+ */
+function estimateScore(results) {
+	const scored = results.filter((r) => r.status !== 'skip');
+	if (scored.length === 0) return 100;
+	const passing = scored.filter((r) => r.status === 'pass').length;
+	// Warnings count as half-pass
+	const warnings = scored.filter((r) => r.status === 'warn').length;
+	return Math.round(((passing + warnings * 0.5) / scored.length) * 100);
+}
+
+function printSummary(report) {
+	console.log(`\nAFDocs Audit — ${report.url}`);
+	console.log(`Score: ${report.score}/100 (${report.grade})`);
+	console.log(
+		`Checks: ${report.total_checks} total | ${report.summary.pass} pass, ${report.summary.fail} fail, ${report.summary.warn} warn, ${report.summary.skip} skip`
+	);
+
+	if (report.issues.length === 0) {
+		console.log('\n✅ All checks passed!');
+		return;
+	}
+
+	const failures = report.issues.filter((i) => i.status === 'fail');
+	const warnings = report.issues.filter((i) => i.status === 'warn');
+
+	if (failures.length > 0) {
+		console.log(`\nFailures (${failures.length}):`);
+		for (const f of failures) {
+			console.log(`  ✗ ${f.id}: ${f.message}`);
+			if (f.fix) console.log(`    Fix: ${f.fix}`);
+		}
+	}
+
+	if (warnings.length > 0) {
+		console.log(`\nWarnings (${warnings.length}):`);
+		for (const w of warnings) {
+			console.log(`  ⚠ ${w.id}: ${w.message}`);
+		}
+	}
+}
+
+// Main
+const args = parseArgs(process.argv.slice(2));
+console.log(`Running AFDocs check on ${args.url}...`);
+
+const raw = runAfdocsCheck(args.url);
+const report = buildReport(raw);
+
+printSummary(report);
+
+if (args.output) {
+	const outputPath = resolve(args.output);
+	writeFileSync(outputPath, JSON.stringify(report, null, 2));
+	console.log(`\nReport written to ${outputPath}`);
+}