Repository navigation
CONFORMANCE AUDIT 2026 09
Historical. This document describes the repository as it was before version 7.0, when agents lived in six per-client directories. It is kept for the record and is not maintained. The current layout is one
skills/directory; start at the documentation index. Original path:docs/CONFORMANCE-AUDIT-2026-09.md.
Audited: 21 September 2026
Steps 1-5 applied: 21 September 2026. All 152 violations cleared;
node scripts/check-skill-conformance.mjs now exits 0.
Scope: skill and agent frontmatter across all platform trees, and the MCP
server's conformance with the current protocol surface.
Method: node scripts/check-skill-conformance.mjs, plus manual inspection
of mcp-server/. Every number below is reproducible from that command.
152 frontmatter violations, concentrated almost entirely in one tree, plus two MCP gaps that cost real usability.
| Tree | Files | BOM | No frontmatter | Non-slug name |
|---|---|---|---|---|
.gemini/extensions/a11y-agents/skills |
101 | 70 | 3 | 64 |
codex-skills |
80 | 0 | 0 | 7 |
claude-code-plugin/agents |
80 | 0 | 0 | 7 |
.claude/agents |
7 | 0 | 0 | 1 |
codex-plugin/skills |
5 | 0 | 0 | 0 |
.github/skills |
26 | 0 | 0 | 0 |
By rule: 79 non-slug names, 70 byte-order marks, 3 files with no frontmatter.
None of this is new drift. It has been there long enough to be invisible, and the reason is in section 2.
scripts/validate-agents.js already has the correct rule. Line 645:
if (frontmatter.name && frontmatter.name !== folderName) {
errors.push(`${relativePath}: Skill name '${frontmatter.name}' must match folder name '${folderName}' per agentskills.io spec`);
}The rule is right. The problem is where it looks. validate-agents.yml
and the validator itself scan four trees:
.github/agents.github/skills.claude/agentsclaude-code-plugin/agents
The Gemini extension and codex-skills/ are not in that list, and never have
been. Both trees have therefore never been validated by anything, on a
repository that validates its other trees on every pull request.
That explains the distribution exactly: the two unscanned trees hold 144 of the 152 violations, and the two well-scanned skill trees hold zero.
70 of 101 Gemini SKILL.md files begin with a UTF-8 BOM (U+FEFF) before the
--- delimiter.
This is not cosmetic. A strict YAML frontmatter parser looks for --- at
byte zero; with a BOM in front of it, the frontmatter block is not
recognised at all, and the skill loses its name and description silently.
Nothing errors. The skill simply does not describe itself.
It is also the failure mode most likely to have been introduced by tooling
rather than by hand - PowerShell's Out-File and Set-Content write a BOM by
default on Windows, which matches a repository with .ps1 installers and
generators.
Fix: node scripts/check-skill-conformance.mjs --fix-bom. It strips the mark
and leaves everything else untouched.
The Agent Skills frontmatter spec requires name to be lowercase letters,
digits and hyphens. 79 files declare Title Case names while sitting in
correctly-slugged folders:
| Declared | Folder |
|---|---|
Accessibility Lead |
accessibility-lead |
Alt Text & Headings |
alt-text-headings |
ARIA Specialist |
aria-specialist |
ePub Accessibility |
epub-accessibility |
i18n Accessibility |
i18n-accessibility |
PowerPoint Accessibility |
powerpoint-accessibility |
The folder name is already correct in every case, so the fix is mechanical:
set name to the folder name. The human-readable title belongs in
description, or in the body's # heading, where it already is.
Seven of these are the same agents mirrored across claude-code-plugin/agents
and codex-skills, so one decision fixes both:
a11y-tool-builder, desktop-a11y-specialist, desktop-a11y-testing-coach,
developer-hub, nvda-addon-specialist, python-specialist,
wxpython-specialist.
.claude/agents/developer-hub.md is the eighth, and the same fix.
Three Gemini skills have no frontmatter block:
.gemini/extensions/a11y-agents/skills/nvda-addon-specialist/SKILL.md.gemini/extensions/a11y-agents/skills/playwright-testing/SKILL.md.gemini/extensions/a11y-agents/skills/wxpython-specialist/SKILL.md
These are not skills as far as any conformant loader is concerned. They need a
name and a description added, copied from their counterparts in the other
trees.
-
server.registerToolthroughout - all 38 tools, the current API rather than the deprecatedserver.tool(). -
Both transports - Streamable HTTP (
server.js) and stdio (stdio.js), sharing tool implementations throughserver-core.js. Streamable HTTP is the current remote transport; nothing here is on the retired SSE-only path. -
A bundle manifest template is present (
mcpb-manifest.template.json).
Not one tool declares annotations. No readOnlyHint, no destructiveHint, no
idempotentHint, anywhere.
This is the highest-value fix in this document, and it is not about
conformance for its own sake. 29 of the 38 tools are read-only scanners.
A client that cannot tell a scan from a mutation has to prompt the user for
every single call. Marking the scanners readOnlyHint: true lets clients
auto-approve them, which is the difference between scanning forty documents
and giving up after six - on a server whose entire purpose is bulk scanning.
Read-only (29): batch_scan_documents, check_apca_contrast,
check_audit_cache, check_color_blindness, check_contrast,
check_form_labels, check_heading_structure, check_link_text,
check_reading_level, check_verapdf_installation,
extract_document_metadata, get_accessibility_guidelines,
get_audit_result, get_audit_trend, glow_audit_document,
glow_generate_report, glow_health_check, list_audit_history,
run_axe_scan, run_playwright_a11y_tree, run_playwright_contrast_scan,
run_playwright_keyboard_scan, run_playwright_viewport_scan,
run_verapdf_scan, scan_epub_document, scan_markdown_document,
scan_office_document, scan_pdf_document, validate_caption_file.
Writes or creates (8): convert_pdf_form_to_html, fix_document_headings,
fix_document_metadata, generate_accessibility_statement,
glow_convert_document, glow_fix_document, save_audit_result,
update_audit_cache.
Destructive (1): prune_audit_history. This one should carry
destructiveHint: true explicitly, because it is the only tool here that
removes data and the only one where an accidental auto-approval would cost
something.
No tool declares an outputSchema. Every result is returned as text for the
model to re-parse. For scanners that already produce structured findings -
severity, WCAG reference, location - this throws away the structure on the way
out and asks the model to reconstruct it.
Worth doing for the scan and check families first; the rest can follow.
mcp-server/tools/glow-tools.js defaults to https://glow.bits-acb.org/mcp.
GLOW now runs at letitglow.app/mcp. mcp-server/README.md repeats the old
host.
Every GLOW bridge tool - glow_audit_document, glow_fix_document,
glow_convert_document, glow_generate_report, glow_health_check - points
at a host that is not the service. Two lines, and it fails the first time
anybody tries it.
Note: glow-tools.js is currently untracked in the working copy, so this
should be fixed by whoever is actively working on it rather than by a
drive-by edit.
Currently on @modelcontextprotocol/sdk ^1.27.1. The latest v1 is
1.30.0 - a minor bump with no migration.
v2 is a real change, and smaller than it sounds here. The monolithic
@modelcontextprotocol/sdk package is replaced by @modelcontextprotocol/server
and @modelcontextprotocol/client. That is a genuine architecture split, but
this repository's import surface against the SDK is three symbols:
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";Three imports across two entry points. Everything else in mcp-server/ is
tool logic that never touches the SDK directly.
Honest sizing: the migration is a day, not a sprint - dominated by re-testing 38 tools against both transports rather than by rewriting code. It should be scheduled on its own merits and should not be attempted in the same change as the annotation work, so that if something breaks it is obvious which change broke it.
An earlier draft of this section said this work does not block the workshop and belongs entirely on a separate track. That was too narrow. "Does not block" and "should not be done first" are different claims, and conflating them missed something real.
This repository is part of the workshop's credibility surface. The session is named Accessibility Agents in Action. The take-home artifact points attendees here. The follow-through design assumes someone curious enough to look will find something that works. Right now, an attendee who installs the Gemini extension after the session gets 70 skills that silently lose their name and description to any strict frontmatter parser.
So the split is not by project. It is by one question: would a conference attendee ever encounter this?
The stale GLOW URL, the BOM strip, the 79 name slugs, the three missing
frontmatter blocks, and folding both unscanned trees into
validate-agents.js.
Every one is attendee-visible: together they decide whether the extension works for someone who installs it on Monday. They are also the lowest-risk changes in this document - no logic, no judgment - and both the new checker and the existing validator prove them.
Second benefit: the workshop's prompt-card derivation reads these same agent definitions. Cleaning them first means that work starts from a tidy source.
Tool annotations, outputSchema, SDK 1.30, the v2 migration. All valuable;
none of it reachable by anyone in the room. Annotations are the highest
user-value item in this audit and they should still wait, because "highest
value" and "highest value before November" are not the same question.
Do steps 1 to 5 by mid-October, or defer the whole thing to December.
Two days of mechanical work eight weeks out is free. The same two days in the final fortnight competes directly with the workshop's timed dry run, and the readiness plan's freeze rule says ship nothing new in the final week. If it has not happened by mid-October, the answer changes to "after the conference" and nothing is lost by saying so.
Neither of these lists. The workshop's Phase 5 - load rehearsal at 30 participants, NVDA then JAWS then VoiceOver, the degraded-network run, the timed facilitator dry run, freeze week - has had no progress since 21 August and is entirely outstanding. It cannot be compressed, because it needs a room, a calendar and other people. Any scheduling decision here should be made in that light rather than against this document's own merits.
Each row, with its #, work and result.
| # | Work | Result |
|---|---|---|
| 1 | Stale GLOW base URL | 6 references repointed at letitglow.app across glow-tools.js and mcp-server/README.md
|
| 2 | Strip byte-order marks | 70 files |
| 3 |
name to the folder slug |
79 files. Frontmatter only; body prose untouched |
| 3b | The three files with no frontmatter | Two had a usable paragraph promoted into frontmatter. playwright-testing/SKILL.md was zero bytes and was replaced from its .github/skills counterpart |
| 4 | Stop it recurring |
skill-conformance.yml now runs on every pull request touching any of the six trees |
Verification: check-skill-conformance.mjs exits 0 across all 336 files;
validate-agents.js --strict still exits 0 on the trees it owns.
Folding the two trees into validate-agents.js was tried and reverted, for a
specific reason worth recording.
It works - 0 errors. But the workflow runs the validator with --strict, and
--strict fails on warnings. Those two trees carry 154 warnings of one
kind: "Description is N chars; agentskills.io spec recommends <200 chars."
The spec's hard limit is 1024. 200 is advice.
So folding them in would turn CI red on a recommendation, and going green would mean rewriting 154 descriptions - the exact text a model reads to decide whether to invoke a skill. Those descriptions are long because they enumerate invocation triggers, which is a reasonable thing for them to do.
That is a deliberate editorial decision, not a conformance cleanup, so it is
left to the maintainers. Until it is made, skill-conformance.yml gates the
two trees for byte-order marks, frontmatter presence and name slugs, which is
what produced 149 of the 152 violations.
Open: trim the 154 descriptions and fold the trees into the main validator, or agree the 200-character figure is advisory here and leave the split as it is.
Each remaining order, with #, work, size and why this order.
| # | Work | Size | Why this order |
|---|---|---|---|
| 1 | Fix the stale GLOW base URL | 2 lines | Live breakage, independent of everything else |
| 2 |
--fix-bom across the Gemini tree |
minutes | Mechanical, no judgment, removes 70 violations |
| 3 | Set name to the folder slug in all 79 files |
an afternoon | Mechanical; the folder name is already correct in every case |
| 4 | Add frontmatter to the 3 files that lack it | minutes | Copy from the counterpart in another tree |
| 5 | Add the two unscanned trees to validate-agents.js and retire this checker |
small | After 2-4, or CI goes red on main |
| 6 | Tool annotations, read-only first | a day | Highest user-visible payoff in this document |
| 7 |
outputSchema for the scan and check families |
a few days | Do after annotations, separately |
| 8 | SDK 1.27 → 1.30 | trivial | Independent |
| 9 | SDK v2 migration | a day, mostly re-testing | Last, and alone |
Step 5 is the one that keeps this from happening again. Everything before it is cleanup; without it, the same two trees drift again the moment someone adds a skill.
Steps 1 to 5 are the pre-conference set (section 6). Steps 6 to 9 wait until December.
node scripts/check-skill-conformance.mjs # the table in section 1
node scripts/check-skill-conformance.mjs --json # every finding, with paths
node scripts/check-skill-conformance.mjs --fix-bom # step 2 aboveExit code is 0 when clean and 1 when anything is found, so it drops into CI as
is. A workflow is in .github/workflows/skill-conformance.yml.
- Agents Overview
- Accessibility Lead
- Web Accessibility Wizard
- Document Accessibility Wizard
- Markdown Assistant
- Developer Hub
- GitHub Hub