Skip to content

Releases: zivtech/accessibility-skills

v1.2.0 — Cross-detector corroboration, supplemental WAVE/Alfa lanes, and a batched per-page audit default

Choose a tag to compare

@AlexU-A AlexU-A released this 14 Sep 17:54
1a1a56f

Supported runtime is unchanged: Claude, via the skills in .claude/skills/ and
the subagent lane behind /a11y-workflow. Everything added since v1.1.0 is either
a default-behavior change inside an existing skill, benchmark infrastructure, or
cost discipline. The local Ollama lane remains a detector, never a verdict
authority.

New since v1.1.0:

  • Cross-detector corroboration + two supplemental detector lanes (#83). A new
    optional evidence-contract status (detected_by + corroboration: single|corroborated) turns "two independent scanners agree on the same WCAG
    criterion + target" into a structured triage-confidence tag — corroborated,
    never "confirmed" (which stays the human/AT tier), never a conformance input.
    WebAIM WAVE and Siteimprove Alfa join a11y-test as optional, detector-only
    supplemental lanes (routed never vendored, operator's own licence, disclaimers
    verified against the live terms) for broad automated coverage + aggregate
    signal. Their unique-coverage delta over axe is deliberately left unmeasured;
    a five-page head-to-head remains the promotion trigger.
  • Batched one-pass per-page audit is now the DEFAULT for rendered-context
    adjudication (#74) — one pass per page instead of per-finding, for the
    rendered-context path the audit skills route through.
  • acr-reporting rule-text closed out (#62) — both open questions from #57
    decided: no sixth admissibility rule, and authorship lives on the attestation
    block. Scorer hardened to match tokens against normalized prose, with #57's
    rows retested against the post-#62 instrument (#64). ACR independence canary
    added as fixture #8, plus the first hosted rows on the Phase 2 fixtures (#61).
  • Context-utilization Phases 0-4.3 (#66, closes #28) — context guard, the
    a11y-evidence-reader lane, and the evidence-volume eval lane, so large
    evidence corpora stay out of the main session.
  • Cross-model critic benchmarks expanded, all detector-lane, no change to the
    recommendation:
    • Gemini 3.8 Flash a11y-critic benchmark with an effort sweep, plus the agy
      runner (#79).
    • Newer local-model critics + local ensemble simulation; qwen3.6:35b holds
      (#80).
    • gpt-oss:120b critic run completed and its earlier "OOM, not runnable"
      recorded as a co-residency misdiagnosis, not a hardware ceiling; it runs
      but stalls on the Phase 0 gate and is not a functioning critic (#81).
    • qwen3.8:27b unblocked by the retry-on-empty runner fix (33/33, champion
      still holds) (#82).
    • qwen pre-sort + cost accounting, Stream 2 Stage-A, no hosted spend (#78).
  • Eval integrity and mirror hygiene:
    • The four unread rubric scoring fields removed from scorer-read positions
      across a11y-critic and perspectives; trap prose relocated to a labeled
      non-scored key (#69), after a probe measured them worth 0-10% of the
      behavior they named (#65). Refs #63.
    • Run manifests re-anchored onto published commits and the verifier restored
      (#70, closes #67).
    • check_mirrors full-text parity gated on mirror pairs (#68, closes #27).
    • Federal planner condition and authorship regression coverage finished
      (#72); content-review returns validated and decision history preserved
      (#71).
  • Cost discipline: CLAUDE.md trimmed to current-state instructions with a
    doc-budget gate (#75); screenshots and image reads kept out of the main
    session (#76); a11y-test progressive-disclosure trim, ~33% off (#77).

Still open, tracked in the issue queue:

  • #57 stays open by design — its final Phase 2 exit criterion is one real retest
    from an engagement, an event rather than code. Phases 1-2 are on main.
  • #63 residual: the a11y-planner rubrics still carry false_positive_trap in 2
    files (a11y-critic and perspectives are fully remediated).
  • #17: Codex/Gemini/Claude-API rows and cloud planner-federal parity.

Refs #17 #27 #28 #57 #63 #74 #83

v1.1.0 — Human verification, OpenACR reporting, and a blind protocol with its leaks measured

Choose a tag to compare

@AlexU-A AlexU-A released this 04 Sep 11:15
57d0894

Supported runtime is unchanged: Claude, via the skills in .claude/skills/ and
the subagent lane behind /a11y-workflow. Everything else remains benchmark
infrastructure, and the local Ollama lane remains a detector, never a verdict
authority — now measured under an A/B rather than inferred.

New since v1.0.0:

  • acr-reporting (#20) — OpenACR Lane A. Serializes a finished audit-scope
    evaluation into a draft ACR, validated and rendered through the exact-pinned
    @openacr/openacr CLI. Always a DRAFT for human sign-off. Phase 2 gate passed;
    Phases 0-4 landed (#21, #24).
  • Human verification stage (#57, via #58 and #59) — attestation on fix closures,
    the acr-reporting unattested-closure gate, the walk-through reference, and
    human-sourced package shapes. A closure attests only on a walked PASS.
  • a11y-test execution mode 6 (#3257637) — baseline URL-list scan, plus --resume
    and a coverage block (#43).
  • maintain-accessibility-skills (#23) — repo hygiene, history-rewrite
    verification, mixed-commit surgery.
  • Verification evidence contract, census heuristics, partial-fix fixtures (#22);
    operation-evidence admissibility with a structured disposition block and
    scorer (#44); append-only evidence checksums (#45); the XLSX error-workbook
    builder with cell-by-cell --verify (#54).
  • Tier-2 learnings folded across a11y-test, acr-reporting, a11y-critic and
    a11y-planner (#33-#38).
  • Blind-prompt protocol (#56) — machine-enforced over every eval suite: one
    canonical cut-line spelling, every heading declared, every fixture title
    pinned, self-tested negative controls in CI.
  • August 2026 funnel reopen: qwen3.8:27b stopped at Stage 1 (#26). Detector
    recommendation unchanged.

Measured, and disclosed as a limit rather than a result (#60): neither fixture
answer-key leak moves detection — H1 titles 150/150 in both arms, features
section 89/90 in both — but the CLEAN half of the critic suite returns a clean
ACCEPT 6/55 of the time with the answer key present and 2/55 without. Both
lanes are null results at low power (a ~3x swing was the smallest catchable
effect), on one local model, with no hosted row.

Still open on #17: Codex/Gemini/Claude-API rows and cloud planner-federal
parity. Open work is tracked in the issue queue.

Refs #17 #51 #57

v1.0.0 — Accessibility Skills, Claude-supported

Choose a tag to compare

@AlexU-A AlexU-A released this 12 Aug 19:08

First tagged release of the standalone Accessibility Skills bundle.

Supported runtime: Claude

v1 supports Claude — the Claude Code skills in .claude/skills/ (npx skills add zivtech/accessibility-skills) and the Claude subagent lane behind /a11y-workflow. Validation receipts for that surface:

  • Opus subagent planner lane: 25/25 PASS across the original planner suite (2026-06-12, evals/results/claude-planner/).
  • Declared-508 federal rows (2026-08-12, evals/results/ict-baseline-phase3/): 11/11 PASS in both crosswalk conditions — the federal condition citing all 62 ICT Testing Baseline web test IDs with a machine-verified exact transcription of the coverage crosswalk, the withheld condition hedging an unfilled "N of 62" with a written fabrication rationale — and a declared-508 bug report at WARN with the lane's first-ever verified stable-ID recomputation (2/2) and zero fabrications.

What's in the box

  • Seven skills covering the accessibility lifecycle: a11y-planner, a11y-critic (two lifecycle checkpoints), a11y-test (five routed execution modes), perspective-audit (7 access perspectives), a11y-role-audit (ARRM roles), a11y-workflow (orchestrator), bug-reporting — with Codex-format mirrors in .agents/skills/.
  • Reporting contracts: the A11y Evidence Finding Contract and the A11y Evaluation Report Contract (WCAG-EM Step 5 shape), with stable fingerprints, trend vocabulary, and severity kept orthogonal to conformance outcomes.
  • Audit-scope methodology: WCAG-EM 2.0 wired into the planner's AUDIT-SCOPE MODE and a11y-test's sampling discipline (structured + 10% random + representativeness check + complete processes).
  • Federal (Revised Section 508) support, adopted through Phase 3 of the ICT Testing Baseline plan: a verified upstream reference pinned at atbcb/ICTTestingBaseline main @ 6c537a3b with a machine-readable test-ID manifest (62 web / 57 documents); the planner FEDERAL PROFILE (the conformance floor declaration is the scope gate and the engagement's conformance target); a hand-built coverage crosswalk (22 covered / 26 partial / 13 not-covered / 1 always-passes — the not-covered rows are the deliverable); optional manifest-validated baseline_test finding fields; an optional federal annex on the report contract; and calibrated eval instruments with baseline-ID fabrication checks (16 synthetic calibration cases, inertness across all prior lanes, nine adjudicated model rows with zero ID fabrications).
  • Evaluation suites: 33 critic, 25 perspective, 27 planner, 7 bug-reporting, plus evaluation-report and chain lanes — fixtures, rubrics, de-hint enforcement, blind-prompt guards, and scorer smoke tests in CI.

Not part of the v1 support surface (TBD, tracked on #17)

  • Local Ollama models — the lane ships as benchmark infrastructure and its own results define its role: a detector, never a verdict authority. No local model is a supported v1 runtime.
  • Codex / Gemini / Claude-API rows for the newest (federal) instruments.
  • Cloud-runner planner-federal condition parity.
  • The batched instrument-revision candidates from the first federal rows.

Known issues

  • The condensed .claude/agents/ definitions predate the Phase 2 FEDERAL PROFILE (the skills themselves are current); a sync plus a marker-based drift check is in flight.
  • Upstream watch items per the adoption assessments (ICT baseline working branches, TT course churn, WCAG-EM recheck triggers).

Hard boundaries that hold everywhere: no Trusted Tester certification claims, no self-declared "baseline-aligned" badge (only "designed to cover N of 62; gaps: …"), no ACR/VPAT generation, and the WCAG 2.2 AA default target never lowers to the federal 2.0 floor.

🤖 Generated with Claude Code