Skip to content

Development

m4bwav edited this page Oct 26, 2026 · 2 revisions

Layout

Path What
skills/askwright/ The skill as it ships: SKILL.md, references/schema.md, references/writing-questions.md and references/study-mode.md, scripts/ask.py, scripts/page/.
tests/ check_skills.py (structure, links, LF line endings, no keys or home paths), test_ask.py (lint rules, server contract, Range, traversal, tokens, status and read), test_page.py (the page in a browser with axe-core), fixtures and a vendored axe.min.js.
evals/ The eval suite for the skill (evals.json, cases and graders, a fixture with a clip, a tone and an image).
scripts/budget.py The install size and description budget.
RESEARCH.md, CHANGELOG.md, LEARNINGS.md, TESTS.md, MAINTENANCE.md, evergreen.json The skill's research, reasoned changes, lessons, test runs and refresh schedule.
ai-docs/ Plans, decisions, notes and the log; HANDOFF.md says where things stand.
AGENTS.md The rules for any agent working in the repository.

Running the checks

python tests/check_skills.py
python tests/test_ask.py
python scripts/budget.py
python tests/test_page.py

On 2026-10-25 (0.2.0), on Windows with Python 3.14, check_skills.py reported 0 problems and test_ask.py ran 67 tests, OK with 1 skipped. test_page.py needs pip install playwright and an installed Chrome or Edge, since Playwright's own Chromium does not play H.264. ASKW_SET=<set folder> python tests/test_page.py GenericPageTests runs the generic page checks against a real set, on a temporary copy.

CI

tests.yml runs on every pull request and every push to main: check_skills.py, test_ask.py and budget.py on Ubuntu, macOS and Windows with Python 3.9 and 3.14 (six jobs), and the page tests in Chrome on Ubuntu (one job).

Releasing

  1. Raise the version in .claude-plugin/plugin.json and VERSION in scripts/ask.py, and add a line under "Release notes" in the CHANGELOG.
  2. Merge, then push a tag vX.Y.Z that matches plugin.json.
  3. release.yml checks that the tag and plugin.json agree, runs the three checks, zips skills/ with README.md, LICENSE and SECURITY.md into askwright-X.Y.Z.zip, and publishes the GitHub release.

Evals

evals/evals.json holds trigger cases, decoys, action cases proven by evidence on disk and an outcome case graded against a rubric. Trigger and decoy cases run through claude plugin eval. On native Windows the cases that need Bash cannot run there, so they were run through the evergreen plugin's worth --ab with a baseline without the skill. The results of each run are in TESTS.md; evals/results/ is git-ignored because it holds local paths.

Rules worth knowing before a change

  • ask.py stays standard-library Python 3.9 or later and prints one JSON object per command.
  • The page stays plain HTML, CSS and JavaScript with no build step, no CDN and no inline script; the server's Content Security Policy forbids inline script.
  • Every new lint rule, command, option or API route gets a test in test_ask.py; every page behaviour one in test_page.py.
  • Servers are stopped by PID only, never by process name.

The full list is in AGENTS.md.

This wiki

Every output on these pages was printed by ai-docs/notes/2026-10-25-wiki-verify.py, run against the unpacked release zip; the saved output is beside it. ai-docs/notes/2026-10-24-github-wiki.md says how to update the wiki at the next release. The figure pictures are SVG files in the wiki repository, copied from that program's run.

Clone this wiki locally