From af1df5352dc6fdeee5c7d3c62ac5ac608e520ae1 Mon Sep 17 00:00:00 2001 From: JP Date: Fri, 7 Aug 2026 12:30:22 +0100 Subject: [PATCH] ci: actually enforce Agent Plugins conformance in the pipeline MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CI workflow invokes each validator script directly rather than running `make gate`, so adding `spec` to the Makefile target never reached CI — validate_agent_plugins.py has not run on a single push. The README claim that conformance is "enforced in CI by make spec" was therefore false when written. Adds the missing step, and an advisory one that validates against the UPSTREAM reference validator and the published JSON schemas rather than our transcription of them, so a spec revision upstream surfaces as a warning instead of silently drifting. That step is network-dependent, so it does not block; the deterministic in-repo gate does. Cherry-picked from the parallel branch on PR #4, which spotted this. Also refreshes research/coverage-matrix.md, which still carried a 2026-07-23 date and no conformance target. --- .github/workflows/gate.yml | 20 ++++++++++++++++++++ research/coverage-matrix.md | 6 +++++- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/.github/workflows/gate.yml b/.github/workflows/gate.yml index 7176684..d6ee960 100644 --- a/.github/workflows/gate.yml +++ b/.github/workflows/gate.yml @@ -22,6 +22,9 @@ jobs: - name: Install deps run: pip install pyyaml pytest + - name: Validate Agent Plugins 1.0.0 conformance + run: python3 scripts/validate_agent_plugins.py + - name: Validate skill contracts run: python3 scripts/validate_skills.py @@ -36,3 +39,20 @@ jobs: - name: Smoke tests run: python3 -m pytest tests/skill-smoke-tests/ -q + + # Advisory: validates against UPSTREAM rather than our transcription of it, + # so it catches spec drift early. Network-dependent, so it warns instead of + # blocking — validate_agent_plugins.py above is the deterministic gate. + - name: Independent conformance check (upstream reference validator) + continue-on-error: true + run: | + pipx install uv >/dev/null 2>&1 || pip install uv + for skill in skills/*/; do + uvx --quiet --from \ + 'git+https://github.com/agentskills/agentskills.git#subdirectory=skills-ref' \ + skills-ref validate "$skill" || exit 1 + done + uvx --quiet check-jsonschema \ + --schemafile https://agent-plugins.org/schemas/1.0.0/plugin.schema.json plugin.json + uvx --quiet check-jsonschema \ + --schemafile https://agent-plugins.org/schemas/1.0.0/mcp.schema.json mcp.json diff --git a/research/coverage-matrix.md b/research/coverage-matrix.md index 41d029b..8862d70 100644 --- a/research/coverage-matrix.md +++ b/research/coverage-matrix.md @@ -6,7 +6,11 @@ Tracks skill status for every first-wave service. Update whenever a skill progre **v0.1 target:** All first-wave services at `validated` before release tag. -Last updated: 2026-07-23 (Phase 1 crawl complete — 75/75 sources resolve, 70 unique URLs live, gate green) +**v0.2 target:** every skill conformant with the [Agent Skills specification](https://agentskills.io/specification) +and discoverable under [Agent Plugins 1.0.0](https://agent-plugins.org/specification) — met at v0.2.0 +(17/17 valid per the upstream `skills-ref` reference validator). + +Last updated: 2026-08-07 (v0.2.0 — Agent Plugins 1.0.0 conformance; 17/17 skills valid, 102 URLs resolve, gate green) ---