Skip to content

MetaSkills 1.2 — Validate the system around the agent

Latest
Latest

Choose a tag to compare

View all tags
@smota smota released this 16 Aug 15:42
1.2
This tag was signed with the committer’s verified signature.
smota Samuel
SSH Key Fingerprint: LKQckbz3EYbLxLJCI94u0ALdmp2+h2Vc8A32gBW1wxA
Verified
Learn about vigilant mode.
2559942
This commit was signed with the committer’s verified signature.
smota Samuel
SSH Key Fingerprint: LKQckbz3EYbLxLJCI94u0ALdmp2+h2Vc8A32gBW1wxA
Verified
Learn about vigilant mode.

MetaSkills 1.2 — Validate the system around the agent

MetaSkills 1.2 adds harness-validator, a portable, read-only skill for checking whether a repository gives AI coding agents a reliable working system—not merely a prompt file.

What changed

  • Added a complete harness assessment across nine capability categories and 78 canonical checks.
  • Added explicit applicability, evidence-state, maturity, confidence, risk, effort, provenance, exception, and recommendation fields.
  • Added non-averaged blockers for unsafe access, exposed secrets, missing executable completion criteria, unreproducible setup, worker-only completion, and unsupported cross-boundary claims.
  • Added conflict analysis for WIP limits, repository versus external state, reopened evidence, flaky-test baselines, evaluator independence, documentation size, and E2E scope.
  • Added course-gap coverage for prompt injection, supply chain, traceability, recovery, non-functional quality, governance, provenance, human comprehension, portability, test strength, and cost.
  • Added Markdown and JSON report templates, a strict assessment validator, a template generator, deterministic eval fixtures, and focused tests.

Why it matters

Strong models still fail when instructions are contradictory, setup is implicit, progress is lost, completion is self-declared, runtime behavior is invisible, or autonomous loops lack containment. harness-validator helps builders find those system-level weaknesses using concrete evidence and prioritize the smallest effective improvements.

The skill evaluates capabilities rather than demanding particular files or products. Small projects can mark advanced orchestration checks not applicable; high-risk or autonomous projects receive stronger security, recovery, verification, and budget expectations.

How to try it

npx skills add https://github.com/smota/metaskills --skill harness-validator --full-depth

Then ask your coding agent:

Use harness-validator to audit this repository for long-running coding-agent work. Do not edit anything. Produce an evidence-backed report with blockers, conflicts, gaps, over-detailed controls, and prioritized improvements.

Validation

  • 14 deterministic unit tests passed.
  • The structured template validates all nine categories and 78 canonical checks.
  • Seven fixture-backed behavioral contract evals passed.
  • External skill-package validation passed.
  • Python compilation, JSON parsing, and diff checks passed.

Links

Assets 2
Loading