Skip to content

feat(skills): gate the skills against the API the platform actually ships - #82

Merged
stxkxs merged 1 commit into
mainfrom
feat/gate-the-skills-against-the-shipped-api
Aug 5, 2026
Merged

feat(skills): gate the skills against the API the platform actually ships#82
stxkxs merged 1 commit into
mainfrom
feat/gate-the-skills-against-the-shipped-api

Conversation

@stxkxs

@stxkxs stxkxs commented Aug 5, 2026

Copy link
Copy Markdown
Member

The skills go into the preamble of every agent session fab runs, so a kind named there is a kind agents will confidently author.

When the batch tier was deleted from eks-agent-platform, BatchJob survived in eks-agent-platform-curation.md — and every session kept being told the platform had a batch API. Nothing could catch it: fab vendors no CRDs, so there was no local list to compare against. It was found by reading, and reading is not a thing that keeps finding it.

What it checks

scripts/check-skill-resources.mjs compares the kinds the skills enumerate against the CRDs the operator chart actually ships, sparse-checked-out in CI — the same mechanic nanohype/scripts/check-slo-constants.mjs already uses to reach this repo.

Both directions, and the second is why this isn't a one-way check:

  • A kind the skills name that no CRD defines — the batch case. Agents are taught an API that does not exist and find out at admission.
  • A kind the operator ships that no skill mentions — quieter, and it is exactly how SLOPolicy came to have a controller, a CRD, a kube-state-metrics projection and three Grafana alerts while nothing in the org ever authored one.

Only API-group enumerations and definition bullets are read, not free prose. A skill may legitimately say "platform" or "tenant" as English; what has to be accurate is where it enumerates the API.

Vacuity

Both empty sets fail rather than passing quietly — a CRD directory that parses to nothing, and a skills tree where no enumeration matches. A gate whose healthy output matches its did-nothing output is the failure it exists to catch.

mutation result
add BatchJob back to a group line FAIL — named but no CRD defines it
remove SLOPolicy from its group line and bullet FAIL — shipped but no skill names it
point at an empty CRD directory FAIL (exit 2) — "asserting nothing"

Currently green: the skills name exactly the nine kinds the operator ships.

…hips

The skills go into the preamble of every agent session fab runs, so a kind named
there is a kind agents will confidently author.

When the batch tier was deleted from eks-agent-platform, `BatchJob` survived in
eks-agent-platform-curation.md, and every session kept being told the platform
had a batch API. Nothing could catch it: fab vendors no CRDs, so there was no
local list to compare against. It was found by reading, and reading is not a
thing that keeps finding it.

`scripts/check-skill-resources.mjs` compares the kinds the skills enumerate
against the CRDs the operator chart ships, sparse-checked-out in CI.

Both directions, and the second is why this is not a one-way check:

  * A kind the skills name that no CRD defines — the batch case. Agents are
    taught an API that does not exist and find out at admission.
  * A kind the operator ships that no skill mentions. Quieter, and it is how
    SLOPolicy came to have a controller, a CRD, a kube-state-metrics projection
    and three Grafana alerts while nothing in the org ever authored one.

Only API-group enumerations and definition bullets are read, not free prose — a
skill may say "platform" or "tenant" as English, and what has to be accurate is
where it enumerates the API.

Both empty sets fail rather than passing vacuously: a CRD directory that parses
to nothing, and a skills tree where no enumeration matches.

Negative-tested, each failing as intended: adding BatchJob back to a group line,
removing SLOPolicy from both its group line and its bullet, and pointing the
check at an empty CRD directory.
@stxkxs
stxkxs merged commit 0a8825b into main Aug 5, 2026
11 checks passed
@stxkxs
stxkxs deleted the feat/gate-the-skills-against-the-shipped-api branch August 5, 2026 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant