ci: build changed registry definitions on the pull request - #134
Merged
Conversation
ci.yml lints, builds and tests the TypeScript packages; nothing there reads registry/*.yaml. A definition naming a branch or docs_path that does not exist passes review and CI unchallenged, then fails at 06:00 UTC in the nightly registry-update — after the PR that introduced it is merged and out of mind. That is the same shape as the sanity breakage that hid for four nights: the check that would have caught it ran nowhere near the change. #133 made it concrete. Two new definitions arrived with zero check runs, so verifying them meant building both by hand. This builds the definitions a PR adds or modifies, on the PR. Only changed ones, so cost tracks the diff rather than the 113-definition registry. Verified before committing, since a validator that passes broken input is worse than none: - A nonexistent git ref exits 1. - A nonexistent docs_path exits 1 rather than producing an empty package, which was the failure mode worth ruling out. - Name extraction handles a name that differs from its filename (registry/hex/phoenix.yaml is "phoenix"), quoted names, and a missing name: field. - The diff detection returns exactly the two definitions from #133 and excludes deletions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NBQQpA86yYzwJUiVz8ph2R
|
Adds a definition naming a branch that does not exist. The new workflow must trigger on it and go red. Reverted in the next commit — this exists only to prove the wiring, which #134 cannot test on itself since it changes no registry files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NBQQpA86yYzwJUiVz8ph2R
The probe did its job. On the previous commit the workflow triggered, resolved the definition name, and failed with: Git clone failed: fatal: Remote branch this-branch-does-not-exist-xyz not found in upstream origin ##[error]definition 'ci-probe-delete-me' failed to build Process completed with exit code 1 That is the failure surfacing for the right reason rather than incidentally, which is the only version of this check worth having. The two commits are kept rather than squashed away so the evidence stays in the branch history. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NBQQpA86yYzwJUiVz8ph2R
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ci.ymllints, builds and tests the TypeScript packages. Nothing readsregistry/*.yaml.So a definition naming a branch or
docs_paththat doesn't exist passes review and CI unchallenged, and first fails at 06:00 UTC in the nightlyregistry-update— after the PR that introduced it is merged and out of mind. Same shape as the sanity breakage that hid for four nights: the check that would have caught it ran nowhere near the change.#133 made it concrete. Two new definitions arrived with zero check runs, so verifying them meant cloning the upstream repo and building both by hand.
What it does
On a PR touching
registry/**/*.yaml, builds each definition the PR adds or modifies via the existingtest-registry test <name>CLI. Only changed definitions, so cost tracks the diff rather than the 113-definition registry. No new tooling — it runs the command a maintainer would run locally.Verification
This PR changes no registry files, so it cannot exercise its own workflow. Rather than merge on local component tests alone, I pushed a deliberate probe (commit
75ea47d) adding a definition pointing at a nonexistent branch, then removed it (856434f). Both commits are kept rather than squashed so the evidence stays in the history.The probe run failed exactly as required, and for the right reason rather than incidentally:
That confirms the whole chain end to end: the
paths:filter triggered, diff detection found the file, the name resolved, the build failed on the bad ref specifically, the error annotation fired, and the job exited non-zero.Checked before that, locally:
docs_path→ exits 1, rather than "succeeding" with an empty package. The failure mode worth ruling out, since a silent empty build would sail through a naive checknamediffering from its filename (registry/hex/phoenix.yaml→phoenix), quoted names, and a missingname:fieldgdscript129 sections / 60,921 tokens,godot14,488 sections / 8,136,608 tokenspermissions: {}at top level withcontents: readon the job; no secrets, so it works on fork PRs.Note on cost
timeout-minutes: 45. A definition can pull a large documentation repo —godotbuilds ~8M tokens from 3.5k files — so the ceiling is deliberately generous but bounded. If large-repo definitions become common, the thing worth deciding is a size budget, not the timeout; there's no documented ceiling in the registry today.🤖 Generated with Claude Code
https://claude.ai/code/session_01NBQQpA86yYzwJUiVz8ph2R