[AAASM-5688] 🔧 (ci): Run the modules ref drift gate on a schedule, not only on config changes - #143
Merged
Conversation
The gate's path filters only fire when this repo edits modules.json, but the event it exists to catch — a source repo changing its default branch — happens elsewhere and touches nothing here. Measured 2026-08-13: 130 commits landed on main after the gate was added, one touched modules.json, and the gate evaluated 0 times in 20 days. This matters most for the drift shape that is invisible. A renamed-away ref stops existing and breaks aggregate.sh's checkout loudly; a ref naming a branch that still exists but is no longer the default builds green forever while publishing a frozen tree. Only a time-based run catches the second. Refs AAASM-5688
…gistry The registry described `ref` as "pins the checkout; use a tag/SHA to pin a release line" and its channel as "master/main HEAD" — neither states the rule that actually governs the field, and the tag/SHA affordance is no longer true: modules-ref-check.yml admits no exceptions and would fail such a pin. State the real contract at the pin site: `ref` must name the repo's live default branch, the gate resolves that with `git ls-remote --symref`, and a deliberate departure needs both a recorded reason and an opt-out in the gate — not a weakened gate. No module pins a tag today, so no opt-out is built. Refs AAASM-5688
The corrected $comment still said "'ref' pins the checkout" unqualified, a clause inherited verbatim from the text it replaced. It is false for 2 of 5 modules: aggregate.sh calls clone_module only when generator != mkdocs-material, and build_python hardcodes `git clone --depth 1 -b gh-pages`, so python-sdk and arena never check out at 'ref'. Leaving that in the very comment this PR rewrites to be a true record of the ref contract invites the concrete mistake: read "MUST name the repo's live default branch", set arena.ref to a tag to freeze a release line, get a red gate over a field with no effect on that build, then weaken the gate. Names the exception and points at AAASM-5728, which owns the inertness itself. Also tightens "on every modules.json change" to what the triggers actually do — a PR to main or a push to main. Refs AAASM-5688
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Description
Read this section first — otherwise the diff looks unrelated to the ticket.
AAASM-5688 reports that
modules.jsonpins"ref": "master"forcore,python-sdk,node-sdkandgo-sdkwhile those repos' default branch ismain. That flip already landed onmainon 2026-07-23, inecd3670,9ce0e37andce64923, and a drift gate (.github/workflows/modules-ref-check.yml) landed on 2026-07-24 ind691254— both roughly two weeks before the ticket was filed on 2026-08-07. All five modules onmaintoday read"ref": "main"."ref": "master"survives only in a stale local checkout, which was 138 commits behindorigin/main.So this PR does not flip any ref. It closes the hole that the already-merged gate left open.
The ticket's central measurement was an artifact
The ticket states
masterandmainare at identical SHAs in all five repos, and treats that as evidence amasterbranch still exists and is mirrored. It is not two branches — it ismainread twice.agent-assembly)git ls-remote --headsrefs/heads/master(25 heads, none namedmaster)gh api .../git/ref/heads/mastergh api .../branches/mastername: "main", shaca2e5e4e9Two controls, because an absence proves nothing on its own:
gh api .../branches/definitely-not-a-branch-xyzreturnsBranch not found, so thebranchesendpoint is not blanket-permissive. It specifically honours themaster→mainrename and reportsmain's SHA under the old name.git ls-remote refs/heads/mainreturns a SHA in all five repos, so the emptymasterresult is a real negative rather than a broken or unauthenticated probe.Same result in all five repos. There are no
mastermirrors. Consequence for risk: a rename fails loudly, not silently —aggregate.shrunsgit clonethengit checkout "$ref"underset -euo pipefail, so a vanished ref is a hardpathspec did not match. The gate's own header records this having broken aggregation three times in one week (AAASM-5047).The actual residual defect: the gate is correct and never runs
modules-ref-check.ymltriggers onpush/pull_requestfiltered topaths: [modules.json, the workflow itself]. The event it exists to catch — a source repo changing its default branch — happens in another repo and touches neither file.Measured 2026-08-13:
mainafter the gate was added; 1 of them touchedmodules.json.7b1cdf8(merge of PR [AAASM-5047] 🐛 (docs-hub): Pin core module ref to main #132), which landed onmainat 20:19:18 +08:00 — 64 seconds before the gate's own merge4cb6729(PR [AAASM-5048] ✨ (ci): Add modules.json ref-drift CI gate #133) at 20:20:22 +08:00.So the sharpest statement of the defect is not "it ran twice": the gate has never once evaluated a
modules.jsonit did not ship with. Its only push-triggered run has headSha4cb6729— its own merge commit. Zero evaluations in the 20 days since.A drift check that only runs when you edit the thing it checks cannot catch drift arriving from outside.
The two drift shapes are also not equally visible, which is why a scheduled run is required rather than merely tidy:
git checkoutfails loudly on the next aggregation cycle.gh-pagespin) clones and builds green forever while publishing a frozen tree — silently stale, which is the failure this ticket is about. Only a time-based run catches that one.Type of Change
What changed
Three commits, 2 files, +22/−1.
0506b24🔧 (ci): Run the modules.json ref-drift gate on a daily scheduleAdds
schedule: cron "17 6 * * *"tomodules-ref-check.yml. Daily rather thanaggregate.yml's*/30: this compares config against upstream default branches, which change on human timescales, and each run is fivegit ls-remotecalls. Offset from the top of the hour because GitHub's scheduler is congested there andscheduleis best-effort (AAASM-4508). The path filters are left in place — they still give fast feedback on config edits; the schedule covers what they structurally cannot.eee583c📝 (modules): Record the ref contract and its enforcing gate in the registryThe registry
$commentadvertisedrefas "pins the checkout; use a tag/SHA to pin a release line". That affordance is no longer true — the gate admits no exceptions and would fail such a pin. Rather than paper over the contradiction, the comment now states the real contract:refmust name the repo's live default branch, the gate resolves that withgit ls-remote --symref, and a deliberate departure needs both a recorded reason and an opt-out in the gate — not a weakened gate.No opt-out mechanism is built, because no module pins a tag today. An unused mechanism is untested code that reads as coverage.
7ed65bd📝 (modules): Name the modules whoserefdoes not control their buildThe rewritten
$commentstill carried the clause "'ref'pins the checkout" inherited verbatim from the text it replaced — false for 2 of 5 modules.aggregate.sh:355callsclone_moduleonly whengenerator != mkdocs-material, andbuild_pythonhardcodesgit clone --depth 1 -b gh-pages, sopython-sdkandarenanever check out atref. Leaving that inside the very comment this PR rewrites to be a true record of the ref contract invites a concrete mistake: read the strengthened "MUST name the repo's live default branch", setarena.refto a tag to freeze a release line, get a red gate over a field with no effect on that build, then weaken the gate. The clause now names the exception and points at AAASM-5728, which owns the inertness itself.Every claim that clause makes was asserted against the code, not from memory: the
mkdocs-materialmodule set is exactly{python-sdk, arena}; the guard is ataggregate.sh:355; thegh-pageshardcode is at:159; and$refappears only insideclone_module.Why pin
mainat all, rather than stop naming a branch?The tempting conclusion is that a hardcoded branch name caused this, so
refshould be dropped entirely andaggregate.shshould just follow the remoteHEAD. That inverts the actual lesson:An implicit-
HEADregistry means an upstream repo could repoint its default branch and the hub would publish the new tree with no diff in this repo, no PR and no review. That trades a detectable inconsistency for an undetectable one — the exact failure class this ticket is about.refalso has to survive for release-line pinning, which the registry advertises.The counter-case was checked rather than assumed:
build_goalready passesMASTER_REF="HEAD", and that is correct there, because it names no branch and no review value is lost by followingHEAD.So: an explicit pin, reviewed in a diff, plus a gate that resolves reality at check time. Both halves.
How to verify
The gate's logic was proven falsifiable before trusting it — it had never failed in CI, and a check that has never failed is measuring nothing.
The checker was extracted verbatim from the shipped workflow rather than retyped, so the thing under test is the artifact that decides, not a copy of it. Mutate → assert → restore ran in a single foreground command with restore guaranteed by a trap and verified by sha256.
core.ref = master(absent upstream)core.ref = trunk(arbitrary name)arena.ref = gh-pages(exists upstream, but is not the default)core.ref = v0.0.1-rc.6(a real release tag)Case D is the proof that matters. B and C would also fail a check that merely verified "the branch exists", which would be indistinguishable from a correct check. Case D is the only one of those that separates "resolves the default branch" from "branch exists" —
gh-pagesgenuinely exists inarena, and must still be rejected because it is not the default.Case E demonstrates the consequence the
$commentnow asserts rather than merely arguing it: the removed tag/SHA affordance really is removed.v0.0.1-rc.6was confirmed to be a real upstream tag first — against a non-existent tag the case would collapse into C and prove nothing.Case D failing output:
Restore hash matched the baseline in every case. Re-extracting the checker after the workflow edit confirmed it is byte-identical — only triggers changed.
Also verified:
actionlint(exit 0) in independent review, with two negative controls proving that exit code means something — a 4-field cron and17 99 * * *each exit 1. This is asserted rather than inferred from absent red, because an unparseable workflow shows zero check runs rather than a failing one. Locally,python-yamlconfirms all four triggers register (push,pull_request,workflow_dispatch,schedule).modules.jsonparses under bothpythonandjq(aggregate.shreads it withjq).generate_compatibility.py --check,generate_hub_components.py --checkandcheck_repo_names.pyall exit 0, captured as true exit codes rather than through a pipeline.check_repo_names.pyproven non-vacuous: injecting a stale repo name makes it exit 1.Two things a future reader would otherwise assume
modules.jsonchange" is really "every PR targetingmain, and every push tomain." Both triggers carrybranches: [main]in addition to the path filter, so amodules.jsonedit on a feature branch with no PR open is unchecked until the PR exists. Practically irrelevant here, but it is not the unconditional guarantee the phrasing suggests.scheduletriggers after 60 days of repository inactivity. Low risk fordocs, which is active, but it is worth naming as a silent-disable path for a gate whose entire purpose is not being silent.Related Issues
Documentation Checklist
cd docs && mdbook build— n/a, no book content changed (CI config + registry comment only)docs/src/SUMMARY.md— n/a, no page addedOut of scope
refis inert forpython-sdkandarena, and the branch that actually drives them is unguarded — now named in the$comment(commit7ed65bd) but not fixed here. The gate validates a field with no effect on those builds whilebuild_python's hardcodedgit clone -b gh-pagesgoes unchecked; if a module'sgh-pagesstops being republished, the hub re-copies the old tree and deploys green. Being fair to the current state:python-sdk'sgh-pagesis 3 days behind itsmain, but that is expected if no docs changed and is not evidence of present staleness. The defect is that the coupling is neither enforced nor observable. Tracked in AAASM-5728.docs/src/is AAASM-5727. Those files are untouched here.aggregate.ymlfailures on 2026-08-12 (20:33, 21:34) are thepeaceiris/actions-hugosetup step, not a ref problem. No action.🤖 Generated with Claude Code
https://claude.ai/code/session_01XWLmA8FgULT9e6ntdCo1H2