feat(ci): implement mirror-tarball workflow (Phase 2)#1
Merged
Conversation
Replaces the Phase 2 stub with a working tarball-mirror pipeline.
Triggers:
- pull_request.closed on packages/**/manifest.json — mirrors versions
added/modified by the PR where tarball is still null.
- workflow_dispatch — scans every manifest and mirrors any version entry
where tarball is null (backfill mode).
Per pending (name, version) entry:
1. Clone source.repo at sourceTag (shallow, strip .git).
2. Produce deterministic tarball:
tar --sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner
| gzip -n
3. sha256sum the tarball.
4. Create GH Release <name>-<version> on this registry if missing,
upload tarball as asset (--clobber on re-run).
5. jq-patch manifest: set versions[i].tarball = release URL,
versions[i].sha256 = sha.
Post-loop commit + push to main as github-actions[bot] with a
concurrency group (mirror-tarball, cancel-in-progress: false) so
near-simultaneous PR merges serialize cleanly.
Schema tightening:
- Keep manifest.schema.json permissive (tarball/sha256 nullable) so
PR submissions validate.
- Add manifest.strict.schema.json with tarball (URL pattern-matched
against this registry's release-asset URL shape) and sha256 (64-hex)
required + non-null.
- Extend validate.yml with a strict-main job gated on
push + refs/heads/main. After mirror-tarball commits back, this
re-validates and any drift (null field escaping the mirror) fails
CI immediately.
Backfill path for the four seed packages (sentry, hotwire, basecoat,
legacy-adapter) is a single workflow_dispatch run after merge.
Refs wheels-dev/wheels#2269, wheels-dev/wheels#2243 Phase 2.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes wheels-dev/wheels#2269. Phase 2 of wheels-dev/wheels#2243.
Summary
Replaces the Phase 2 stub in
.github/workflows/mirror-tarball.ymlwith a working tarball-mirror pipeline, plus a strict post-merge schema check.What this changes
.github/workflows/mirror-tarball.yml— full implementation.Triggers:
pull_request.closedonpackages/**/manifest.json— mirrors only the versions added/modified by the PR wheretarballis stillnull.workflow_dispatch— scans every manifest and mirrors any pending entry (backfill mode).Per pending
(name, version)entry:source.repoatsourceTag(shallow, strip.git).tar --sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner | gzip -n.sha256sumthe tarball.<name>-<version>on this registry if missing, upload tarball as release asset (--clobberon re-run).jq-patch manifest: setversions[i].tarballto the release-asset URL andversions[i].sha256to the computed hash.Post-loop, commit + push to
mainasgithub-actions[bot]. A concurrency group (mirror-tarball,cancel-in-progress: false) serializes near-simultaneous PR merges so the writeback tomainnever races.schema/manifest.strict.schema.json— new file.Same shape as
manifest.schema.jsonbuttarballandsha256are required and non-nullable. Thetarballpattern is pinned to this registry's release-asset URL shape — author-hosted URLs are rejected at the schema level (Attack A defense)..github/workflows/validate.yml— addsstrict-mainjob.Gated on
pushtorefs/heads/main. Aftermirror-tarball.ymlcommits back,strict-mainre-validates and any null field that escaped the mirror (e.g. mirror job errored) fails CI immediately.Keeping the PR-time schema permissive and enforcing the strict shape post-merge avoids the chicken-and-egg problem where PRs submit with nulls that the mirror is responsible for filling in.
README.md— documents the new distribution flow.Test plan
\${{ }}expressions involving event data go viaenv:).gh workflow run mirror-tarball.yml --repo wheels-dev/wheels-packages) to backfill the four seed packages (sentry, hotwire, basecoat, legacy-adapter).wheels-sentry-1.0.0etc.) with a.tar.gzasset each.mainupdates all four manifests withtarballURL +sha256.strict-mainjob onvalidate.ymlgoes green after the backfill commit.(source.repo, sourceTag).Notes
mainis not currently branch-protected on this repo, sogithub-actions[bot]can push directly. If protection is added later, the workflow will need either a PAT with bypass permission or a bot-opened writeback PR.