-
Notifications
You must be signed in to change notification settings - Fork 0
Closed
Description
Problem
The mirror-binaries script currently reads from the local manifest file to determine what versions to sync to R2. This creates a chicken-and-egg problem where new upstream releases (like Node.js 20.20.0) are never discovered or mirrored.
Current flow:
mirror-syncreads local manifest → syncs those versions to R2generate-manifests-from-r2reads R2 → generates manifests
Since new versions aren't in the manifest, they're never synced, so they're never added to the manifest.
Solution
Refactor mirror-binaries to discover versions directly from upstream sources instead of reading the local manifest:
Node.js
- Source:
https://nodejs.org/dist/index.json - Single comprehensive source covering all platforms
Python
Multiple sources for comprehensive coverage:
- Official:
https://www.python.org/ftp/python/ - Standalone builds:
https://github.com/astral-sh/python-build-standalone/releases
Ruby
Multiple sources for comprehensive coverage:
- Windows:
https://github.com/oneclick/rubyinstaller2/releases - Other platforms:
https://github.com/ruby/ruby-builder/releases
Implementation
- Add upstream source fetching logic for each runtime
- Build job list from upstream version data (not local manifest)
- Compare against existing R2 contents
- Download and upload any missing binaries
generate-manifests-from-r2will then automatically pick up new versions
Acceptance Criteria
-
mirror-binariesdiscovers new versions from upstream sources - Node.js, Python, and Ruby all have their upstream sources implemented
- New releases are automatically mirrored within 24 hours of the daily cron
- Existing R2 content is preserved (no re-downloading)
Metadata
Metadata
Assignees
Labels
No labels