Skip to content

Conversation

@CalvinAllen
Copy link
Contributor

Summary

  • Refactor mirror-binaries to fetch version lists directly from upstream sources instead of reading local manifests
  • Fixes the chicken-and-egg problem where new releases (like Node.js 20.20.0) were never discovered or mirrored

Upstream Sources

Runtime Source Coverage
Node.js nodejs.org/dist/index.json Comprehensive (all platforms)
Python astral-sh/python-build-standalone Linux, macOS, Windows
Ruby oneclick/rubyinstaller2 Windows
Ruby ruby/ruby-builder Linux, macOS

Changes

  • Add sources.go with UpstreamSource interface and HTTP retry logic
  • Add node.go for Node.js official source with SHASUMS256 parsing
  • Add python.go for Python standalone builds
  • Add ruby.go for RubyInstaller and ruby-builder
  • Update main.go to use upstream discovery instead of local manifest
  • Remove --manifest-dir flag from workflow (no longer needed)

Test Plan

  • Dry run test with Node.js - discovered 3026 versions including 20.20.0
  • Dry run test with Ruby - discovered 497 versions from both sources
  • Full sync run on CI to verify R2 upload works
  • Verify generate-manifests-from-r2 picks up new versions

Closes #208

@CalvinAllen CalvinAllen force-pushed the fix/mirror/upstream-discovery branch from af7beff to 543af71 Compare January 25, 2026 23:00
Refactor mirror-binaries to fetch version lists directly from upstream
sources instead of reading from local manifests. This fixes the
chicken-and-egg problem where new releases were never discovered.

Upstream sources:
- Node.js: nodejs.org/dist/index.json (comprehensive)
- Python: python-build-standalone (GitHub releases)
- Ruby: rubyinstaller2 (Windows) + ruby-builder (Linux/macOS)

Changes:
- Add sources.go with UpstreamSource interface and retry logic
- Add node.go for Node.js official source
- Add python.go for Python standalone builds
- Add ruby.go for RubyInstaller and ruby-builder
- Update main.go to use upstream discovery
- Remove --manifest-dir flag from workflow (no longer needed)

Closes #208
@CalvinAllen CalvinAllen force-pushed the fix/mirror/upstream-discovery branch from 543af71 to fd51310 Compare January 25, 2026 23:03
@CalvinAllen CalvinAllen merged commit bd61a47 into main Jan 25, 2026
12 checks passed
@CalvinAllen CalvinAllen deleted the fix/mirror/upstream-discovery branch January 25, 2026 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

fix(mirror): discover new versions from upstream sources

2 participants