Skip to content

feat(ampup): add token resolution, rate limiter, and --jobs CLI flag#4

Draft
mitchhs12 wants to merge 2 commits intomainfrom
mitchhs12/ampup-rate-limiter-token-resolution
Draft

feat(ampup): add token resolution, rate limiter, and --jobs CLI flag#4
mitchhs12 wants to merge 2 commits intomainfrom
mitchhs12/ampup-rate-limiter-token-resolution

Conversation

@mitchhs12
Copy link

@mitchhs12 mitchhs12 commented Feb 16, 2026

Part 1 of edgeandnode/amp#1762 (bounded-concurrency parallel downloads)

Summary

  • Add GitHub token resolution with fallback chain: explicit token → gh auth token → unauthenticated
  • Add --jobs / -j CLI flag to install and update commands for controlling concurrent downloads (default: 4)
  • Add GitHubRateLimiter that parses X-RateLimit-Remaining, X-RateLimit-Reset, and Retry-After headers per GitHub REST API docs
  • On 429 or rate-limit 403, sets a global pause shared across all concurrent downloads and retries once
  • Preemptively pauses when remaining hits 0 (avoids burning requests that will fail)
  • Integrates rate limiter into all GitHubClient HTTP methods via send_with_rate_limit
  • Make GitHubClient cloneable for future DownloadManager task spawning

Changes

Token resolution (token.rs — new)

  • resolve_github_token(): explicit → gh auth tokenNone
  • Security note: --github-token values visible in ps aux, prefer env var or gh

Rate limiter (rate_limiter.rs — new)

  • GitHubRateLimiter with tokio::sync::Mutex-protected state
  • wait_if_paused(): blocking gate, drops lock before sleeping
  • update_from_response()update_state(): testable state machine
  • 429 and 403 (with rate-limit signal) trigger global backoff
  • Retry-After default: 60s per GitHub docs when header absent
  • extend_pause(): only extends pauses, never shortens

GitHub client (github.rs — modified)

  • send_with_rate_limit(): wraps all HTTP requests with rate-limit awareness and one retry on 429
  • RateLimited error variant with actionable user message
  • #[derive(Clone)] on GitHubClient (cheap — reqwest::Client and Arc<GitHubRateLimiter> are both Arc-backed)

CLI (main.rs — modified)

  • --jobs / -j flag on install and update commands

Test plan

  • 10 rate limiter unit tests (blocking gate, state machine edge cases, real HTTP header parsing via mock TCP server)
  • 2 token resolution unit tests
  • All 23 ampup tests pass
  • Zero clippy warnings

Add GitHub token resolution with fallback chain: explicit token →
`gh auth token` → unauthenticated. Add `--jobs` / `-j` CLI flag to
`install` and `update` commands for controlling concurrent downloads
(default: 4).
Add GitHubRateLimiter that parses X-RateLimit-Remaining, X-RateLimit-Reset,
and Retry-After headers. On 429 or rate-limit 403, sets a global pause shared
across all concurrent downloads and retries once. Preemptively pauses when
remaining hits 0. Integrates into GitHubClient via send_with_rate_limit, which
wraps all HTTP methods with rate-limit awareness.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant