Skip to content

perf(delta-upgrade): lazy chain walk, GHCR retry, parallel I/O, offline cache#360

Merged
BYK merged 1 commit intomainfrom
perf/delta-upgrade-lazy-walk
Mar 6, 2026
Merged

perf(delta-upgrade): lazy chain walk, GHCR retry, parallel I/O, offline cache#360
BYK merged 1 commit intomainfrom
perf/delta-upgrade-lazy-walk

Conversation

@BYK
Copy link
Member

@BYK BYK commented Mar 5, 2026

Summary

Four performance optimizations for the delta upgrade system, addressing the 5-30s latency identified via telemetry in PR #355. Applies to both nightly and stable channels.

Before

  • buildNightlyPatchGraph fetches ALL ~14 patch manifests from GHCR in parallel (N HTTP calls)
  • GHCR latency variance: 126ms to 29,761ms for identical requests (cold start spikes)
  • fetchManifest(targetTag) and listTags() run sequentially despite no data dependency
  • No timeout or retry on GHCR requests
  • Every upgrade requires network access — no caching at all

After

  1. Lazy chain resolution — Filter tags by version range, fetch only the 1-2 manifests in the chain path instead of all N
  2. GHCR retry with timeout — 10s timeout + 1 retry on transient errors; caps worst-case at ~20s instead of 30s+
  3. Parallel I/OfetchManifest(targetTag) and listTags() run concurrently via Promise.all after token acquisition
  4. Patch pre-fetch for offline upgrades — Background version check downloads delta patches (~50-80KB) to ~/.sentry/patch-cache/. Subsequent sentry cli upgrade applies patches offline with zero network calls. Patches accumulate across version check runs for multi-hop offline upgrades.

Expected impact

Scenario Before After
Typical single-step upgrade 4s-54s (network-bound) ~2.5s (offline, cache hit)
Manifest fetches per upgrade ~14 HTTP calls 0 (cached) or 1-2 (miss)
build-patch-graph phase 1.5s-22s eliminated
fetch-target-manifest + list-tags sequential parallel
Cold-start spike up to 30s capped at ~20s (10s timeout + retry)

Changes

Core optimizations:

  • src/lib/delta-upgrade.ts — Replace buildNightlyPatchGraph + walkNightlyChain with filterAndSortChainTags (pure function). Add preloadedTags for parallelization. Cache-first path in resolveStableDelta/resolveNightlyDelta. New prefetchNightlyPatches and prefetchStablePatches with abort signal support.
  • src/lib/ghcr.tsfetchWithRetry helper with 10s timeout + 1 retry for transient errors.

New module:

  • src/lib/patch-cache.ts — File-based cache: savePatchesToCache, loadCachedChain (stitches patches from multiple runs), cleanupPatchCache (7-day TTL). Version-based naming is channel-agnostic.

Integration:

  • src/lib/version-check.ts — After storing latest version, calls maybePrefetchPatches() to download patches in background. Opportunistic cache cleanup.

Telemetry:

  • delta.source span attribute: "cache" vs "network"
  • chain.tags_total, chain.tags_filtered, chain.fetched_count

Tests:

  • test/lib/patch-cache.test.ts — 23 tests: save/load/cleanup with edge cases (cross-run stitching, corrupt metadata, missing files, nightly versions)
  • Updated test/lib/delta-upgrade.test.ts and test/isolated/delta-upgrade.test.ts

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

Semver Impact of This PR

🟢 Patch (bug fixes)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).


Internal Changes 🔧

  • (delta-upgrade) Lazy chain walk, GHCR retry, parallel I/O, offline cache by BYK in #360

🤖 This preview updates automatically when you update the PR.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

Codecov Results 📊

101 passed | Total: 101 | Pass Rate: 100% | Execution Time: 0ms

📊 Comparison with Base Branch

Metric Change
Total Tests
Passed Tests
Failed Tests
Skipped Tests

✨ No test changes detected

All tests are passing successfully.

✅ Patch coverage is 82.68%. Project has 3649 uncovered lines.
❌ Project coverage is 81.51%. Comparing base (base) to head (head).

Files with missing lines (5)
File Patch % Lines
upgrade.ts 81.19% ⚠️ 85 Missing
delta-upgrade.ts 87.83% ⚠️ 84 Missing
patch-cache.ts 87.64% ⚠️ 33 Missing
version-check.ts 76.00% ⚠️ 30 Missing
ghcr.ts 91.67% ⚠️ 23 Missing
Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
- Coverage    81.61%    81.51%     -0.1%
==========================================
  Files          127       128        +1
  Lines        19177     19736      +559
  Branches         0         0         —
==========================================
+ Hits         15651     16087      +436
- Misses        3526      3649      +123
- Partials         0         0         —

Generated by Codecov Action

@BYK BYK force-pushed the perf/delta-upgrade-lazy-walk branch from 85309f1 to fa6085f Compare March 5, 2026 20:42
@BYK BYK changed the title perf(delta-upgrade): lazy chain walk, manifest cache, GHCR retry with timeout perf(delta-upgrade): lazy chain walk, GHCR retry, parallel I/O Mar 5, 2026
@BYK BYK force-pushed the perf/delta-upgrade-lazy-walk branch from fa6085f to 29eee29 Compare March 5, 2026 21:02
@BYK BYK changed the title perf(delta-upgrade): lazy chain walk, GHCR retry, parallel I/O perf(delta-upgrade): lazy chain walk, GHCR retry, parallel I/O, offline cache Mar 5, 2026
@BYK BYK force-pushed the perf/delta-upgrade-lazy-walk branch from 29ea02c to e845ba5 Compare March 5, 2026 22:06
@BYK BYK force-pushed the perf/delta-upgrade-lazy-walk branch 2 times, most recently from 60cf42f to af8de8c Compare March 6, 2026 09:53
@BYK BYK force-pushed the perf/delta-upgrade-lazy-walk branch from af8de8c to 282aceb Compare March 6, 2026 10:31
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

…ne cache

Four performance optimizations for the delta upgrade system:

## Lazy chain resolution (replaces eager graph build)

Replace buildNightlyPatchGraph + walkNightlyChain with a lazy approach
in resolveNightlyChain:

- List all patch tags (1 HTTP call, kept)
- Filter tags to only those between currentVersion and targetVersion
  using Bun.semver.order() — since patches are sequential, this gives
  the exact chain without fetching any manifests
- Sort filtered tags by version
- Fetch only the 1-2 manifests in the chain (instead of all N)

This reduces manifest fetches from ~14 to 1-2 for typical upgrades.

## Retry with timeout for GHCR requests

Add fetchWithRetry helper in ghcr.ts:
- 10s timeout per request (AbortSignal.timeout)
- 1 retry on transient errors (timeout, network, connection reset)
- Applied to getAnonymousToken, fetchManifest, fetchTagPage
- downloadNightlyBlob gets 30s timeout (no retry — large downloads)

## Parallel I/O

fetchManifest(targetTag) and listTags() run concurrently via Promise.all
after token acquisition — both only depend on the token, not each other.

## Patch pre-fetch for offline upgrades

New file-based cache in ~/.sentry/patch-cache/ enables fully offline
delta upgrades for both nightly and stable channels:

- Background version check now pre-fetches delta patches (~50-80KB)
  after discovering a new version
- Patches accumulate across runs: skip 3 versions, all 3 patches are
  cached, multi-hop upgrade is fully offline
- resolveStableDelta/resolveNightlyDelta check cache before network
- delta.source span attribute tracks cache vs network for telemetry
- 7-day TTL with opportunistic cleanup during version checks
- Two-pass cleanup preserves shared patches referenced by live chains
@BYK BYK force-pushed the perf/delta-upgrade-lazy-walk branch from 282aceb to 582d093 Compare March 6, 2026 10:58
@BYK BYK merged commit a909191 into main Mar 6, 2026
20 checks passed
@BYK BYK deleted the perf/delta-upgrade-lazy-walk branch March 6, 2026 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant