retryable_download: preserve partial download on network errors

[liukun]

• Have you followed the guidelines in our Contributing document?
• Have you checked to ensure there aren't other open Pull Requests for the same change?
• Have you added an explanation of what your changes do and why you'd like us to include them?
• Have you written new tests (excluding integration tests) for your changes? (happy to add one — see "Tests" section below)
• Have you successfully run brew lgtm (style, typechecking and tests) with your changes locally?

---

• AI was used to generate or assist with generating this PR.
  The root-cause analysis and patch were drafted with Claude (Opus 4.7) assistance. I then manually read retryable_download.rb, download_strategy.rb, and utils/curl.rb to verify the trace, reproduced the symptom, ran bin/brew style, bin/brew typecheck, and bin/brew tests --only=utils/curl locally before committing.

---

Summary

Fix a long-standing bug where bottle downloads on unstable networks always restart from zero instead of resuming from .incomplete, making large bottles effectively undownloadable over flaky links.

Closes #21518.

Observed symptom

On a slow/flaky connection (e.g. behind a proxy that drops TLS streams mid-transfer), the user sees:

1. brew install <formula> downloads N MB, then the connection drops.
2. brew prints Retrying download in 2s… and starts again from zero — the partial MB is gone.
3. After exhausting retries, it errors out (e.g. Failed to download resource "llvm@21").
4. If the user re-runs brew install …, the first request actually resumes (curl --continue-at - picks up from the current .incomplete size), gets partway further, and drops again.
5. As soon as brew's own retry kicks in inside the same invocation, the cycle repeats from zero.

Net effect: every fresh brew install invocation downloads a slice, then throws the slice away. If no single streaming session can complete the full bottle, the install can never succeed regardless of retries, even though each attempt does make forward progress.

Root cause

RetryableDownload#fetch (Library/Homebrew/retryable_download.rb) wraps the download strategy with exponential-backoff retries. Before every retry it unconditionally calls:

rescue DownloadError, ChecksumMismatchError, Resource::BottleManifest::Error
  ...
  sleep wait
  downloadable.clear_cache    # ← problem
  retry
end

For CurlDownloadStrategy (Library/Homebrew/download_strategy.rb), clear_cache does:

def clear_cache
  super
  rm_rf(temporary_path)    # temporary_path = "<cached_location>.incomplete"
end

So the .incomplete file — which curl_download would otherwise pass to --continue-at - on the next attempt (Library/Homebrew/utils/curl.rb, line ~278) — is deleted before every retry. The try_partial machinery added in #5421 and #11381 is effectively bypassed for any download that hits a DownloadError more than once. The "kill the process and restart" workaround described in #21518 works precisely because SIGINT bypasses this rescue path and leaves the .incomplete file in place.

The three kinds of exceptions caught have different semantics, but clear_cache treats them identically:

Exception	Meaning	.incomplete at catch time	Correct action
DownloadError	connection dropped mid-transfer	valid partial, resumable	keep, let next attempt --continue-at
ChecksumMismatchError	full download finished but hash is wrong	already renamed to cached_location; it's bad	clear
Resource::BottleManifest::Error	manifest mismatch on a finished download	same	clear

Change

Only clear the cache for the "finished but bad" cases. For DownloadError the partial is kept, so the next iteration's curl_download sees destination.exist?, takes the try_partial branch, and passes --continue-at -:

rescue DownloadError, ChecksumMismatchError, Resource::BottleManifest::Error => e
  ...
  sleep wait

  # Preserve the partial `.incomplete` file on network errors so the next
  # attempt can resume via `--continue-at`. Clear the cache only when the
  # fully-downloaded file is known-bad (checksum or manifest mismatch).
  downloadable.clear_cache unless e.is_a?(DownloadError)
  retry
end

Total diff: +5 / −2 on a single file.

Self-healing against a corrupted partial

If a proxy writes garbage bytes into .incomplete (e.g. a captive-portal HTML response), a resumed download will eventually finish with a wrong sha256. That surfaces as ChecksumMismatchError on the next iteration, which does still call clear_cache, wiping the bad partial. The subsequent attempt downloads fresh. So the loop is still self-terminating under adversarial middleware — it just doesn't give up the first 200 MB of good data to an unrelated TCP hiccup.

Tests

bin/brew tests --only=utils/curl — all 65 specs pass.

No dedicated spec exists for RetryableDownload#fetch's exception-handling branches today. Happy to add one that mocks downloadable and asserts:

• raising DownloadError during #fetch does not call clear_cache on retry;
• raising ChecksumMismatchError does call clear_cache on retry.

Let me know if you'd like that folded into this PR or tracked separately.

Prior art / related

Open reports of the same symptom

• Option to save and resume partial download #21518 (Feb 2026, closed without a code change) — user ballo describes the exact "kill the process and re-run and it resumes, but let brew's own retry run and it restarts from zero" behavior this PR fixes.
• brew upgrade failed because slow network, Can brew support breakpoint resume download #14934 (2023, closed) — "Can brew support breakpoint resume download". The advice in the closing comment (HOMEBREW_NO_INSTALL_FROM_API=1) doesn't change how the bottle blob itself is fetched, so it doesn't resolve the underlying loop. The symptom (slow-connection installs that never finish even though each attempt makes progress) matches Option to save and resume partial download #21518 and the one this PR is submitted against.

Merged PRs that built resume support but are silently defeated today

The machinery for resumable downloads already exists and works correctly for the single-attempt case. What's missing is that the outer retry loop in RetryableDownload#fetch deletes the .incomplete file before the next attempt, so none of the following take effect once brew's own retry kicks in:

• Allow incomplete downloads to be resumed even when server rejects HEAD requests #5421 (2018-12, merged, "Allow incomplete downloads to be resumed even when server rejects HEAD requests", utils/curl.rb) — changed the range-support probe from HEAD to GET so servers that reject HEAD don't defeat resume. Working as intended — but only reaches the point of emitting --continue-at - if an .incomplete file is still on disk on the next attempt.
• Fix range requests with curl #11381 (2021-05, merged, "Fix range requests with curl", utils/curl.rb) — reworked how curl_download detects range support (previously it sent a range-request and checked Content-Range; now it does a proper HEAD and looks at Accept-Ranges). Same situation: correct probe, but the partial file it would probe against on retry has already been removed.
• #curl_download: default try_partial to false #13179 (2022-04, merged, "#curl_download: default try_partial to false") — made partial-download mode opt-in to save a HEAD when it isn't useful. CurlDownloadStrategy still opts in (@try_partial = true), so bottle fetches still request resume; this PR is what finally lets that opt-in actually produce savings on the second+ retry.

Recent touches to RetryableDownload that did not look at this path

• retryable_download: retry on bottle manifest errors. #20288 (2025-07, merged, "retryable_download: retry on bottle manifest errors") — widened the set of exception classes that get retried to include bottle-manifest errors. Did not change how the cache is treated between retries.
• Cleanup partial download queue installations on error/interrupt #20966 (2025-11, merged, "Cleanup partial download queue installations on error/interrupt") — ported error/interrupt cleanup for partial installations (extracted Cellar state in bottle_tmp_keg) from formula_installer.rb. Orthogonal to .incomplete file lifetime; doesn't touch clear_cache.

Both PRs touched retryable_download.rb recently without noticing that clear_cache runs unconditionally between retries.

Longstanding feature requests this helps but does not fully resolve

• Suggestion - brew should retry download before abandoning bottle and D/L'ing source #3400 (2017, issue, closed) and Retry bottle download a few times before switching to source build #5219 (2018, issue, closed) — "retry the bottle a few times before falling back to source build". This PR does not change bottle→source fallback policy; it just lets the existing retries make cumulative progress on flaky networks instead of burning them all on restarts from zero. Downloads that genuinely cannot be resumed will fall back exactly as they do today.

Out of scope

The utils/curl.rb side could additionally pass --retry-all-errors (curl ≥ 7.71) inside the retries&.positive? block so curl's own retry can survive connection drops, reducing how often the outer RetryableDownload loop runs at all. I deliberately left that out of this PR to keep the change minimal and unambiguous — happy to open a follow-up if reviewers want it.

[MikeMcQuaid]

Makes sense, thanks!