Skip to content

Keep one failure from aborting a batch install - #23525

Merged
MikeMcQuaid merged 1 commit into
mainfrom
isolate-batch-install-failures
Aug 14, 2026
Merged

Keep one failure from aborting a batch install#23525
MikeMcQuaid merged 1 commit into
mainfrom
isolate-batch-install-failures

Conversation

@MikeMcQuaid

Copy link
Copy Markdown
Member
  • A bottle that fails to extract aborted the whole brew upgrade (or multi-formula brew install) run, so every remaining formula and cask was skipped: report the failure and carry on with the rest instead, still exiting nonzero.
  • Bottle#downloaded_and_valid? trusts an immutable GitHub Packages blob without rehashing it, so a corrupt cached bottle failed to extract on every run. Verify it after a failed extraction, then discard it and retry the extraction with a fresh download.
  • brew install and brew reinstall exited as soon as anything had failed, before installing anything at all: install what did download instead, skipping only the packages whose downloads failed.
  • Isolate each cask in brew install and each formula in brew reinstall, and widen the errors tolerated per package, so one failure no longer skips everything after it.
  • Track which downloads failed in DownloadQueue so a failed bottle only distrusts the shared formula prefetch, rather than also making every already downloaded cask be fetched and verified again.

Fixes #23519.


  • Have you followed our Contributing guidelines?
  • Have you checked for other open Pull Requests for the same change?
  • Have you explained what your changes do? Performance claims (e.g. "this is faster") must include Hyperfine benchmarks.
  • Have you explained why you'd like these changes included, not just what they do?
  • For bug fixes, have you given step-by-step brew commands to reproduce the bug?
  • Have you written new tests (excluding integration tests)? Here's an example.
  • Have you successfully run brew lgtm (style, typechecking and tests) locally?

  • I did not use AI/LLM to create this PR, or I disclosed the tool/model below and reviewed its output; I did not attribute commits to AI and will answer maintainer questions and review comments myself without AI/LLM.

Claude Opus 5 high with local review and testing.


Copilot AI lite review requested due to automatic review settings August 14, 2026 15:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves Homebrew’s batch install/upgrade/reinstall resilience by ensuring that individual package failures (e.g., bottle extraction or download issues) are reported but do not abort the remainder of the run, while still exiting non-zero when any failures occurred.

Changes:

  • Continue batch brew upgrade, multi-formula brew install, and brew reinstall after per-formula/cask failures instead of aborting the entire run.
  • Add retry behavior for bottles that fail to extract due to a corrupt cached download (rehash after failure, discard if mismatched, redownload once).
  • Replace DownloadQueue#fetch_failed with failed_downloads tracking to scope “distrust prefetch” behavior to only the package type that actually failed.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
Library/Homebrew/upgrade.rb Ensures a single formula upgrade failure doesn’t abort subsequent upgrades.
Library/Homebrew/install.rb Skips installing only formulae whose downloads failed; continues other installs; tolerates per-formula install failures.
Library/Homebrew/bottle.rb Adds post-extraction-failure checksum verification and a single redownload retry for corrupt cached bottles.
Library/Homebrew/formula_installer.rb Uses Bottle#stage (with corrupt-cache retry) when pouring bottles outside the download-queue extraction path.
Library/Homebrew/download_queue.rb Tracks failed_downloads instead of a single boolean, enabling finer-grained retry/skip decisions.
Library/Homebrew/cmd/upgrade.rb Re-does unprefetched fetch only for the package type that actually failed in shared prefetch.
Library/Homebrew/cmd/reinstall.rb Continues reinstalling remaining formulae after failures; skips only those whose downloads failed.
Library/Homebrew/cmd/install.rb Continues installing remaining formulae/casks after individual failures; skips only formulae whose downloads failed.
Library/Homebrew/cask/reinstall.rb Avoids aborting cask reinstall due to earlier failures in the same run.
Library/Homebrew/test/upgrade_spec.rb Adds coverage that upgrade reports failure without aborting the batch.
Library/Homebrew/test/install_spec.rb Adds coverage for skipping only failed-download formulae and continuing installs after an install failure.
Library/Homebrew/test/formula_installer_bottle_spec.rb Adds coverage for corrupt cached bottle handling during install.
Library/Homebrew/test/download_queue_spec.rb Updates expectations to validate failed_downloads behavior.
Library/Homebrew/test/cmd/upgrade_spec.rb Updates queue doubles and adds coverage for scoped distrust of shared prefetch failures.
Library/Homebrew/test/cmd/reinstall_spec.rb Adds coverage for continuing reinstall after one formula fails.
Library/Homebrew/test/cmd/install_spec.rb Adds coverage ensuring installs proceed after earlier failures and updates queue doubles.
Library/Homebrew/test/cask/reinstall_spec.rb Adds coverage that cask reinstall continues after earlier failures in the run.
Library/Homebrew/test/bottle_spec.rb Adds unit coverage for corrupt cached bottle redownload retry vs. checksum-matching failures.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread Library/Homebrew/cmd/install.rb
- A bottle that fails to extract aborted the whole `brew upgrade` (or
  multi-formula `brew install`) run, so every remaining formula and cask
  was skipped: report the failure and carry on with the rest instead,
  still exiting nonzero.
- `Bottle#downloaded_and_valid?` trusts an immutable GitHub Packages
  blob without rehashing it, so a corrupt cached bottle failed to
  extract on every run. Verify it after a failed extraction, then
  discard it and retry the extraction with a fresh download.
- `brew install` and `brew reinstall` exited as soon as anything had
  failed, before installing anything at all: install what did download
  instead, skipping only the packages whose downloads failed.
- Isolate each cask in `brew install` and each formula in
  `brew reinstall`, and widen the errors tolerated per package, so one
  failure no longer skips everything after it.
- Report each cask failure as it happens with `ofail`, naming the cask,
  rather than collecting them into a `MultipleCaskErrors` raised once
  the batch is over: cask failures now read the same as formula ones.
- Track which downloads failed in `DownloadQueue` so a failed bottle
  only distrusts the shared formula prefetch, rather than also making
  every already downloaded cask be fetched and verified again.

Fixes #23519.
@MikeMcQuaid
MikeMcQuaid force-pushed the isolate-batch-install-failures branch from be5f44a to bd463ee Compare August 14, 2026 16:21

@p-linnane p-linnane left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@MikeMcQuaid
MikeMcQuaid enabled auto-merge August 14, 2026 16:30
@MikeMcQuaid
MikeMcQuaid added this pull request to the merge queue Aug 14, 2026
Merged via the queue into main with commit 6d224db Aug 14, 2026
68 of 69 checks passed
@MikeMcQuaid
MikeMcQuaid deleted the isolate-batch-install-failures branch August 14, 2026 17:16
@joshka

joshka commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Thanks Mike

@MikeMcQuaid

Copy link
Copy Markdown
Member Author

@joshka you're welcome, thanks for the good bug report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failing unpack fails batch upgrade

4 participants