-
-
Notifications
You must be signed in to change notification settings - Fork 9.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
brew update
can be terribly slow for stale taps
#13224
Labels
bug
Reproducible Homebrew/brew bug
help wanted
We want help addressing this
outdated
PR was locked due to age
Comments
boblail
added a commit
to boblail/brew
that referenced
this issue
May 2, 2022
…update` Why? ==== When humans run `brew update`, the update report is helpful feedback; but when `brew update` is composed into other scripts, the update report is rarely useful. Scripts could always capture or discard its output (we've been capturing it and displaying it only when `brew update` fails), but since generating the update report can be very expensive (Homebrew#13224), this seems like a lot of wasted effort. This solution differs from `--preinstall` in two important ways where `--preinstall`'s behavior is undesirable: 1. `--preinstall` limits but still produces the update report 2. `--preinstall` takes several shortcuts in `brew update`, softening its guarantees
7 tasks
Will review and consider accepting a PR that either:
|
boblail
added a commit
to boblail/brew
that referenced
this issue
May 2, 2022
…update` Why? ==== When humans run `brew update`, the update report is helpful feedback; but when `brew update` is composed into other scripts, the update report is rarely useful. Scripts could always capture or discard its output (we've been capturing it and displaying it only when `brew update` fails), but since generating the update report can be very expensive (Homebrew#13224), this seems like a lot of wasted effort. This solution differs from `--preinstall` in two important ways where `--preinstall`'s behavior is undesirable: 1. `--preinstall` limits but still produces the update report 2. `--preinstall` takes several shortcuts in `brew update`, softening its guarantees
boblail
added a commit
to boblail/brew
that referenced
this issue
May 4, 2022
`brew update` fast-forwards each tap from one commit to another. `update-report` lists all the formulae that were modified in that commit range. This clause was filtering out modified formulae _whose `pkg_version` had not changed._ The outcome of this filtering: - If you rewind homebrew/core **2,000 commits** (or 2 weeks), **819** formulae were modified. This clause would have filtered out **153** (18.7%) of them whose `pkg_version` had not changed. It would've taken **29.0 seconds**. - If you rewind homebrew/core **20,000 commits** (or 5 months), 3532 formulae were modified. This clause would have filtered out 1121 (31.7%) of them whose `pkg_version` had not changed. It would've taken **119.4 seconds**. | Branch | Command | Commits | Time | Modified | | --- | --- | --- | --- | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k.txt"` | 2,000 | 35.6s | 666 | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k.txt"` | 20,000 | 129.8s | 2411 | | `lail/xyz` | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k.txt"` | 2,000 | 6.6s | 819 | | `lail/xyz` | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k.txt"` | 20,000 | 10.4s | 3532 | This operation was so expensive because it shelled out to `git cat-file` for each formula that had been touched in the commit window. Fixes Homebrew#13224
boblail
added a commit
to boblail/brew
that referenced
this issue
May 4, 2022
`brew update` fast-forwards each tap from one commit to another. `update-report` lists all the formulae that were modified in that commit range. The clause this commit removes was filtering out modified formulae _whose `pkg_version` had not changed._ The outcome of this filtering: - If you rewind homebrew/core **2,000 commits** (or 2 weeks), **819** formulae were modified. This clause would have taken **29.0 seconds** to filter out **153** (18.7%) of them. - If you rewind homebrew/core **20,000 commits** (or 5 months), 3532 formulae were modified. This clause would have taken **119.4 seconds** to filter out **1121** (31.7%) of them. Data ========= | Homebrew Branch | Command | Commits Rewound | Time Elapsed | Updated Formulae | | --- | --- | --- | --- | --- | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k.txt"` | 2,000 | 35.6s | 666 | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k.txt"` | 20,000 | 129.8s | 2411 | | _this one_ | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k.txt"` | 2,000 | 6.6s | 819 | | _this one_ | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k.txt"` | 20,000 | 10.4s | 3532 | Root Cause ========= This operation was so expensive because it shelled out to `git cat-file` for each formula that had been touched in the commit window. Fixes Homebrew#13224
7 tasks
boblail
added a commit
to boblail/brew
that referenced
this issue
May 4, 2022
| Homebrew Branch | Command | Commits Rewound | Time Elapsed | Updated Formulae | | --- | --- | --- | --- | --- | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k.txt"` | 2,000 | 35.6s | 666 | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k.txt"` | 20,000 | 129.8s | 2411 | | Homebrew#13234 | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k-unfiltered.txt"` | 2,000 | 6.6s | 819 | | Homebrew#13234 | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k-unfiltered.txt"` | 20,000 | 10.4s | 3532 | | _this one_ | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k-optimized.txt"` | 2,000 | 7.1s | 657† | | _this one_ | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k-optimized.txt"` | 20,000 | 17.6s | 2411 | † I haven't determined why there's a discrepancy between "Updated Formulae" on row 5 and row 1. Fixes Homebrew#13224
7 tasks
boblail
added a commit
to boblail/brew
that referenced
this issue
May 4, 2022
| Homebrew Branch | Command | Commits Rewound | Time Elapsed | Updated Formulae | | --- | --- | --- | --- | --- | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k.txt"` | 2,000 | 35.6s | 666 | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k.txt"` | 20,000 | 129.8s | 2411 | | Homebrew#13234 | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k-unfiltered.txt"` | 2,000 | 6.6s | 819 | | Homebrew#13234 | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k-unfiltered.txt"` | 20,000 | 10.4s | 3532 | | _this one_ | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k-optimized.txt"` | 2,000 | 7.1s | 657† | | _this one_ | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k-optimized.txt"` | 20,000 | 17.6s | 2411 | † I haven't determined why there's a discrepancy between "Updated Formulae" on row 5 and row 1. Fixes Homebrew#13224
boblail
added a commit
to boblail/brew
that referenced
this issue
May 4, 2022
| Homebrew Branch | Command | Commits Rewound | Time Elapsed | Updated Formulae | | --- | --- | --- | --- | --- | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k.txt"` | 2,000 | 35.6s | 666 | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k.txt"` | 20,000 | 129.8s | 2411 | | Homebrew#13234 | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k-unfiltered.txt"` | 2,000 | 6.6s | 819 | | Homebrew#13234 | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k-unfiltered.txt"` | 20,000 | 10.4s | 3532 | | _this one_ | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k-optimized.txt"` | 2,000 | 7.1s | 657† | | _this one_ | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k-optimized.txt"` | 20,000 | 17.6s | 2411 | † I haven't determined why there's a discrepancy between "Updated Formulae" on row 5 and row 1. Fixes Homebrew#13224
boblail
added a commit
to boblail/brew
that referenced
this issue
May 4, 2022
| Homebrew Branch | Command | Commits Rewound | Time Elapsed | Updated Formulae | | --- | --- | --- | --- | --- | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k.txt"` | 2,000 | 35.6s | 666 | | `master` | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k.txt"` | 20,000 | 129.8s | 2411 | | Homebrew#13234 | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k-unfiltered.txt"` | 2,000 | 6.6s | 819 | | Homebrew#13234 | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k-unfiltered.txt"` | 20,000 | 10.4s | 3532 | | _this one_ | `git -C "$(brew --repo homebrew/core)" reset --hard aa1b3d5; time sh -c "brew update &> output-2k-optimized.txt"` | 2,000 | 7.1s | 657† | | _this one_ | `git -C "$(brew --repo homebrew/core)" reset --hard 11e6919; time sh -c "brew update &> output-20k-optimized.txt"` | 20,000 | 17.6s | 2411 | † I haven't determined why there's a discrepancy between "Updated Formulae" on row 5 and row 1. Fixes Homebrew#13224
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Labels
bug
Reproducible Homebrew/brew bug
help wanted
We want help addressing this
outdated
PR was locked due to age
brew config
outputbrew doctor
outputVerification
brew update
and am still able to reproduce my issue.brew doctor
and that did not fix my problem.What were you trying to do (and why)?
If you run
brew update
on a very stale tap, theupdate-report
takes a long time.For example,
takes 2m40s on my MacBook Pro (2.3 GHz 8-Core Intel Core i9)
If I comment out just the call to
brew update-report
, the same operation takes 3.8s — so the expense is entirely inbrew update-report
.Alternately, if I comment out just lines 373-383 in
update-report.rb
, the same operation takes 10.3s.brew update --preinstall
also bypasses these lines; but it cuts corners inupdate.sh
as well.What happened (include all command output)?
(Described above)
What did you expect to happen?
I would expect
brew update
not to take minutes, even for stale tapsStep-by-step reproduction instructions (by running
brew
commands)The text was updated successfully, but these errors were encountered: