New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update.sh: update to latest Homebrew/brew tag. #998

Merged
merged 2 commits into from Sep 21, 2016

Conversation

Projects
None yet
9 participants
@MikeMcQuaid
Copy link
Member

MikeMcQuaid commented Sep 17, 2016

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew tests with your changes locally?

Rather than following every change on master let’s have non-developer users (i.e. those who have never run a dev-cmd or set HOMEBREW_DEVELOPER) update between tags.

This provides a fairly natural beta (the master branch) and stable (the tags) approach without restricting us to any particular way of managing our tags.

Also, ensure we always fetch tags and allow forcing tag update functionality by setting
HOMEBREW_UPDATE_TO_TAG.

This shouldn't be merged until after we tag 1.0.0 as 0.9.9 is too old to have users on for now.

@MikeMcQuaid MikeMcQuaid added this to the 1.0.0 milestone Sep 17, 2016

@chdiza

This comment has been minimized.

Copy link
Contributor

chdiza commented Sep 18, 2016

I'm not sure I understand what this involves. If this means that the "non-developer" who does brew update won't receive an update unless a new tag has been made since their last update, then I don't think this is a good idea.

Reason: Suppose a user files an issue and some fix is made in the brew repo. But no new tag is created. Then we can't just tell users "do brew update and try again"; we'd have to say "set HOMEBREW_DEVELOPER and then do brew update and try again" (or "run a dev-cmd and then brew update and then try again). Besides being windy, that's just asking for user confusion. If to avoid this new tags are created frequently, this also seems like not a good idea. (Just wait until some tagged commit has to be reverted, etc.)

Apologies if I've misunderstood what this PR does.

@zmwangx

This comment has been minimized.

Copy link
Contributor

zmwangx commented Sep 18, 2016

If this means that the "non-developer" who does brew update won't receive an update unless a new tag has been made since their last update, then I don't think this is a good idea.

We've had a couple quite widespread breakages recently (including the linked kegs migration debacle today of which we're still dealing with the fallout) we definitely need to be more careful at upgrading all users to master.

Adding a homebrew_developer? check to every feature then remove it after a while doesn't scale.

We didn't say how often tags will be made by the way.

we'd have to say "set HOMEBREW_DEVELOPER and then do brew update and try again"

$ HOMEBREW_DEVELOPER=1 brew update

What's so difficult about that.

that's just asking for user confusion.

Slight user confusion when communicating to a single user or a few users is better than "no-commands-run" for thousands of at least many users (which happened today, or yesterday depending on where you are).

(Just wait until some tagged commit has to be reverted, etc.)

Reverting a tagged commit has no effect on that tag. We don't force push here. I don't see how it's worse than reverting a commit on the master branch when users are updated to the master branch.

@MikeMcQuaid MikeMcQuaid force-pushed the MikeMcQuaid:update-between-tags branch from 5bbeb4f to cbdaa24 Sep 18, 2016

@codecov-io

This comment has been minimized.

Copy link

codecov-io commented Sep 18, 2016

Current coverage is 83.25% (diff: 100%)

Merging #998 into master will increase coverage by 7.99%

@@             master       #998   diff @@
==========================================
  Files           360        340     -20   
  Lines         15543      14051   -1492   
  Methods           0          0           
  Messages          0          0           
  Branches          0          0           
==========================================
+ Hits          11697      11698      +1   
+ Misses         3846       2353   -1493   
  Partials          0          0           

Powered by Codecov. Last update f4a8d28...d1c3793

@MikeMcQuaid MikeMcQuaid referenced this pull request Sep 18, 2016

Merged

update.sh: turn origin/branch into a variable. #1007

5 of 5 tasks complete
@MikeMcQuaid

This comment has been minimized.

Copy link
Member

MikeMcQuaid commented Sep 18, 2016

Then we can't just tell users "do brew update and try again"; we'd have to say "set HOMEBREW_DEVELOPER and then do brew update and try again" (or "run a dev-cmd and then brew update and then try again). Besides being windy, that's just asking for user confusion.

Really this is a question/debate about rolling release vs. tagged release software development approaches. I agree the tag approach would not make sense for taps (for now, at least) but except for when we've broken something with another change it's extremely rare that there are user fixes that we need to rapidly make changes to fix. In those cases, we can push them out more quickly to HOMEBREW_DEVELOPERs and, if things look good after a few days, make a tag.

@MikeMcQuaid MikeMcQuaid force-pushed the MikeMcQuaid:update-between-tags branch from cbdaa24 to a7e74f4 Sep 18, 2016

@chdiza

This comment has been minimized.

Copy link
Contributor

chdiza commented Sep 18, 2016

I like the idea of "tagged release" for brew internals. Very much so. I just was doubtful about the implementation of putting it behind HOMEBREW_DEVELOPER.

Instead of having to tell users to do HOMEBREW_DEVELOPER=1 brew update, there should perhaps be a new flag for update. I don't know what the name would be, but something like brew update --rolling or something.

What's so difficult about that.

It's longer and it looks weird to the sort of newbies who are prone to getting confused.

Slight user confusion when communicating to a single user or a few users is better than "no-commands-run" for thousands of at least many users (which happened today, or yesterday depending on where you are).

There aren't just these two possibilities.

@MikeMcQuaid

This comment has been minimized.

Copy link
Member

MikeMcQuaid commented Sep 18, 2016

@chdiza It's behind HOMEBREW_DEVELOPER or if you've ever run a dev-cmd. We may iterate on the approach but this will be the one we start with.

@MikeMcQuaid MikeMcQuaid force-pushed the MikeMcQuaid:update-between-tags branch from a7e74f4 to d1c3793 Sep 18, 2016

@MikeMcQuaid

This comment has been minimized.

Copy link
Member

MikeMcQuaid commented Sep 20, 2016

@jawshooah @reitermarkus A heads up that this will be happening in the next few days as it'll affect you too.

REMOTE_REF="origin/$UPSTREAM_BRANCH"
if [[ "$DIR" = "$HOMEBREW_REPOSITORY" && -z "$HOMEBREW_NO_UPDATE_CLEANUP" ]]
then
UPSTREAM_TAG="$(git tag --list --sort=-version:refname | head -n1)"

This comment has been minimized.

@zmwangx

zmwangx Sep 20, 2016

Contributor

On a regular install tags aren't fetched, so UPSTREAM_TAG would be empty here.

This comment has been minimized.

@MikeMcQuaid

MikeMcQuaid Sep 20, 2016

Member

Handled by #1046

This comment has been minimized.

@MikeMcQuaid

MikeMcQuaid Sep 21, 2016

Member

Further tweaked the approach to handle if this is empty.

@zmwangx

This comment has been minimized.

Copy link
Contributor

zmwangx commented Sep 20, 2016

As it currently stands this branch seems problematic due to lack of tags locally. Reproducer on clean VM:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install hub
cd /usr/local/Homebrew
hub checkout https://github.com/Homebrew/brew/pull/998
brew update

and that last brew update produces

Checking out v in /usr/local/Homebrew...
To checkout MikeMcQuaid-update-between-tags in /usr/local/Homebrew run:
  'cd /usr/local/Homebrew && git checkout MikeMcQuaid-update-between-tags
fatal: Cannot update paths and switch to branch 'v' at the same time.
Did you intend to checkout 'refs/tags/' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/
Already up-to-date.

It's trying to check out v because UPSTREAM_TAG is empty.

@MikeMcQuaid

This comment has been minimized.

Copy link
Member

MikeMcQuaid commented Sep 20, 2016

@zmwangx Well noticed. We'll need to do a forced-git fetch if there's no tags.

@MikeMcQuaid MikeMcQuaid referenced this pull request Sep 20, 2016

Merged

Update: ensure tags are fetched #1046

5 of 5 tasks complete

@MikeMcQuaid MikeMcQuaid force-pushed the MikeMcQuaid:update-between-tags branch 2 times, most recently from 5096eef to af55af2 Sep 20, 2016

MikeMcQuaid added some commits Sep 17, 2016

update.sh: update to latest Homebrew/brew tag.
Rather than following every change on `master` let’s have non-developer
users (i.e. those who have never run a `dev-cmd` or set
`HOMEBREW_DEVELOPER`) update between tags.

This provides a fairly natural beta (the `master` branch`) and stable
(the tags) approach without restricting us to any particular way of
managing our tags.
update.sh: allow forcing tag update.
To test the tag update functionality allow setting
`HOMEBREW_UPDATE_TO_TAG`.

@MikeMcQuaid MikeMcQuaid force-pushed the MikeMcQuaid:update-between-tags branch from af55af2 to e8ebcb8 Sep 21, 2016

@MikeMcQuaid MikeMcQuaid merged commit d5b6ecf into Homebrew:master Sep 21, 2016

1 of 2 checks passed

default Build started sha1 is merged.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@MikeMcQuaid MikeMcQuaid deleted the MikeMcQuaid:update-between-tags branch Sep 21, 2016

@mgol

This comment has been minimized.

Copy link
Contributor

mgol commented Sep 21, 2016

This broke my setup. I had updated to d5b6ecf about 5-6 hours ago via brew update && brew upgrade. I installed the s3cmd package 30 minutes ago with the following output:

$ brew install s3cmd
Checking out v1.0.0 in /usr/local/Homebrew...
To checkout master in /usr/local/Homebrew run:
  'cd /usr/local/Homebrew && git checkout master
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
To checkout master in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core run:
  'cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
==> Downloading https://homebrew.bintray.com/bottles/s3cmd-1.6.1.el_capitan.bottle.tar.gz
######################################################################## 100.0%
==> Pouring s3cmd-1.6.1.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/s3cmd/1.6.1: 59 files, 784.4K

These messages worried me so I invoked brew update:

$  brew update
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask...
To checkout master in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask run:
  'cd /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
To checkout master in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core run:
  'cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Already up-to-date.
+ brew upgrade --all
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask...
To checkout master in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask run:
  'cd /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
To checkout master in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core run:
  'cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0

This happened every time so I went into /usr/local/Homebrew and invoked git prune & git gc & git fsck and repeated brew update:

$ Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask...
To checkout master in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask run:
  'cd /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
To checkout master in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core run:
  'cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Already up-to-date.
+ brew upgrade --all
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask...
To checkout master in /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask run:
  'cd /usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
Checking out v1.0.0 in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
To checkout master in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core run:
  'cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core && git checkout master
fatal: Cannot update paths and switch to branch 'v1.0.0' at the same time.
Did you intend to checkout 'refs/tags/1.0.0' which can not be resolved as commit?
fatal: Needed a single revision
invalid upstream refs/tags/1.0.0
@MikeMcQuaid

This comment has been minimized.

Copy link
Member

MikeMcQuaid commented Sep 21, 2016

@mgol Can I see the output of cd $(brew --repo) && git config homebrew.devcmdrun? Also, can you try to brew pull https://github.com/Homebrew/brew/pull/1060 && brew update --force and see how that behaves? Thanks!

@MikeMcQuaid MikeMcQuaid referenced this pull request Sep 21, 2016

Merged

update.sh: tweak tag handling. #1060

5 of 5 tasks complete
@MikeMcQuaid

This comment has been minimized.

Copy link
Member

MikeMcQuaid commented Sep 21, 2016

@mgol and also, actually, echo HOMEBREW_DEVELOPER: $HOMEBREW_DEVELOPER. Thanks!

@mgol

This comment has been minimized.

Copy link
Contributor

mgol commented Sep 21, 2016

This:

cd $(brew --repo) && git config homebrew.devcmdrun

cds to /usr/local/Homebrew and prints nothing.

This:

brew pull https://github.com/Homebrew/brew/pull/1060 && brew update --force

resulted in:

rew pull https://github.com/Homebrew/brew/pull/1060 && brew update --force
Warning: Current branch is v1.0.0: do you need to pull inside master?
==> Fetching patch 
Patch: https://github.com/Homebrew/brew/pull/1060.patch
==> Applying patch
Applying: update.sh: tweak tag handling.
==> Patch closes issue #1060
==> Patch changed:
 Library/Homebrew/cmd/update.sh | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Updated Homebrew from d5b6ecf to 09e0b09.
Updated 2 taps (caskroom/cask, homebrew/core).
==> Updated Formulae
antigen               fzf                   gupnp                 ipfs                  offlineimap           sonar-runner        
aws-sdk-cpp           global                imagemagick           khard                 reposurgeon         

And now brew update && brew upgrade behaves normally.

@mgol

This comment has been minimized.

Copy link
Contributor

mgol commented Sep 21, 2016

About the last bit:

$ echo HOMEBREW_DEVELOPER: $HOMEBREW_DEVELOPER
HOMEBREW_DEVELOPER:

So it's empty.

@MikeMcQuaid

This comment has been minimized.

Copy link
Member

MikeMcQuaid commented Sep 21, 2016

@mgol Thanks, will 🚢 #1060 to fix this shortly.

@zmwangx zmwangx referenced this pull request Sep 21, 2016

Closed

Cannot brew update anymore #1067

3 of 3 tasks complete
@hkdobrev

This comment has been minimized.

Copy link

hkdobrev commented Sep 21, 2016

Just my 2 cents on tag handling - Composer did a very nice thing when going stable. They have 3 update channels:

  • stable is the default and updates only to stable releases (following semver)
  • preview updates to any tag including alpha, beta and rc tags according to semver
  • snapshot updates to latest master.

The other nice thing is that once you update with a specific channel it keeps you on that channel and it explicitly stays what channel it uses the next time you update. This is really nicely working for Composer, not sure if it'd be good for Homebrew.

In short I really like the use of non-stable and stable tags which Composer does and hope Homebrew does something like that.

PS:

@jtsom

This comment has been minimized.

Copy link

jtsom commented Sep 21, 2016

So how do I get Homebrew to update from 1.0.0 to 1.0.1? brew update just says I'm up to date.

I did get the fatal: Cannot update paths and switch to branch 'v1.0.1' at the same time. message, but I can't get 1.0.1 now. Short of nuking the whole thing (which I don't want to have to reinstall all my packages), what's the solutions?

@holybit

This comment has been minimized.

Copy link

holybit commented Sep 21, 2016

In my case it seems as if Homebrew updated correctly to 1.0.1 but the brew command doesn't seem to be showing the correct version?

$ brew update
Checking out v1.0.1 in /usr/local/Homebrew...
To checkout v1.0.1 in /usr/local/Homebrew run:
  'cd /usr/local/Homebrew && git checkout v1.0.1
Already up-to-date.

$ cd /usr/local/Homebrew && git checkout v1.0.1
Already on 'v1.0.1'

$ brew --version
Homebrew 1.0.0 (git revision a381b; last commit 2016-09-21)
Homebrew/homebrew-core (git revision 48e0; last commit 2016-09-21)

$ cd /usr/local/Homebrew
$ git show-ref --tags | tail -2
d5b6ecfc5078041ddf5f61b259c57f81d5c50fcc refs/tags/1.0.0
a381b597d3ebf40887b50657df269d867a209eb7 refs/tags/1.0.1
@MikeMcQuaid

This comment has been minimized.

Copy link
Member

MikeMcQuaid commented Sep 22, 2016

but the brew command doesn't seem to be showing the correct version?

Ignore this, it was me being an idiot and not remembering to bump the version number in the output.

@Homebrew Homebrew locked and limited conversation to collaborators May 3, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.