Suggestion to initially pin packages that break things by automatic upgrade

[rudyryk]

For example brew upgrade systematically breaks PostgreSQL installation because 9.6 is not compatible with 9.5 and so on. The recent 10.0 update breaks things again.

Having installed PostgreSQL 9.6 with non-empty databases I just run:

brew update
brew upgrade

I would expect that under normal conditions it will just upgrade without breaking things. But with PostgreSQL it breaks every time.

I would expect to be warned and prevented from breaking upgrades.

The simplest solution as I can see would be to just pin some packages by default. User will be warned that upgrade for some packages was skipped. And it's far better (I believe) then silently break things.

[rudyryk]

Well, developers spend days setting up their environment, also doing this gradually for months. And it can be broken with single upgrade command. Better not upgrade at all. It's amazing how modern software just doesn't care of being safe to upgrade.

[ilovezfs]

@rudyryk I recommend you read #19062 and look at how much work went in to it before you make sweeping statements like that.

[ilovezfs]

If you need to stay on an older version use postgresql@9.4, postgresql@9.5, or postgresql@9.6.

[rudyryk]

@ilovezfs And still we need to go thought this: https://gist.github.com/giannisp/ebaca117ac9e44231421f04e7796d5ca

Not the best things to do on weekend (or holidays). It's great that manual upgrading workflow is supported and quite smooth, but why in the first place not avoid such situations at all?! "Move fast, break things" is probably good idea for startups, but not for package managers, I believe.

[ilovezfs]

@rudyryk you may enjoy reading this https://medium.com/@mikeal/stop-supporting-old-releases-70cfa0e04b0c

[rudyryk]

@ilovezfs Ha-ha :) Ok and what about deprecation warnings approach? I see nothing bad in showing BIG RED WARNING of deprecated package and forcing update after some grace period, that's established practice in C, Python world for example. And this is good one.

[ilovezfs]

Like I said, you can

brew uninstall postgresql
brew install postgresql@9.6

if you need to stay on an older version.

In terms of warnings about deprecations, that's primarily the function of upstream documentation. The current caveats in postgresql are sufficient for most people, though.

  def caveats; <<~EOS
    To migrate existing data from a previous major version of PostgreSQL, see:
      https://www.postgresql.org/docs/10/static/upgrading.html

      You will need your previous PostgreSQL installation from brew to perform
      `pg_upgrade` or `pg_dumpall` depending on your upgrade method.

      Do not run `brew cleanup postgresql` until you have performed the migration.
    EOS
  end

If you think another sentence should be added there, feel free to open a PR and we can take a look.

[rudyryk]

@ilovezfs

if you need to stay on an older version.

No. I need to have option to upgrade safely when it's time. Or upgrade right now after confirmation: "Yes, I understand things may break after doing this". The current default behaviour of upgrade command is like a bomb. You can't even see upfront and control what things will break after upgrading.

But usually I'd expect from package manager to do safe upgrades, that's the purpose of using package manager after all. I would insist that default behaviour which 100% will break environment is wrong for package manager. Most mature package managers (except node - their package manager is one of most evil thing ever) will go interactive on dangerous upgrade.

I'm not saying this feature is needed ASAP, but why not consider it instead of just ignoring?

[ilovezfs]

The brew outdated command is there so you can see what will be upgraded, and brew info will show you the caveats about packages before installing them.

As a design principle, brew has always avoided any form of interactive commands.

The current default behaviour of upgrade command is like a bomb. You can't even see upfront and control what things will break after upgrading.

This is not true. Reverse dependencies are updated and rebottled at the same time, in the same PR, as their dependencies. So brew upgrade is safe for the vast majority of use cases.

[rudyryk]

@ilovezfs Ok. That means it is up to user to know whether update is safe or not. "Hmm... is that service is safe to upgrade?.. ok, let's try! Ooops..." With PostgreSQL update is never safe, know that from (unlucky) experience :)

UPD. Hard to imagine to check every of dozens package listed after brew outdated.

[ilovezfs]

That means it is up to user to know whether update is safe or not.

Correct. We ship the latest stable version of each package if it, and all of its reverse dependencies, pass CI. If you consider that unsafe in the case of particular packages, then that is indeed up to your judgment.

[rudyryk]

If you consider that unsafe

Hmm, not me considering that unsafe. Data format of newer PosgreSQL is by design backward incompatible with previous versions for a long-long time. Ubuntu distribs even ships with locked PosgreSQL as far as I know. So in this case we usually need to specify explicitly new version - when need new features.

So now after update to PostgreSQL 10.0 I just have to brew pin postgresql to prevent accidental break by upgrade (with coming soon 10.1)

[ilovezfs]

So now after update to PostgreSQL 10.0 I just have to brew pin postgresql to prevent accidental break by upgrade (with coming soon 10.1)

brew pin doesn't offer much protection actually. If something depends_on :postgresql then it will override the pin.

[ilovezfs]

Here is an example:

iMac-TMP:homebrew-core joe$ git checkout ca57e9712ff3a5a0cc1424b5b4ea1c4945ae0b40 -- Formula/vim.rb Formula/perl.rb
iMac-TMP:homebrew-core joe$ brew install vim 
==> Installing dependencies for vim: perl, libyaml, ruby
==> Installing vim dependency: perl
==> Downloading https://homebrew.bintray.com/bottles/perl-5.26.0.el_capitan.bottle.tar.gz
######################################################################## 100.0%
==> Pouring perl-5.26.0.el_capitan.bottle.tar.gz
==> Caveats
By default non-brewed cpan modules are installed to the Cellar. If you wish
for your modules to persist across updates we recommend using `local::lib`.

You can set that up like this:
  PERL_MM_OPT="INSTALL_BASE=$HOME/perl5" cpan local::lib
  echo 'eval "$(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)"' >> ~/.bash_profile
==> Summary
🍺  /usr/local/Cellar/perl/5.26.0: 2,389 files, 57.3MB
==> Installing vim dependency: libyaml
==> Downloading https://homebrew.bintray.com/bottles/libyaml-0.1.7.el_capitan.bottle.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/libyaml-0.1.7.el_capitan.bottle.tar.gz
==> Pouring libyaml-0.1.7.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/libyaml/0.1.7: 8 files, 312.6KB
==> Installing vim dependency: ruby
==> Downloading https://homebrew.bintray.com/bottles/ruby-2.4.2_1.el_capitan.bottle.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/ruby-2.4.2_1.el_capitan.bottle.tar.gz
==> Pouring ruby-2.4.2_1.el_capitan.bottle.tar.gz
==> Caveats
Emacs Lisp files have been installed to:
  /usr/local/share/emacs/site-lisp/ruby
==> Summary
🍺  /usr/local/Cellar/ruby/2.4.2_1: 3,042 files, 16.3MB
==> Installing vim
==> Downloading https://homebrew.bintray.com/bottles/vim-8.0.1150.el_capitan.bottle.tar.gz
######################################################################## 100.0%
==> Pouring vim-8.0.1150.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/vim/8.0.1150: 1,418 files, 22.7MB
iMac-TMP:homebrew-core joe$ brew pin perl
iMac-TMP:homebrew-core joe$ git checkout origin/master -- Formula/vim.rb Formula/perl.rb
iMac-TMP:homebrew-core joe$ brew outdated
aws-shell (0.2.0) < 0.2.0_1
coq (8.7.0) < 8.7.0_1
menhir (20171013) < 20171013_1
perl (5.26.0) < 5.26.1 [pinned at 5.26.0]
vim (8.0.1150) < 8.0.1250
iMac-TMP:homebrew-core joe$ brew upgrade vim
==> Upgrading 1 outdated package, with result:
vim 8.0.1250
==> Upgrading vim 
==> Installing dependencies for vim: perl
==> Installing vim dependency: perl
==> Downloading https://homebrew.bintray.com/bottles/perl-5.26.1.el_capitan.bottle.1.tar.gz
Already downloaded: /Users/joe/Library/Caches/Homebrew/perl-5.26.1.el_capitan.bottle.1.tar.gz
==> Pouring perl-5.26.1.el_capitan.bottle.1.tar.gz
==> Caveats
By default non-brewed cpan modules are installed to the Cellar. If you wish
for your modules to persist across updates we recommend using `local::lib`.

You can set that up like this:
  PERL_MM_OPT="INSTALL_BASE=$HOME/perl5" cpan local::lib
  echo 'eval "$(perl -I$HOME/perl5/lib/perl5 -Mlocal::lib)"' >> ~/.bash_profile
==> Summary
🍺  /usr/local/Cellar/perl/5.26.1: 2,395 files, 57.2MB
==> Installing vim
==> Downloading https://homebrew.bintray.com/bottles/vim-8.0.1250.el_capitan.bottle.tar.gz
######################################################################## 100.0%
==> Pouring vim-8.0.1250.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/vim/8.0.1250: 1,419 files, 22.7MB
iMac-TMP:homebrew-core joe$ 

[rudyryk]

@ilovezfs Ok, I see - the problem is deeper than it seemed to me. Probably some services (like PorstgreSQL) are just not supposed to be handled like that and that's core of the problem described above.

[ilovezfs]

@rudyryk really your best bet, if you're looking for a less disruptive postgresql experience, is to use one of the @ formulae. That is our supported way of achieving "pinning" in the way you're thinking about it. When 10.1 comes out there will likely be a new postgresql@10.0 formula, which you can switch over to instead of upgrading to 10.1.

[MikeMcQuaid]

@rudyryk I've opened a PR to try and make this better in #21244. Would you mind reviewing and testing?

[MikeMcQuaid]

#21244 has been merged.