Unable to install Postgres after hitting incompatible database files, even after Postgres is removed

[aprescott]

This seems to be similar to #34885.

At some point in the last few days, I upgraded my installed Homebrew packages, and then today I encountered this error when trying to start Postgres:

$ postgres -D /usr/local/var/postgres
LOG:  skipping missing configuration file "/usr/local/var/postgres/postgresql.auto.conf"
FATAL:  database files are incompatible with server
DETAIL:  The data directory was initialized by PostgreSQL version 9.3, which is not compatible with this version 9.4.0.

So I figured I would just reinstall Postgres using brew. The same error kept showing, though. I decided to manually remove Postgres from /usr/local and then try the install again.

The shell commands I ran are below, but note that, like, #34885, even starting with a clean slate, I end up with /usr/local/var/postgres owned by root, not my user. The directory is in fact empty, and no errors were given during installation, as you can see here.

I've searched for the database incompatibility error and the error about the missing .conf file, but nothing came up, except for #34885, which was closed due to user error from manually creating /usr/local/var/postgres. However, in my case, even after rming that directory, it comes back as owned by root.

It seems I'm unable to install Postgres.

$ brew uninstall postgres
Uninstalling /usr/local/Cellar/postgresql/9.4.0...
$ brew uninstall postgresql
Error: No such keg: /usr/local/Cellar/postgresql
$ brew remove postgres
Error: No such keg: /usr/local/Cellar/postgresql
$ brew remove postgresql
Error: No such keg: /usr/local/Cellar/postgresql
$ rm /Library/Caches/Homebrew/postgresql-9.*
$ ps aux | grep postgres
adamprescott    18364   0.0  0.0  2432772    472 s000  R+   10:47AM   0:00.00 grep postgres
$ ps aux | grep psql
adamprescott    18366   0.0  0.0  2441988    668 s000  S+   10:47AM   0:00.00 grep psql
$ rm -rf /usr/local/var/postgres/*
$ rmdir /usr/local/var/postgres
$ find /usr/local -iname "*postgres*"
/usr/local/Library/Aliases/postgres
/usr/local/Library/Formula/check_postgres.rb
/usr/local/Library/Formula/postgres-xc.rb
/usr/local/Library/Formula/postgresql.rb
$ brew upgrade
$ brew install postgres
==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/postgresql-9.4.0.yosemite.bottle.tar.gz
######################################################################## 100.0%
==> Pouring postgresql-9.4.0.yosemite.bottle.tar.gz
==> Caveats
If builds of PostgreSQL 9 are failing and you have version 8.x installed,
you may need to remove the previous version first. See:
  https://github.com/Homebrew/homebrew/issues/2510

To migrate existing data from a previous major version (pre-9.4) of PostgreSQL, see:
  http://www.postgresql.org/docs/9.4/static/upgrading.html

When installing the postgres gem, including ARCHFLAGS is recommended:
  ARCHFLAGS="-arch x86_64" gem install pg

To install gems without sudo, see the Homebrew documentation:
https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/Gems,-Eggs-and-Perl-Modules.md

To have launchd start postgresql at login:
    ln -sfv /usr/local/opt/postgresql/*.plist ~/Library/LaunchAgents
Then to load postgresql now:
    launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Or, if you don't want/need launchctl, you can just run:
    postgres -D /usr/local/var/postgres
==> Summary
�  /usr/local/Cellar/postgresql/9.4.0: 2990 files, 40M
$ ln -sfv /usr/local/opt/postgresql/*.plist ~/Library/LaunchAgents
/Users/adamprescott/Library/LaunchAgents/homebrew.mxcl.postgresql.plist -> /usr/local/opt/postgresql/homebrew.mxcl.postgresql.plist
$ brew info postgres
# ...
$ launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
$ launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
$ postgres -D /usr/local/var/postgres
postgres cannot access the server configuration file "/usr/local/var/postgres/postgresql.conf": Permission denied
$ ls -la /usr/local/var
total 0
drwxr-xr-x  10 adamprescott  admin   340B Dec 24 10:52 .
drwxrwxr-x  22 root          admin   748B Dec 19 10:58 ..
drwxr-xr-x   3 adamprescott  admin   102B Oct 16 20:41 cache
drwxr-xr-x   3 adamprescott  admin   102B Oct 16 20:41 db
drwxr-xr-x   4 adamprescott  admin   136B Oct 16 20:41 log
drwxr-xr-x   5 adamprescott  admin   170B Oct 16 21:03 mongodb
drwxr-xr-x  25 adamprescott  admin   850B Dec 24 10:11 mysql
drwxr--r--   2 root          admin    68B Dec 24 10:52 postgres
drwxr-xr-x   2 adamprescott  admin    68B Dec 19 10:59 run
-rw-r--r--   1 adamprescott  admin     0B Mar 20  2014 stdout
$ sudo ls -la /usr/local/var/postgres
Password:
total 0
drwxr--r--   2 root          admin   68 Dec 24 10:52 .
drwxr-xr-x  10 adamprescott  admin  340 Dec 24 10:52 ..
$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry and just ignore them. Thanks!

Warning: A newer Command Line Tools release is available.
Update them from Software Update in the App Store.

[aprescott]

rm -rf /usr/local/var/postgres && initdb /usr/local/var/postgres -E utf8 does in fact solve this problem, so I'm a little curious why the other issue was closed. Shouldn't brew install postgres handle this for me? Am I missing something?

[MikeMcQuaid]

The caveats say:

To migrate existing data from a previous major version (pre-9.4) of PostgreSQL, see:
http://www.postgresql.org/docs/9.4/static/upgrading.html

Did that not help?

[aprescott]

@MikeMcQuaid I missed that when upgrading. But the part I don't understand here is I removed postgres and ran (among other things in my shell paste)

$ rm -rf /usr/local/var/postgres/*
$ rmdir /usr/local/var/postgres

which should have given me a completely blank slate as far as the postgres package goes, right? But then my brew install postgres leaves me with a root-owned /usr/local/var/postgres, and in no runnable state. The initdb step was necessary even when starting from an empty state.

[MikeMcQuaid]

If #{var}/postgres doesn't exist we should run it for you on reinstall.

[aprescott]

It is created on install, but owned by root and empty. Isn't that a bug?

[jacknagel]

Who owns /usr/local/var?

[aprescott]

@jacknagel you can see this in my paste in my opening issue comment with the rest of the info:

drwxr-xr-x  10 adamprescott  admin   340B Dec 24 10:52 .

[mrThe]

I just found this issue, and i fixed it by this manual: http://stackoverflow.com/questions/24379373/how-to-upgrade-the-postgre-from-9-3-to-9-4-without-lossing-data

tl;dr: just migrate your database from 9.3 to 9.4.

[aprescott]

@mrThe, again, the point of this issue is that my installation is broken when starting with no database. There is no database to upgrade because I removed it.

[aprescott]

I have renamed this issue title to hopefully clarify that this is beyond not catching the upgrade warning (which to be honest should be clearer in error messaging, but alas).

[deoxxa]

Same deal here, not sure what the deal is. Somehow, this root-owned postgres directory is being created upon install, stopping the createdb command from working.

➜  var git:(master) ls -l
total 0
drwxr-xr-x   9 conrad  admin  306  9 Feb 14:41 cayley
drwxr-xr-x   3 conrad  admin  102 26 Oct 17:30 db
drwxr-xr-x   3 conrad  admin  102 26 Oct 17:39 elasticsearch
drwxr-xr-x   3 conrad  admin  102  8 Nov 07:07 lib
drwxr-xr-x   6 conrad  admin  204 28 Nov 21:17 log
drwxr-xr-x  10 conrad  admin  340 26 Oct 17:55 mongodb
drwxr-xr-x  13 conrad  admin  442  9 Feb 14:41 mysql
drwxr-xr-x   2 conrad  admin   68 10 Dec 09:24 run
➜  var git:(master) brew install postgres
==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/postgresql-9.4.1.yosemite.bottle.1.tar.gz
Already downloaded: /Library/Caches/Homebrew/postgresql-9.4.1.yosemite.bottle.1.tar.gz
==> Pouring postgresql-9.4.1.yosemite.bottle.1.tar.gz
==> Caveats
If builds of PostgreSQL 9 are failing and you have version 8.x installed,
you may need to remove the previous version first. See:
  https://github.com/Homebrew/homebrew/issues/2510

To migrate existing data from a previous major version (pre-9.4) of PostgreSQL, see:
  http://www.postgresql.org/docs/9.4/static/upgrading.html

To have launchd start postgresql at login:
    ln -sfv /usr/local/opt/postgresql/*.plist ~/Library/LaunchAgents
Then to load postgresql now:
    launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Or, if you don't want/need launchctl, you can just run:
    postgres -D /usr/local/var/postgres
==> Summary
🍺  /usr/local/Cellar/postgresql/9.4.1: 2996 files, 40M
➜  var git:(master) ls -l
total 0
drwxr-xr-x   9 conrad  admin  306  9 Feb 14:41 cayley
drwxr-xr-x   3 conrad  admin  102 26 Oct 17:30 db
drwxr-xr-x   3 conrad  admin  102 26 Oct 17:39 elasticsearch
drwxr-xr-x   3 conrad  admin  102  8 Nov 07:07 lib
drwxr-xr-x   6 conrad  admin  204 28 Nov 21:17 log
drwxr-xr-x  10 conrad  admin  340 26 Oct 17:55 mongodb
drwxr-xr-x  13 conrad  admin  442  9 Feb 14:41 mysql
drwxr--r--   2 root    admin   68 10 Feb 08:53 postgres
drwxr-xr-x   2 conrad  admin   68 10 Dec 09:24 run
➜  var git:(master) postgres -D /usr/local/var/postgres
postgres cannot access the server configuration file "/usr/local/var/postgres/postgresql.conf": Permission denied

[command-tab]

I just ran into this issue. I had nothing critical in my local postgres, so rather than figuring out how to upgrade from 9.3.x to 9.4.x, I just blew away all the installed versions and data by running brew uninstall --force postgresql and rm -rf /usr/local/var/postgres. And, as others have noted, performing a re-install succeeds, but the data directory is never populated, so the install won't actually start or initdb.

Here's a gist of the brew install postgresql -v after performing the above removals: https://gist.github.com/commandtab/92049df70fe8cd7b349f

I'm on OS X 10.10.2.