Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgres 9 service pg_ctl command is returning invalid path #9011

Closed
sinemetu1 opened this issue Jan 4, 2018 · 6 comments
Closed

Postgres 9 service pg_ctl command is returning invalid path #9011

sinemetu1 opened this issue Jan 4, 2018 · 6 comments

Comments

@sinemetu1
Copy link

@sinemetu1 sinemetu1 commented Jan 4, 2018

Hello,

Our travis builds are using the postgresql service feature. For some reason builds in the past few days have been failing because pg_ctl --bindir reports an invalid path (for version 10? 馃). I've created a travis build and yaml to reproduce this. Here is the .travis.yaml:

sudo: false
language: python
python: 3.6

services:
    - postgresql

script:
    - which pg_ctl
    - echo $PATH
    - pg_config --bindir
    - ls -lh /usr/lib/postgresql/
    - ls -lh /usr/lib/postgresql/10/bin/
    - ls -lh /usr/lib/postgresql/10/bin/pg_ctl

Here's the travis build: https://travis-ci.org/sinemetu1/test-pg-travis/builds/325130012

And the related section of the logs:

...
$ which pg_ctl
The command "which pg_ctl" exited with 1.
$ echo $PATH
/home/travis/virtualenv/python3.6.3/bin:/home/travis/bin:/home/travis/.local/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.9.1/bin:/home/travis/.kiex/elixirs/elixir-1.4.5/bin:/home/travis/.kiex/bin:/home/travis/.rvm/gems/ruby-2.4.1/bin:/home/travis/.rvm/gems/ruby-2.4.1@global/bin:/home/travis/.rvm/rubies/ruby-2.4.1/bin:/home/travis/gopath/bin:/usr/local/phantomjs/bin:/usr/local/phantomjs:/usr/local/neo4j-3.2.7/bin:/usr/local/maven-3.5.2/bin:/usr/local/cmake-3.9.2/bin:/usr/local/clang-5.0.0/bin:/home/travis/.gimme/versions/go1.7.4.linux.amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin
The command "echo $PATH" exited with 0.
$ pg_config --bindir
/usr/lib/postgresql/10/bin
The command "pg_config --bindir" exited with 0.
$ ls -lh /usr/lib/postgresql/
total 0
drwxr-xr-x 4 root root 26 Dec  5 19:59 9.2
drwxr-xr-x 4 root root 26 Dec  5 19:59 9.3
drwxr-xr-x 4 root root 26 Dec  5 20:00 9.4
drwxr-xr-x 4 root root 26 Dec  5 20:00 9.5
drwxr-xr-x 4 root root 26 Dec  5 20:00 9.6
The command "ls -lh /usr/lib/postgresql/" exited with 0.
$ ls -lh /usr/lib/postgresql/10/bin/
ls: cannot access /usr/lib/postgresql/10/bin/: No such file or directory
...

Thanks for your time and help!

@BanzaiMan
Copy link
Contributor

@BanzaiMan BanzaiMan commented Jan 4, 2018

This feels like a packaging issue. We are installing postgresql-common, which provides this file. Even when there is no PostgreSQL 10 package installed on the system, the binary /usr/bin/pg_config.libpq-dev has strings pointing to version 10.

travis@42efb11a6c57:~$ dpkg -l | grep postgres
ii  pgdg-keyring                             2017.3                                     all          keyring for apt.postgresql.org
ii  postgresql-9.2                           9.2.24-1.pgdg14.04+1                       amd64        object-relational SQL database, version 9.2 server
ii  postgresql-9.2-postgis-2.3-scripts       2.3.3+dfsg-1.pgdg14.04+1                   all          Geographic objects support for PostgreSQL 9.2 -- scripts
ii  postgresql-9.3                           9.3.20-1.pgdg14.04+1                       amd64        object-relational SQL database, version 9.3 server
ii  postgresql-9.3-postgis-2.3-scripts       2.3.3+dfsg-1.pgdg14.04+1                   all          Geographic objects support for PostgreSQL 9.3 -- scripts
ii  postgresql-9.4                           9.4.15-1.pgdg14.04+1                       amd64        object-relational SQL database, version 9.4 server
ii  postgresql-9.4-postgis-2.3-scripts       2.3.3+dfsg-1.pgdg14.04+1                   all          Geographic objects support for PostgreSQL 9.4 -- scripts
ii  postgresql-9.5                           9.5.10-1.pgdg14.04+1                       amd64        object-relational SQL database, version 9.5 server
ii  postgresql-9.5-postgis-2.3-scripts       2.3.3+dfsg-1.pgdg14.04+1                   all          Geographic objects support for PostgreSQL 9.5 -- scripts
ii  postgresql-9.6                           9.6.6-1.pgdg14.04+1                        amd64        object-relational SQL database, version 9.6 server
ii  postgresql-9.6-postgis-2.3-scripts       2.3.3+dfsg-1.pgdg14.04+1                   all          Geographic objects support for PostgreSQL 9.6 -- scripts
ii  postgresql-client                        9.3+154ubuntu1.1                           all          front-end programs for PostgreSQL (supported version)
ii  postgresql-client-9.2                    9.2.24-1.pgdg14.04+1                       amd64        front-end programs for PostgreSQL 9.2
ii  postgresql-client-9.3                    9.3.20-0ubuntu0.14.04                      amd64        front-end programs for PostgreSQL 9.3
ii  postgresql-client-9.4                    9.4.15-1.pgdg14.04+1                       amd64        front-end programs for PostgreSQL 9.4
ii  postgresql-client-9.5                    9.5.10-1.pgdg14.04+1                       amd64        front-end programs for PostgreSQL 9.5
ii  postgresql-client-9.6                    9.6.6-1.pgdg14.04+1                        amd64        front-end programs for PostgreSQL 9.6
ii  postgresql-client-common                 188.pgdg14.04+1                            all          manager for multiple PostgreSQL client versions
ii  postgresql-common                        188.pgdg14.04+1                            all          PostgreSQL database-cluster manager
ii  postgresql-contrib-9.2                   9.2.24-1.pgdg14.04+1                       amd64        additional facilities for PostgreSQL
ii  postgresql-contrib-9.3                   9.3.20-1.pgdg14.04+1                       amd64        additional facilities for PostgreSQL
ii  postgresql-contrib-9.4                   9.4.15-1.pgdg14.04+1                       amd64        additional facilities for PostgreSQL
ii  postgresql-contrib-9.5                   9.5.10-1.pgdg14.04+1                       amd64        additional facilities for PostgreSQL
ii  postgresql-contrib-9.6                   9.6.6-1.pgdg14.04+1                        amd64        additional facilities for PostgreSQL
travis@42efb11a6c57:~$ strings /usr/bin/pg_config.libpq-dev | grep postgres
/usr/lib/postgresql/10/bin
Report bugs to <pgsql-bugs@postgresql.org>.
'--with-icu' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' '--with-tclconfig=/usr/lib/x86_64-linux-gnu/tcl8.6' '--with-includes=/usr/include/tcl8.6' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/10/man' '--docdir=/usr/share/doc/postgresql-doc-10' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/10' '--bindir=/usr/lib/postgresql/10/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' 'CFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' '--with-selinux' 'CPPFLAGS=-D_FORTIFY_SOURCE=2'
postgres-10
/usr/share/postgresql/10
/etc/postgresql-common
/usr/include/postgresql
/usr/lib/postgresql/10/lib
/usr/share/postgresql/10/man
/usr/include/postgresql/10/server
/usr/share/doc/postgresql-doc-10

@BanzaiMan
Copy link
Contributor

@BanzaiMan BanzaiMan commented Jan 4, 2018

libpq-dev points to version 10:

travis@42efb11a6c57:~$ dpkg -l | grep libpq
ii  libpq-dev                                10.1-1.pgdg14.04+1                         amd64        header files for libpq5 (PostgreSQL library)
ii  libpq5:amd64                             10.1-1.pgdg14.04+1                         amd64        PostgreSQL C client library

@BanzaiMan
Copy link
Contributor

@BanzaiMan BanzaiMan commented Jan 4, 2018

Looking at the content of /usr/bin/pg_config (which is just a wrapper), I believe that it is (somewhat) assuming that postgresql-server-dev-X.Y is also installed.

travis@988f659570cf:~$ sudo apt-get install postgresql-server-dev-9.4
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  postgresql-server-dev-9.4
0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded.
Need to get 645 kB of archives.
After this operation, 3,344 kB of additional disk space will be used.
Get:1 http://apt.postgresql.org/pub/repos/apt trusty-pgdg/main amd64 postgresql-server-dev-9.4 amd64 9.4.15-1.pgdg14.04+1 [645 kB]
Fetched 645 kB in 0s (685 kB/s)
Selecting previously unselected package postgresql-server-dev-9.4.
(Reading database ... 72205 files and directories currently installed.)
Preparing to unpack .../postgresql-server-dev-9.4_9.4.15-1.pgdg14.04+1_amd64.deb ...
Unpacking postgresql-server-dev-9.4 (9.4.15-1.pgdg14.04+1) ...
Setting up postgresql-server-dev-9.4 (9.4.15-1.pgdg14.04+1) ...
travis@988f659570cf:~$ pg_config --bindir
/usr/lib/postgresql/9.4/bin

Unfortunately, we can't install these packages for all the versions we install on our build images, because /usr/bin/pg_config always runs the latest version. (See https://stackoverflow.com/q/30143046/136333.)

Ultimately, I think this is a bug with PostgreSQL.

@sinemetu1
Copy link
Author

@sinemetu1 sinemetu1 commented Jan 4, 2018

Ah okay. Thanks for the help!

@BanzaiMan
Copy link
Contributor

@BanzaiMan BanzaiMan commented Jan 5, 2018

If you are interested in getting useful output from pg_config, I think you can install postgresql-server-dev-9.4 (or whatever version you are using); e.g.,

addons:
  apt:
    packages:
      - postgresql-server-dev-9.4

@sinemetu1
Copy link
Author

@sinemetu1 sinemetu1 commented Jan 5, 2018

Using the addon worked for us. Thanks!

benjaminjkraft added a commit to learning-unlimited/ESP-Website that referenced this issue Jan 21, 2018
Something is funky with Travis's postgres packaging; I used the fix
suggested here: travis-ci/travis-ci#9011 (comment)
hwatheod added a commit to learning-unlimited/ESP-Website that referenced this issue Feb 4, 2018
Something is funky with Travis's postgres packaging; I used the fix
suggested here: travis-ci/travis-ci#9011 (comment)
hwatheod added a commit to learning-unlimited/ESP-Website that referenced this issue Feb 4, 2018
Something is funky with Travis's postgres packaging; I used the fix
suggested here: travis-ci/travis-ci#9011 (comment)
sfllaw added a commit to sfllaw/pghkp that referenced this issue Apr 16, 2018
travis-ci/travis-ci#9011 has trouble with PostgreSQL 10.

Falling back to PostgreSQL 9.4 works just fine.
sfllaw added a commit to sfllaw/pghkp that referenced this issue Apr 16, 2018
sfllaw added a commit to sfllaw/pghkp that referenced this issue Apr 16, 2018
sfllaw added a commit to sfllaw/pghkp that referenced this issue Apr 16, 2018
dev-zero added a commit to dev-zero/aiida-core that referenced this issue Jul 6, 2018
Based on travis-ci/travis-ci#9011 it seems we
have to specify explicitly the dependency on the postgresql-dev package
by now.
dev-zero added a commit to dev-zero/aiida-core that referenced this issue Jul 9, 2018
Based on travis-ci/travis-ci#9011 it seems we
have to specify explicitly the dependency on the postgresql-dev package
by now.
DropD added a commit to aiidateam/aiida-core that referenced this issue Jul 10, 2018
* travis: explicitly install postgresql-server-dev-9.5

Based on travis-ci/travis-ci#9011 it seems we
have to specify explicitly the dependency on the postgresql-dev package
by now.

* Revert "skip pgtest related testcases until #1722 is fixed (#1724)"

This reverts commit 777c16c and fixes #1723

* travis: add proper postgresql bin-dir to path

fixes #1722

* travis: fix ssh-keygen for private instances
dev-zero added a commit to dev-zero/aiida-core that referenced this issue Jul 10, 2018
Based on travis-ci/travis-ci#9011 it seems we
have to specify explicitly the dependency on the postgresql-dev package
by now.
DropD added a commit to aiidateam/aiida-core that referenced this issue Jul 10, 2018
* travis: explicitly install postgresql-server-dev-9.5

Based on travis-ci/travis-ci#9011 it seems we
have to specify explicitly the dependency on the postgresql-dev package
by now.

* travis: add proper postgresql bin-dir to path

fixes #1722

* travis: fix ssh-keygen for private instances
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants