Dependency on psycopg2-binary

[jacopofar]

Related to #144 I guess, since today the installation of the backage is broken on our build because it tries to compile the psycopg2 package and needs additional system dependencies.
To reproduce in a Dockerfile:

FROM debian:stretch

ENV LANG C.UTF-8

RUN apt-get update -qq && apt-get install -y python3-pip \
    python3-setuptools \
    python3-wheel
RUN pip3 install sqlalchemy-redshift

Will fail with the error pg_config executable not found.

I resolved by adding apt-get install -y build-essential libpq-dev python3-dev to the build.
Is there any plan to move to psycopg2-binary as a dependency, or maybe to create a parallel package that uses that?

[jacopofar]

It's also possible instead to just add psycopg2==2.7.7 to the dependencies for the project, since psycopg2 2.8 was released 5 hours ago and is the first version to require the building tools.

[venthur]

It is probably better if sqlalchemy-redshift would depend on psycopg2-binary.

[mattyb]

http://initd.org/psycopg/docs/install.html#binary-install-from-pypi

Note The -binary package is meant for beginners to start playing with Python and PostgreSQL without the need to meet the build requirements. If you are the maintainer of a publish package depending on psycopg2 you shouldn’t use psycopg2-binary as a module dependency. For production use you are advised to use the source distribution.

[jklukas]

The workaround of staying on psycopg2=2.7.7 is the easiest thing to do for an immediate fix if your builds are failing.

As @mattyb linked, though, explicitly depending on psycopg2-binary is recommended against. We're in the unfortunate situation of there being a burden on users to understand the difference between psycopg2 and psycopg2-binary and users having to explicitly opt in to using the -binary package.

If we had a callout in the README and built documentation about the problem, would that have been helpful in figuring this out, @jacopofar?

If someone can provide a sample pip install configuration that gets this project working with psycopg2-binary, I'd appreciate seeing it. I think there's some pip override needed to prevent this package from pulling in psycopg2.

[jacopofar]

Yes given that the problem with psycopg2 wheels seems far to be solved I think an edit to the README is the easiest option.

What I did in my project was to put this in requirements.txt:

# specific psycopg2 version required to not break sqlalchemy-redshift
# see https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/165
psycopg2==2.7.7
sqlalchemy-redshift==0.7.3

as for solving the problem in this project I don't think there's an easy way, either one pins psycopg2 2.7.7 or installs a C compiler, Python headers and libpq5 headers on the system. Maybe an additional sqlalchemy-redshift-binary would do the trick but after seeing the problem at the link above I would just point users to install the system dependencies to compile psycopg2 on their systems.

[fizyk]

An idea to solve that issue

• would be to add explanation to documentation/readme
• not require psycopg2 for sqlalchemy-redshift
• and add specific import error message within sqlalchemy-redshift

Something like I did ClearcodeHQ/pytest-postgresql#190

[gaslitbytech]

apt-get install -y build-essential libpq-dev python3-dev

Thank you. Adding the following to the top of my Dockerfile worked for me.

FROM python:3.7-slim
RUN apt-get update
RUN apt-get install -y build-essential libpq-dev python3-dev 

[jklukas]

As stated in #168 (comment), I've come to agree with basically with @fizyk's approach. I'm open to a PR that removes psycopg2 as a dependency and adds relevant documentation.