New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing psycopg2-binary with Python:3.6.4-alpine doesn't work #684

Closed
barrachri opened this Issue Feb 28, 2018 · 12 comments

Comments

Projects
None yet
7 participants
@barrachri
Copy link

barrachri commented Feb 28, 2018

When you try to install the binary version of the lib you get the source version error.

$ docker run -it --rm python:3.6.4-alpine sh
$ pip install psycopg2-binary

Collecting psycopg2-binary
  Downloading psycopg2-binary-2.7.4.tar.gz (426kB)
    100% |████████████████████████████████| 430kB 52kB/s
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/psycopg2_binary.egg-info
    writing pip-egg-info/psycopg2_binary.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/psycopg2_binary.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/psycopg2_binary.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/psycopg2_binary.egg-info/SOURCES.txt'

    Error: pg_config executable not found.

    pg_config is required to build psycopg2 from source.  Please add the directory
    containing pg_config to the $PATH or specify the full executable path with the
    option:

        python setup.py build_ext --pg-config /path/to/pg_config build ...

    or with the pg_config option in 'setup.cfg'.

    If you prefer to avoid building psycopg2 from source, please install the PyPI
    'psycopg2-binary' package instead.

    For further information please check the 'doc/src/install.rst' file (also at
    <http://initd.org/psycopg/docs/install.html>).
@dvarrazzo

This comment has been minimized.

Copy link
Member

dvarrazzo commented Feb 28, 2018

Alpine Linux use musl libc and manylinux wheels don't support it: see e.g. pypa/manylinux#37. Not a psycopg problem but a lack of a standard for it, such as PEP 513 defininig manylinux1. Happy to support Alpine as soon as there will be a standard and tools for it.

@dvarrazzo dvarrazzo closed this Feb 28, 2018

@nourchawich

This comment has been minimized.

Copy link

nourchawich commented May 25, 2018

I found this solution which worked quite fine.

FROM python:3.6-alpine

RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev

pip install psycopg2-binary

Edit

The above compiles the package from source, so I would rather

FROM python:3.6-alpine
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev
pip install psycopg2

not to get confused.

@barrachri

This comment has been minimized.

Copy link

barrachri commented May 25, 2018

But I think you are pseudo-using the binary package and building from the source at the end.

@dvarrazzo

This comment has been minimized.

Copy link
Member

dvarrazzo commented May 25, 2018

@nourchawich your "solution" is about compiling psycopg from source, which is totally supported. Binary packages are not (for the reason described above).

@nourchawich

This comment has been minimized.

Copy link

nourchawich commented May 27, 2018

Thank for the clarification @barrachri and @dvarrazzo

I just noticed what was actually happening. All I wanted was to have psycopg2 (regardless whether binary or not) installed on an alpine based image, hence considering it a solution.

@cscetbon

This comment has been minimized.

Copy link

cscetbon commented Oct 12, 2018

@nourchawich after it's been built, I suppose we can get rid of those dev packages ?

@siaarzh

This comment has been minimized.

Copy link

siaarzh commented Oct 22, 2018

@nourchawich after it's been built, I suppose we can get rid of those dev packages ?

Yes, but make sure the apt add and apt del commands are in one layer. Otherwise you won't make the image any smaller.
E.g.:

RUN apk add --no-cache --virtual .build-deps \
    gcc \
    python3-dev \
    musl-dev \
    postgresql-dev \
    && pip install --no-cache-dir psycopg2 \
    && apk del --no-cache .build-deps
@cscetbon

This comment has been minimized.

Copy link

cscetbon commented Oct 22, 2018

Yeah I did, thanks !

@Cerebus

This comment has been minimized.

Copy link

Cerebus commented Jan 13, 2019

Since this is the top google hit for this problem, use a multi-stage build to minimize the final image size.

FROM python:3.6-alpine as base                                                                                                
                                                                                                                              
FROM base as builder                                                                                                          
                                                                                                                              
RUN mkdir /install                                                                                                            
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev                                                             
WORKDIR /install                                                                                                              
COPY requirements.txt /requirements.txt                                                                                       
RUN pip install --install-option="--prefix=/install" -r /requirements.txt                                                     
                                                                                                                              
FROM base                                                                                                                     
                                                                                                                              
COPY --from=builder /install /usr/local                                                                                       
COPY src /app                                                                                                         
RUN apk --no-cache add libpq                                                                                                  
WORKDIR /app
@hugobuddel

This comment has been minimized.

Copy link

hugobuddel commented Jan 14, 2019

This is still a bit unclear to me. The point of installing psycopg2-binary is that compilers like gcc etc. are not necessary right?

I'm getting the same message as the original poster. When trying to install psycopg2-binary, it complains about not finding gcc and suggests to, well, install psycopg2-binary (which is what I'm trying to do).

@hugobuddel

This comment has been minimized.

Copy link

hugobuddel commented Jan 14, 2019

I'm on Centos 7.4, and apparently I had to yum install postgresql-devel to make psycopg2-binary to work.

@Cerebus

This comment has been minimized.

Copy link

Cerebus commented Jan 14, 2019

@hugobuddel The problem is when pip decides it can't install the psycopg2-binary wheel it falls back on building from source, which naturally requires dev packages. On alpine that's because of muslc instead of glibc. I should think this shouldn't happen on CentOS but maybe there's another conflict.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment