Install using pip requires rust #8105

kolaente · 2021-02-08T10:49:20Z

Description of the issue

Installing docker-compose with pip as per the instructions from the docs fails with

  error: Can not find Rust compiler
  ----------------------------------------
  ERROR: Failed building wheel for cryptography

It looks like the cryptography dependency (now?) requires rust.

Of course it would not be a problem to install the rust compiler but then it should at least be mentioned in the docs. On the other hand I don't think this is intentional, therefore the issue.

Context information (for bug reports)

Output of docker version

Client: Docker Engine - Community
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        48a66213fe
 Built:             Mon Jun 22 15:45:36 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.12
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       48a66213fe
  Built:            Mon Jun 22 15:44:07 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Steps to reproduce the issue

Dockerfile to reproduce:

FROM docker:19-dind

RUN apk add py-pip python3-dev libffi-dev openssl-dev gcc libc-dev make curl && \
  pip install docker-compose

Observed result

Building the image fails with the mentioned error message.

Expected result

Building the image works without issues.

Stacktrace / full error message

https://gist.github.com/kolaente/d725fe1e4d4dbae6dadd1c649bec4a74

Additional information

Installing in a docker image

The text was updated successfully, but these errors were encountered:

kolaente · 2021-02-08T10:51:35Z

Looks like they require cargo now: https://github.com/pyca/cryptography/blob/master/docs/installation.rst

As a non-python developer it seems a bit odd that docker-compose always uses the latest dependencies and they are not pinned? Could be the way to do things with pip, just seems a bit odd to me.

aiordache · 2021-02-08T15:43:21Z

Hi @kolaente ! In docker-compose, we can only pin dependencies we can install with pip (Python packages). We cannot manage the installation of indirect dependencies which are not Python packages.

But I see the problem is in the docs, we need to update them and add rust and cargo to dependencies.
Thanks for opening the issue!

JoelMiller002 · 2021-02-08T18:31:05Z

I ran into the same issue and upgrading my pip resolved.

They included some steps in the docs if that does not resolve.
https://github.com/pyca/cryptography/blob/master/docs/installation.rst

shiplu · 2021-02-09T11:44:05Z

I had the same issue. I just added this export line before I install docker-compose with pip.

export CRYPTOGRAPHY_DONT_BUILD_RUST=1

This is already suggested in the build failure message. This worked for me as I only build and test my project with docker-compose in alpine Linux. I don't run with docker-compose, I do that with docker run in the production environment.

simao-silva · 2021-02-09T13:22:30Z

The way I manage to bypass the error was installing setuptools_rust.

pip3 install setuptools_rust docker-compose

anairamzap-mobomo · 2021-02-09T14:26:33Z

Hi, in our case we are using docker-compose in alpine based docker image for our CI/CD pipelines. What solved this dependencies issue was:

Update the docker image (to run alpine 13.3).
Add the new required dependencies (rust and cargo)
Update pip (to 21.0) and
Finally use pip to install docker-compose

Adding here a link to the issue on cryptography project: pyca/cryptography#5771

shiplu · 2021-02-09T15:18:53Z

@anairamzap-mobomo I ended up doing the same as you.

Apparently, CRYPTOGRAPHY_DONT_BUILD_RUST variable will not work in the next release!.

kolaente · 2021-02-09T18:21:45Z

I was able to fix it by installing the dependencies for alpine in the container before installing docker-compose.

KDGundermann · 2021-02-10T15:18:10Z

using the documented dependencies for alpine (now with rust)
https://docs.docker.com/compose/install/#install-compose-on-linux-systems
our CI build still fails:

docker build -t registry.intern.xxxx.de/xxx/publisher:264d30d9 -t registry.intern.xxxx.de/xxx/publisher:0.6.2c -t registry.intern.xxxxx.de/xxx/publisher:latest .
Step 1/9 : FROM docker:19.03.15
19.03.15: Pulling from library/docker
31ed15b4f9e6: Pull complete
Digest: sha256:3b7414686e1bdb833f8ee24400b94e7c591a4e8605721b13db6791cacc5b0dd8
Status: Downloaded newer image for docker:19.03.15
 ---> 934a8cac43d2
Step 2/9 : RUN apk add --no-cache py-pip python-dev libffi-dev openssl-dev gcc libc-dev rust cargo make  ruby ca-certificates
 ---> Running in 23dcc89702e2
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
ERROR: unable to select packages:
  python-dev (no such package):
    required by: world[python-dev]
The command '/bin/sh -c apk add --no-cache py-pip python-dev libffi-dev openssl-dev gcc libc-dev rust cargo make  ruby ca-certificates' returned a non-zero code: 1

`

cbuffett · 2021-02-10T17:35:27Z

using the documented dependencies for alpine (now with rust)
https://docs.docker.com/compose/install/#install-compose-on-linux-systems
our CI build still fails:

docker build -t registry.intern.xxxx.de/xxx/publisher:264d30d9 -t registry.intern.xxxx.de/xxx/publisher:0.6.2c -t registry.intern.xxxxx.de/xxx/publisher:latest .
Step 1/9 : FROM docker:19.03.15
19.03.15: Pulling from library/docker
31ed15b4f9e6: Pull complete
Digest: sha256:3b7414686e1bdb833f8ee24400b94e7c591a4e8605721b13db6791cacc5b0dd8
Status: Downloaded newer image for docker:19.03.15
 ---> 934a8cac43d2
Step 2/9 : RUN apk add --no-cache py-pip python-dev libffi-dev openssl-dev gcc libc-dev rust cargo make  ruby ca-certificates
 ---> Running in 23dcc89702e2
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
ERROR: unable to select packages:
  python-dev (no such package):
    required by: world[python-dev]
The command '/bin/sh -c apk add --no-cache py-pip python-dev libffi-dev openssl-dev gcc libc-dev rust cargo make  ruby ca-certificates' returned a non-zero code: 1

`

Newer versions of Alpine no longer support python-dev. You need to explicitly indicate the version you want (python2-dev or python3-dev)

https://pkgs.alpinelinux.org/packages?name=python*-dev&branch=v3.13

heckler · 2021-02-10T23:11:44Z

FWIIW ... On Ubuntu18.4 the solutions mentioned above of pre=installing setuptools_rust, updating pip, or exporting CRYPTOGRAPHY_DONT_BUILD_RUST=1 did not work at all.

What ended-up doing the trick is pinning the version of cryptography at 3.3.2; in case it helps anyone:

echo "cryptography==3.3.2" > /tmp/requirements.txt
sudo pip3 install -U docker-compose -r /tmp/requirements.txt
rm /tmp/requirements.txt

giner · 2021-02-12T02:53:10Z

Upgrading pip3 helps as well:

pip3 install "pip>=20"

You may want to use user's home or venv based installation though so that the original pip provided by the deb package is not replaced

Fix: docker/compose#8105

docker/compose#8105 (comment)

Newer versions of the pip package "cryptography" make use of the rust compiler when being built [1], so we will need the cargo package in order for this to be successful. This issue is primarily appearant in 32-bit ARM devices, which do not have any pre-compiled wheels they can download, so it will have to build it from source. [1]: docker/compose#8105 (comment)

tomekit · 2021-04-15T15:37:34Z

I am using: docker:latest image and after adding this: ENV CRYPTOGRAPHY_DONT_BUILD_RUST=1 just before the RUN pip install docker-compose in my Dockerfile things are working again.

cryptography now requires rust to compile: docker/compose#8105

Relates to docker/compose#8105

- requires setuptools_rust - make - openssl-dev Relates to docker/compose#8105

pennyloafers · 2023-01-29T18:09:13Z

Running into this issue with Rasbian Bullseye, I can overcome the issue if I install docker-compose without sudo, but ran into other issues after.

sillmnvg · 2023-02-01T05:22:12Z

Also had the same issue on Rasbian with a recent fresh lite install, ended up installing rust to get around the issue (not ideal at all, but for some homelab Pis easy works:

curl https://sh.rustup.rs -sSf | sh

Edit: Due to an unrelated issue (Loki doesn't work on 32 bit OS) I reinstalled with the 64bit version of rasbian lite, and did not encounter this issue.

FerdinaKusumah · 2024-09-23T06:40:52Z

Fix using this one

# Stage 1: Build stage
FROM python:3.11-slim as compile-image

## Virtualenv setup
ENV VIRTUAL_ENV=/opt/venv
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
ENV CRYPTOGRAPHY_DONT_BUILD_RUST=1

# Install necessary system dependencies for building
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    python3-dev \
    build-essential \
    libffi-dev \
    libpq-dev \
    gcc \
    libssl-dev && \
    pip install --upgrade pip && \
    pip install cryptography==3.3.2

kolaente added the kind/bug label Feb 8, 2021

t83714 mentioned this issue Feb 9, 2021

CI issue magda-io/magda#3074

Closed

aiordache linked a pull request Feb 12, 2021 that will close this issue

[Security] Bump cryptography from 3.2.1 to 3.3.2 #8109

Merged

rachmadaniHaryono mentioned this issue Feb 15, 2021

Docker build stuck on cryptography dependency jarun/buku#501

Closed

aiordache removed a link to a pull request Feb 15, 2021

[Security] Bump cryptography from 3.2.1 to 3.3.2 #8109

Merged

ulyssessouza mentioned this issue Feb 15, 2021

Update dependencies on install docker/docs#12334

Merged

usha-mandya closed this as completed in docker/docs#12334 Feb 16, 2021

longwand added a commit to longwand/topimmo-engine that referenced this issue Feb 27, 2021

Update Dockerfile

313e0a0

Fix: docker/compose#8105

longwand mentioned this issue Feb 27, 2021

Update Dockerfile aichner/topimmo-engine#2

Merged

6 tasks

nqb mentioned this issue Mar 1, 2021

Installation of docker-compose failed lefilament/ansible_role_docker_server#4

Closed

petermetz mentioned this issue Mar 1, 2021

fix(tools): fabric all-in-one build cannot find rust compiler hyperledger-cacti/cacti#617

Closed

northwestwitch mentioned this issue Mar 4, 2021

New release failed to create an image to upload to Docker Hub Clinical-Genomics/scout#2419

Closed

mumrau mentioned this issue Mar 11, 2021

[DOCKER] Fix failing docker build on arm due to cryptography trying to build rust healthchecks/healthchecks#492

Merged

Shylock-Hg mentioned this issue Mar 23, 2021

Fix the docker-compose installation. vesoft-inc/nebula-cpp#22

Merged

tombola pushed a commit to torchbox/rca-wagtail-2019 that referenced this issue Mar 29, 2021

Pin cryptography to avoid rust dependency

a6ac16d

docker/compose#8105 (comment)

kaito3desuyo mentioned this issue Apr 8, 2021

Update Docker to 20.10 and Docker Compose 1.28.x tmaier/docker-compose#29

Merged

kiarn mentioned this issue May 13, 2021

Raspbian 10 "buster", network plugin is not working ajenti/ajenti#1325

Open

chiangf added a commit to wevo-conversion/docker-alpine-node-14-awscli that referenced this issue Aug 4, 2021

Fix docker build issues

9daba1f

cryptography now requires rust to compile: docker/compose#8105

papanito added a commit to papanito/atlantis-docker-image that referenced this issue Sep 2, 2022

fix: build requires setuptools_rust

1ae7f5c

Relates to docker/compose#8105

papanito added a commit to papanito/atlantis-docker-image that referenced this issue Sep 2, 2022

fix: add missing tools

e203fd8

- requires setuptools_rust - make - openssl-dev Relates to docker/compose#8105

papanito added a commit to papanito/atlantis-docker-image that referenced this issue Sep 2, 2022

fix: add missing tools

8e0e9f3

- requires setuptools_rust - make - openssl-dev Relates to docker/compose#8105

This was referenced Feb 2, 2023

Fix CI issues in mailcow role PrivacyLx/privacylx-issue-tracker#100

Closed

error: can't find Rust compiler #9834

Closed

pwwang mentioned this issue Jun 26, 2023

[BUG] Unable to install datar in Python 3.11 & Windows 10 due to ERROR: Failed building wheel for rtoml pwwang/datar#180

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Install using pip requires rust #8105

Install using pip requires rust #8105

kolaente commented Feb 8, 2021 •

edited

Loading

kolaente commented Feb 8, 2021

aiordache commented Feb 8, 2021

JoelMiller002 commented Feb 8, 2021

shiplu commented Feb 9, 2021

simao-silva commented Feb 9, 2021 •

edited

Loading

anairamzap-mobomo commented Feb 9, 2021

shiplu commented Feb 9, 2021

kolaente commented Feb 9, 2021 •

edited

Loading

KDGundermann commented Feb 10, 2021

cbuffett commented Feb 10, 2021 •

edited

Loading

heckler commented Feb 10, 2021

giner commented Feb 12, 2021 •

edited

Loading

tomekit commented Apr 15, 2021

pennyloafers commented Jan 29, 2023

sillmnvg commented Feb 1, 2023 •

edited

Loading

FerdinaKusumah commented Sep 23, 2024 •

edited

Loading

Install using pip requires rust #8105

Install using pip requires rust #8105

Comments

kolaente commented Feb 8, 2021 • edited Loading

Description of the issue

Context information (for bug reports)

Steps to reproduce the issue

Observed result

Expected result

Stacktrace / full error message

Additional information

kolaente commented Feb 8, 2021

aiordache commented Feb 8, 2021

JoelMiller002 commented Feb 8, 2021

shiplu commented Feb 9, 2021

simao-silva commented Feb 9, 2021 • edited Loading

anairamzap-mobomo commented Feb 9, 2021

shiplu commented Feb 9, 2021

kolaente commented Feb 9, 2021 • edited Loading

KDGundermann commented Feb 10, 2021

cbuffett commented Feb 10, 2021 • edited Loading

heckler commented Feb 10, 2021

giner commented Feb 12, 2021 • edited Loading

tomekit commented Apr 15, 2021

pennyloafers commented Jan 29, 2023

sillmnvg commented Feb 1, 2023 • edited Loading

FerdinaKusumah commented Sep 23, 2024 • edited Loading

kolaente commented Feb 8, 2021 •

edited

Loading

simao-silva commented Feb 9, 2021 •

edited

Loading

kolaente commented Feb 9, 2021 •

edited

Loading

cbuffett commented Feb 10, 2021 •

edited

Loading

giner commented Feb 12, 2021 •

edited

Loading

sillmnvg commented Feb 1, 2023 •

edited

Loading

FerdinaKusumah commented Sep 23, 2024 •

edited

Loading