cibuildwheel fails on ubuntu aarch64

[RobertCraigie]

https://github.com/RobertCraigie/prisma-client-py/runs/4497941477?check_suite_focus=true

Is this architecture supported? Am I doing something wrong?

Relevant part of the actions workflow:

jobs:
  build-query-engine:
    name: Build Query Engine py3${{ matrix.python-version }} on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        python-version: ["6", "7", "8", "9", "10"]
        os: [ubuntu-20.04, windows-2019, macos-10.15]
    env:
      # TODO: run tests on the built wheels
      CIBW_BUILD: "cp3${{ matrix.python-version }}-*"
      CIBW_BUILD_VERBOSITY: 1
      CIBW_BEFORE_ALL_LINUX: "curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain stable -y && yum install -y openssl-devel"
      CIBW_BEFORE_BUILD_LINUX: "pip3 install --upgrade keyrings.alt"
      CIBW_ARCHS_LINUX: "auto aarch64"
      CIBW_ENVIRONMENT: 'PATH="$PATH:$HOME/.cargo/bin"'
      CIBW_SKIP: "cp27-* cp34-* cp35-* pp* *-win32"

    steps:
      - uses: actions/checkout@v2

      - uses: actions-rs/toolchain@v1
        with:
          profile: minimal
          toolchain: stable
          override: true

      - uses: actions/setup-python@v2
        with:
          python-version: "3.10"

      - name: Setup env
        run: |
          python -m pip install --upgrade wheel setuptools maturin

      - if: runner.os == 'Linux'
        name: Set up QEMU
        id: qemu
        uses: docker/setup-qemu-action@v1
        with:
          image: tonistiigi/binfmt:latest
          platforms: arm64

      - name: Install cibuildwheel
        run: |
          python -m pip install -U cibuildwheel==2.3.0
      - name: Build wheels
        working-directory: src/query-engine
        run: |
          python -m cibuildwheel --output-dir wheelhouse
      - uses: actions/upload-artifact@v2
        with:
          path: ./src/query-engine/wheelhouse/*.whl

pyproject.toml

[tool.black]
skip-string-normalization = true

# TODO: metadata
# https://maturin.rs/metadata.html

[build-system]
requires = ["maturin>=0.12,<0.13"]
build-backend = "maturin"

cargo.toml

[package]
name = "prisma-py-query-engine"
version = "0.0.1"
edition = "2018"
authors = ["Robert Craigie <robert@craigie.dev>"]

[lib]
name = "_prisma_query_engine"
path = "src/lib.rs"
crate-type = ["cdylib"]

[dependencies]
query-core = { git = "https://github.com/prisma/prisma-engines", rev = "33838b0" }
pyo3-asyncio = { git = "https://github.com/awestlake87/pyo3-asyncio", features = ["attributes", "tokio-runtime"] }
request-handlers = { git = "https://github.com/prisma/prisma-engines", rev = "33838b0" }
query-connector = { git = "https://github.com/prisma/prisma-engines", rev = "33838b0" }
user-facing-errors = { git = "https://github.com/prisma/prisma-engines", rev = "33838b0" }
datamodel = { git = "https://github.com/prisma/prisma-engines", rev = "33838b0" }
sql-connector = { git = "https://github.com/prisma/prisma-engines", package = "sql-query-connector", rev = "33838b0" }
prisma-models = { git = "https://github.com/prisma/prisma-engines", rev = "33838b0"  }
datamodel-connector = { git = "https://github.com/prisma/prisma-engines", rev = "33838b0" }
url = "2"
serde = "1"
connection-string = "0.1"
tokio = { version = "1", features = ["sync"] }
serde_json = { version = "1", features = ["preserve_order", "float_roundtrip"] }
tracing = "0.1"
tracing-futures = "0.2"
tracing-subscriber = "0.2"
tracing-opentelemetry = "0.11"
thiserror = "1.0"
opentelemetry = "0.12"
opentelemetry-otlp = { version = "0.5", features = ["tls", "tls-roots"] }
crossbeam-channel = "0.5.1"

[dependencies.pyo3]
version = "0.14.5"
features = ["extension-module"]

[messense]

💥 maturin failed
Caused by: Cargo metadata failed. Does your crate compile with cargo build?
Caused by: cargo metadata exited with an error: Updating crates.io index
Updating git repository https://github.com/prisma/prisma-engines
Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/pip-modern-metadata-lp697kh4', '--interpreter', '/opt/python/cp310-cp310/bin/python']' returned non-zero exit status 1.

Looks like network issue to me. Can you try if cargo build works in emulated aarch64 platform possibly using Docker and QEMU? See https://github.com/multiarch/qemu-user-static

[RobertCraigie]

Thank you, will have a look as soon as possible.

I have ran this build multiple times and have encountered the same error every time if that changes anything?

[messense]

pip install . works for me on my local manylinux_2_24 aarch64 docker container (I'm on M1 Mac):

root@779bdd341c3b:~/prisma-client-py/src/query-engine# /opt/python/cp39-cp39/bin/pip install -v .
Using pip 21.3.1 from /opt/_internal/cpython-3.9.9/lib/python3.9/site-packages/pip (python 3.9)
Processing /root/prisma-client-py/src/query-engine
  Running command /opt/_internal/cpython-3.9.9/bin/python /tmp/pip-standalone-pip-iillk1zb/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-j34wulcp/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'maturin>=0.12,<0.13'
  Collecting maturin<0.13,>=0.12
    Using cached maturin-0.12.4-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (8.6 MB)
  Collecting toml~=0.10.2
    Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
  Installing collected packages: toml, maturin
  Successfully installed maturin-0.12.4 toml-0.10.2
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  Installing build dependencies ... done
  Running command /opt/_internal/cpython-3.9.9/bin/python /opt/_internal/cpython-3.9.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpo92t_pmg
  Getting requirements to build wheel ... done
  Running command /opt/_internal/cpython-3.9.9/bin/python /opt/_internal/cpython-3.9.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpusi166sz
  🔗 Found pyo3 bindings
  🐍 Found CPython 3.9 at /opt/_internal/cpython-3.9.9/bin/python
  prisma_query_engine-0.0.1.dist-info
  Checking for Rust toolchain....
  Running `maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-671hrota --interpreter /opt/_internal/cpython-3.9.9/bin/python`
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: prisma-query-engine
  Running command /opt/_internal/cpython-3.9.9/bin/python /opt/_internal/cpython-3.9.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpdqa1uwcr
  Running `maturin pep517 build-wheel -i /opt/_internal/cpython-3.9.9/bin/python`
     Compiling openssl-sys v0.9.67
     Compiling crossbeam-utils v0.8.5
     Compiling matchers v0.0.1
     Compiling encoding v0.2.33
     Compiling anyhow v1.0.44
     Compiling crc32fast v1.2.1
     Compiling const_fn v0.4.8
     Compiling phf v0.8.0
     Compiling metrics v0.12.1
     Compiling pest_meta v2.1.3
     Compiling pyo3-build-config v0.14.5
     Compiling rustc_version v0.2.3
     Compiling rustversion v1.0.5
     Compiling maybe-uninit v2.0.0
     Compiling httparse v1.5.1
     Compiling http-body v0.4.4
     Compiling quote v1.0.10
     Compiling getrandom v0.2.3
     Compiling parking_lot_core v0.8.5
     Compiling num_cpus v1.13.0
     Compiling signal-hook-registry v1.4.0
     Compiling time v0.1.43
     Compiling cpufeatures v0.2.1
     Compiling socket2 v0.4.2
     Compiling getrandom v0.1.16
     Compiling parking_lot_core v0.7.2
     Compiling hostname v0.1.5
     Compiling atty v0.2.14
     Compiling hostname v0.3.1
     Compiling num-traits v0.2.14
     Compiling aho-corasick v0.7.18
     Compiling object v0.27.1
     Compiling which v3.1.1
     Compiling combine v3.8.1
     Compiling miniz_oxide v0.4.4
     Compiling crossbeam-utils v0.7.2
     Compiling mio v0.7.14
     Compiling standback v0.2.17
     Compiling os_info v3.0.7
     Compiling want v0.3.0
     Compiling tracing-log v0.1.2
     Compiling ring v0.16.20
     Compiling paste-impl v0.1.18
     Compiling futures-channel v0.3.17
     Compiling generic-array v0.14.4
     Compiling async-channel v1.6.1
     Compiling sized-chunks v0.3.1
     Compiling idna v0.2.3
     Compiling stringprep v0.1.2
     Compiling libz-sys v1.1.3
     Compiling lexical-core v0.7.6
     Compiling libsqlite3-sys v0.22.2
     Compiling openssl v0.10.36
     Compiling native-tls v0.2.8
     Compiling crossbeam-channel v0.5.1
     Compiling im v13.0.0
     Compiling pyo3 v0.14.5
     Compiling ahash v0.7.6
     Compiling rand_core v0.6.3
     Compiling syn v1.0.81
     Compiling parking_lot v0.11.2
     Compiling dashmap v4.0.2
     Compiling rand_core v0.5.1
     Compiling parking_lot v0.10.2
     Compiling colored v2.0.0
     Compiling resolv-conf v0.7.0
     Compiling regex v1.5.4
     Compiling num-integer v0.1.44
     Compiling prost-build v0.6.1
     Compiling crossbeam-queue v0.2.3
     Compiling backtrace v0.3.62
     Compiling paste v0.1.18
     Compiling async-std v1.10.0
     Compiling digest v0.9.0
     Compiling block-buffer v0.9.0
     Compiling crypto-mac v0.10.1
     Compiling crypto-mac v0.11.1
     Compiling webpki v0.21.4
     Compiling sct v0.6.1
     Compiling url v2.2.2
     Compiling flate2 v1.0.22
     Compiling lexical v5.2.2
     Compiling hashbrown v0.11.2
     Compiling rand_chacha v0.3.1
     Compiling rand_chacha v0.2.2
     Compiling num-bigint v0.3.3
     Compiling pem v0.8.3
     Compiling prisma-inflector v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling pest_generator v2.1.3
     Compiling darling_core v0.13.0
     Compiling pyo3-macros-backend v0.14.5
     Compiling serde_derive v1.0.130
     Compiling futures-macro v0.3.17
     Compiling tokio-macros v1.5.0
     Compiling thiserror-impl v1.0.30
     Compiling async-trait v0.1.51
     Compiling tracing-attributes v0.1.18
     Compiling pin-project-internal v1.0.8
     Compiling time-macros-impl v0.1.2
     Compiling async-stream-impl v0.2.1
     Compiling enumflags2_derive v0.7.0
     Compiling prost-derive v0.6.1
     Compiling enum-as-inner v0.3.3
     Compiling user-facing-error-macros v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling typed-builder v0.9.1
     Compiling derivative v2.2.0
     Compiling async-stream-impl v0.3.2
     Compiling indoc-impl v0.3.6
     Compiling prost-derive v0.7.0
     Compiling ghost v0.1.2
     Compiling inventory-impl v0.1.10
     Compiling ctor v0.1.21
     Compiling pyo3-asyncio-macros v0.14.0 (https://github.com/awestlake87/pyo3-asyncio#1f231dcd)
     Compiling sha2 v0.9.8
     Compiling md-5 v0.9.1
     Compiling sha-1 v0.9.8
     Compiling hmac v0.10.1
     Compiling hmac v0.11.0
     Compiling pbkdf2 v0.8.0
     Compiling rustls v0.19.1
     Compiling webpki-roots v0.21.1
     Compiling hashlink v0.7.0
     Compiling lru v0.6.6
     Compiling rand v0.8.4
     Compiling rand v0.7.3
     Compiling pest_derive v2.1.0
     Compiling futures-util v0.3.17
     Compiling darling_macro v0.13.0
     Compiling pyo3-macros v0.14.5
     Compiling tokio v1.12.0
     Compiling thiserror v1.0.30
     Compiling serde v1.0.130
     Compiling tracing v0.1.29
     Compiling time-macros v0.1.1
     Compiling async-stream v0.2.1
     Compiling enumflags2 v0.7.1
     Compiling pin-project v1.0.8
     Compiling prost v0.6.1
     Compiling async-stream v0.3.2
     Compiling indoc v0.3.6
     Compiling prost v0.7.0
     Compiling inventory v0.1.10
     Compiling rustls-native-certs v0.5.0
     Compiling postgres-protocol v0.6.1 (https://github.com/pimeys/rust-postgres?branch=pgbouncer-mode#8a61d469)
     Compiling twox-hash v1.6.1
     Compiling cuid v0.1.0 (https://github.com/prisma/cuid-rust#4ffb2e47)
     Compiling tempfile v3.2.0
     Compiling darling v0.13.0
     Compiling async-native-tls v0.3.3
     Compiling futures-executor v0.3.17
     Compiling asynchronous-codec v0.5.0
     Compiling indexmap v1.7.0
     Compiling chrono v0.4.19
     Compiling uuid v0.8.2
     Compiling bigdecimal v0.2.2
     Compiling rust_decimal v1.17.0
     Compiling serde_bytes v0.11.5
     Compiling tracing-serde v0.1.2
     Compiling tokio-util v0.6.8
     Compiling tokio-native-tls v0.3.0 (https://github.com/pimeys/tls?branch=vendored-openssl#6d0e6fc7)
     Compiling mobc v0.7.3
     Compiling tokio-rustls v0.22.0
     Compiling trust-dns-proto v0.20.3
     Compiling tokio-stream v0.1.7
     Compiling time v0.2.27
     Compiling tracing-futures v0.2.5
     Compiling prost-types v0.6.1
     Compiling serde_with_macros v1.5.1
     Compiling futures v0.3.17
     Compiling serde_json v1.0.68
     Compiling graphql-parser v0.3.0 (https://github.com/prisma/graphql-parser#6a3f58bd)
     Compiling rusqlite v0.25.3
     Compiling h2 v0.3.7
     Compiling tower v0.4.10
     Compiling petgraph v0.5.1
     Compiling trust-dns-resolver v0.20.3
     Compiling tiberius v0.6.4
     Compiling opentelemetry v0.12.0
     Compiling postgres-types v0.2.1 (https://github.com/pimeys/rust-postgres?branch=pgbouncer-mode#8a61d469)
     Compiling mysql_common v0.26.0
     Compiling native-types v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling bson v2.0.1
     Compiling tracing-subscriber v0.2.25
     Compiling pyo3-asyncio v0.14.0 (https://github.com/awestlake87/pyo3-asyncio#1f231dcd)
     Compiling serde_with v1.11.0
     Compiling tokio-postgres v0.7.1 (https://github.com/pimeys/rust-postgres?branch=pgbouncer-mode#8a61d469)
     Compiling hyper v0.14.14
     Compiling mysql_async v0.27.0 (https://github.com/prisma/mysql_async?branch=vendored-openssl#a38f0ace)
     Compiling tracing-opentelemetry v0.11.0
     Compiling mongodb v2.0.1
     Compiling tonic-build v0.3.1
     Compiling postgres-native-tls v0.5.0 (https://github.com/pimeys/rust-postgres?branch=pgbouncer-mode#8a61d469)
     Compiling opentelemetry-otlp v0.5.0
     Compiling quaint v0.2.0-alpha.13 (https://github.com/prisma/quaint#e077df3d)
     Compiling tonic v0.4.3
     Compiling prisma-value v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling user-facing-errors v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling dml v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling datamodel-connector v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling mongodb-datamodel-connector v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling sql-datamodel-connector v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling datamodel v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling prisma-models v0.0.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling query-connector v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling mongodb-query-connector v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling sql-query-connector v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling query-core v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling request-handlers v0.1.0 (https://github.com/prisma/prisma-engines?rev=33838b0#33838b0f)
     Compiling prisma-py-query-engine v0.0.1 (/root/prisma-client-py/src/query-engine)
      Finished release [optimized] target(s) in 3m 38s
  🔗 Found pyo3 bindings
  🐍 Found CPython 3.9 at /opt/_internal/cpython-3.9.9/bin/python
  warning: field is never read: `datasource_overrides`
    --> src/engine.rs:45:5
     |
  45 |     datasource_overrides: Vec<(String, String)>,
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     |
     = note: `#[warn(dead_code)]` on by default


  warning: field is never read: `guard`
    --> src/logger.rs:34:5
     |
  34 |     guard: Option<Arc<Uninstall>>,
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


  warning: 2 warnings emitted


  ⚠️  Warning: No compatible platform tag found, using the linux tag instead. You won't be able to upload those wheels to PyPI.
  📖 Found type stub file at _prisma_query_engine.pyi
  📦 Built wheel for CPython 3.9 to /root/prisma-client-py/src/query-engine/target/wheels/prisma_query_engine-0.0.1-cp39-cp39-linux_aarch64.whl
  /root/prisma-client-py/src/query-engine/target/wheels/prisma_query_engine-0.0.1-cp39-cp39-linux_aarch64.whl
  Building wheel for prisma-query-engine (pyproject.toml) ... done
  Created wheel for prisma-query-engine: filename=prisma_query_engine-0.0.1-cp39-cp39-linux_aarch64.whl size=12544276 sha256=c1cefbc72f417457305d05f80e2d50434d5620fa8ec46a56f78ba03ed64dd134
  Stored in directory: /root/.cache/pip/wheels/59/13/57/0643c17986278eaa2afc3991eb0d148564cc92bacfe57e04b9
Successfully built prisma-query-engine
Installing collected packages: prisma-query-engine
Successfully installed prisma-query-engine-0.0.1

[RobertCraigie]

Could this be a cibuildwheel issue then?