openssl-sys v0.9.87 fails to build in a CI

[aswinnnn]

I'm using this CI workflow file and this keeps happening, I have tried a number of solutions including OPENSSL_DIR and other things found in:

• Ubuntu 18: failed to run custom build command for openssl-sys v0.9.39  #1021

• https://stackoverflow.com/questions/65553557/why-rust-is-failing-to-build-command-for-openssl-sys-v0-9-60-even-after-local-in

Can't set up my CI till this is fixed, pretty unfortunate.

Here's the logs, entire thing over here :

📦 Including license file "/home/runner/work/pyscan/pyscan/LICENSE"
🔗 Found bin bindings
📡 Using build options bindings from pyproject.toml
   Compiling libc v0.2.144
   Compiling proc-macro2 v1.0.57
   Compiling autocfg v1.1.0
   Compiling quote v1.0.27
   Compiling unicode-ident v1.0.8
   Compiling cfg-if v1.0.0
   Compiling cc v1.0.79
   Compiling once_cell v1.18.0
   Compiling syn v2.0.16
   Compiling log v0.4.17
   Compiling pkg-config v0.3.27
   Compiling bitflags v1.3.2
   Compiling pin-project-lite v0.2.9
   Compiling openssl-sys v0.9.87
   Compiling futures-core v0.3.28
   Compiling bytes v1.4.0
   Compiling serde v1.0.163
   Compiling memchr v2.5.0
error: failed to run custom build command for `openssl-sys v0.9.87`

Caused by:
  process didn't exit successfully: `/home/runner/work/pyscan/pyscan/target/release/build/openssl-sys-d7abbfc9ddad199f/build-script-main` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR
  X86_64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
  OPENSSL_LIB_DIR unset
  cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_INCLUDE_DIR
  X86_64_UNKNOWN_LINUX_GNU_OPENSSL_INCLUDE_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
  OPENSSL_INCLUDE_DIR unset
  cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_DIR
  X86_64_UNKNOWN_LINUX_GNU_OPENSSL_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_DIR
  OPENSSL_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=OPENSSL_STATIC
  cargo:rerun-if-env-changed=OPENSSL_DYNAMIC
  cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
  cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  run pkg_config fail: `PKG_CONFIG_ALLOW_SYSTEM_CFLAGS="1" PKG_CONFIG_PATH="/usr/local/lib/pkgconfig" "pkg-config" "--libs" "--cflags" "openssl"` did not exit successfully: exit status: 1
  error: could not find system library 'openssl' required by the 'openssl-sys' crate

  --- stderr
  Package openssl was not found in the pkg-config search path.
  Perhaps you should add the directory containing `openssl.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'openssl' found


  --- stderr
  thread 'main' panicked at '

  Could not find directory of OpenSSL installation, and this `-sys` crate cannot
  proceed without this knowledge. If OpenSSL is installed and this crate had
  trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
  compilation process.

  Make sure you also have the development packages of openssl installed.
  For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.

  If you're in a situation where you think the directory *should* be found
  automatically, please open a bug at https://github.com/sfackler/rust-openssl
  and include information about your system as well as this message.

  $HOST = x86_64-unknown-linux-gnu
  $TARGET = x86_64-unknown-linux-gnu
  openssl-sys = 0.9.87

  ', /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssl-sys-0.9.87/build/find_normal.rs:190:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `"cargo" "rustc" "--target" "x86_64-unknown-linux-gnu" "--message-format" "json-render-diagnostics" "--manifest-path" "/home/runner/work/pyscan/pyscan/Cargo.toml" "--release" "--bin" "pyscan"`
Error: The process '/usr/bin/docker' failed with exit code 1
Error: The process '/usr/bin/docker' failed with exit code 1
    at ExecState._setResult (/home/runner/work/_actions/PyO3/maturin-action/v1/dist/index.js:1702:25)
    at ExecState.CheckComplete (/home/runner/work/_actions/PyO3/maturin-action/v1/dist/index.js:1685:18)
    at ChildProcess.<anonymous> (/home/runner/work/_actions/PyO3/maturin-action/v1/dist/index.js:1579:27)
    at ChildProcess.emit (node:events:527:28)
    at maybeClose (node:internal/child_process:1092:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)

[sfackler]

What openssl are you trying to point the build to? OPENSSL_DIR is unset in that error message.

[aswinnnn]

I am not sure? I'm following an auto-generated CI file by maturin. Any help on where openssl will be in the system, so I can set it and try again?

[sfackler]

It looks like your build is running inside of the manylinux container, so it would need to be installed there. I am not familiar with Python build infrastructure, but @alex may be able to help there.

[alex]

manylinux containers do not have an openssl installed in them, because they provide only the bare set of libraries that can be assumed on all linux installations (e.g., an old glibc). To resolve this, you need to bring your own openssl and statically link it (and therefore, update it whenever openssl does a security release).

https://github.com/pyca/cryptography/blob/main/.github/workflows/wheel-builder.yml#L49 is how we do it (containers are built elsewhere)

[aswinnnn]

that was helpful, appreciate it. The manylinux container is so complicated, it took me a while to figure out.