OpenBSD detection for SSL configuration fails

[davidhewitt]

Copied from PyO3/pyo3#3807 (comment)

David Hewitt asked me to file this issue to see if we can make certain aspects of Python work better for OpenBSD users. Please let me know if there is other information I should provide.

For what it's worth, these are the patches involved in getting this particular package working on OpenBSD. https://cvsweb.openbsd.org/ports/security/py-cryptography/patches/

/tmp$ python3 -m venv foo /tmp$ . foo/bin/activate (foo) /tmp$ cd foo (foo) /tmp/foo$ uname -a OpenBSD mbp.sweet.home 7.4 GENERIC.MP#1663 amd64 (foo) /tmp/foo$ export RUST_BACKTRACE=1 (foo) /tmp/foo$ pip3 install --log ~/openbsd-pip-install-fail.txt cryptography Collecting cryptography Downloading cryptography-42.0.2.tar.gz (672 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 672.8/672.8 kB 5.0 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting cffi>=1.12 Using cached cffi-1.16.0-cp310-cp310-openbsd_7_4_amd64.whl Collecting pycparser Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB) Building wheels for collected packages: cryptography error: subprocess-exited-with-error

× Building wheel for cryptography (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. Building wheel for cryptography (pyproject.toml) ... error ERROR: Failed building wheel for cryptography Failed to build cryptography ERROR: Could not build wheels for cryptography, which is required to install pyproject.toml-based projects

openbsd-pip-install-fail.txt

[alex]

@holsta, does this still reproduce with cryptography 42.0.5? And do you have instructions that we can use to reproduce from a fresh openbsd?

[holsta]

@alex Thanks for following up! I get the following failure with 42.0.5 -- both when not and when setting OPENSSL_DIR="/usr". Do you need the openbsd-pip-install-fail.txt log? It's 3MB.

$ cd /tmp
/tmp $ python3 -m venv foo
/tmp $ uname -a
OpenBSD x395.holsta.dk 7.5 GENERIC.MP#16 amd64
/tmp $ . foo/bin/activate
/tmp $ cd foo/
/tmp/foo $ export RUST_BACKTRACE=1
/tmp/foo $ pip3 install --log openbsd-pip-install-fail.txt cryptography
Collecting cryptography
  Downloading cryptography-42.0.5.tar.gz (671 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 671.0/671.0 kB 1.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting cffi>=1.12
  Using cached cffi-1.16.0-cp310-cp310-openbsd_7_5_amd64.whl
Collecting pycparser
  Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Building wheels for collected packages: cryptography
  error: subprocess-exited-with-error
  
  × Building wheel for cryptography (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem wi.
  Building wheel for cryptography (pyproject.toml) ... error
  ERROR: Failed building wheel for cryptography
Failed to build cryptography
ERROR: Could not build wheels for cryptography, which is required to install pys

[alex]

Yeah, we'll definitely need that log, it's where all the info is :D

[holsta]

openbsd-pip-install-fail.txt

Hope this contains a clue.

[alex]

It does!

Good news: It's a new error, rust-openssl is correctly finding your OpenSSL. The error is that the version of LibreSSL you have isn't supported.
Good news: We have support for LibreSSL 3.9.x in main: #10565
Bad news: We don't yet have this fix in a release.

[holsta]

Oh, pinning against particular versions is bound to cause sporadic failures until the end of time. Is there a better mechanism we could look at? Testing features / functionality?

[alex]

The reason rust-openssl limits the versions of LibreSSL that are verified to work is that, to my knowledge, LibreSSL does not offer API or ABI stability between major versions. We'll look at backporting the rust-openssl fix to cryptography 42

…

On Mon, Apr 15, 2024 at 12:36 PM Alex Holst ***@***.***> wrote: Oh, pinning against particular versions is bound to cause sporadic failures until the end of time. Is there a better mechanism we could look at? Testing features / functionality? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

-- All that is necessary for evil to succeed is for good people to do nothing.

[botovq]

LibreSSL does not offer API or ABI stability between major versions.

Correct. This won't change.

I have been trying to reduce the friction for rust-openssl to a minimum by doing the necessary changes myself. If cryptography could merge support for the latest stable libressl into their latest stable branch at the earliest opportunity, that would be nice and should obviate the problem @holsta runs into. If there's anything I can or should do to help with that, let me know.

The py3-cryptography port of OpenBSD is usually up to date and it can be used to sidestep problems with pip. It needs no patches.

[alex]

Yes, I think the amount of work really was minimal, it's just a matter of us needing to do a release.

[alex]

Ok, 42.0.6 release that's going out shoudl fix this. Sorry we sat on this for so long.