Update to OpenSSL v3.3

[nibanks]

No description provided.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.81%. Comparing base (56ef97b) to head (68529c4).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4274      +/-   ##
==========================================
- Coverage   86.00%   85.81%   -0.20%     
==========================================
  Files          56       56              
  Lines       17354    17354              
==========================================
- Hits        14925    14892      -33     
- Misses       2429     2462      +33     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[nibanks]

Seems all the 24.04 builds are failing for some reason

[nibanks]

This PR is blocked on linking to the built-in system libcrypto for Ubuntu 24.04. Apparently, the built in version hasn't been updated to a recent enough version to support dynamic linking from libssl v3.3. cc @wfurt

[nibanks]

According to GitHub copilot:

The error in the logs indicates multiple "undefined reference" issues related to OpenSSL functions, such as BIO_get_wpoll_descriptor, ossl_time_now, and OSSL_STACK_OF_X509_free, among others. These unresolved references suggest that the linker is unable to find the necessary implementations for these functions, which are likely part of the OpenSSL library.

Potential Fixes:

1. Check OpenSSL Library Version: Ensure that the version of OpenSSL being linked against is correct. The functions referenced might belong to a specific version of OpenSSL (e.g., OpenSSL 3.x).
2. Library Path Configuration: Verify that the library paths for OpenSSL are correctly set up in the build configuration files (e.g., CMakeLists.txt). Ensure that the linker can locate the correct libraries.
3. Update Dependencies: Make sure all dependencies, including OpenSSL, are up to date and properly installed.
4. Rebuild OpenSSL: If the custom build of OpenSSL is being used, make sure it is correctly built and all necessary components are included.

[nibanks]

These errors are only occurring when we build against the latest v3.3 release of quictls/openssl but statically link against the distro's libcrytpo. It seems there are breaking changes around these APIs for these versions.

cc @wfurt for possible suggestions on a path forward

[wfurt]

I'll take a look next week. I assume this fails on the Ubuntu 24?

[nibanks]

I'll take a look next week. I assume this fails on the Ubuntu 24?

It fails on 22.04 and 24.04.

[wfurt]

I look at it @nibanks and it make sense. Here is what happened. It seems like OpenSSL 3.3 added some new functions and new libssl now depends on new functionality in lib crypto. That is pretty normal IMHO. But for msquic that is breaking change because the libssl part you link cannot find what it expects. To verify this I simply build your branch on Ubuntu 24.10 that has updated OpenSSL and everything works just like it used to:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 24.10"
NAME="Ubuntu"
VERSION_ID="24.10"
VERSION="24.10 (Oracular Oriole)"

 /tmp/pwsh/pwsh scripts/build.ps1 -Config Release -Arch x64 -Tls openssl3 -UseSystemOpenSSLCrypto -DisableTest -DisableTools -DisablePerf

...
...
...

Consolidate compiler generated dependencies of target msquic
gmake[2]: Leaving directory '/home/furt/github/msquic/build/linux/x64_openssl3'
/usr/bin/gmake  -f src/bin/CMakeFiles/msquic.dir/build.make src/bin/CMakeFiles/msquic.dir/build
gmake[2]: Entering directory '/home/furt/github/msquic/build/linux/x64_openssl3'
gmake[2]: Nothing to be done for 'src/bin/CMakeFiles/msquic.dir/build'.
gmake[2]: Leaving directory '/home/furt/github/msquic/build/linux/x64_openssl3'
[100%] Built target msquic
gmake[1]: Leaving directory '/home/furt/github/msquic/build/linux/x64_openssl3'
/usr/bin/cmake -E cmake_progress_start /home/furt/github/msquic/build/linux/x64_openssl3/CMakeFiles 0
[11/01/2024 11:14:27] Done.

So in general case you need lib crypto of matching or newer version. I think we just got lucky that not much changed or was added in 3.1 and 3.2.

The other way still seems to work. I took package build on/for 22.04 and the library loss just fine with Ubuntu 24.10 and libcrypto 3.3.

root@e483ba15cd61:/tmp# ldd /usr/lib/x86_64-linux-gnu/libmsquic.so.2
	linux-vdso.so.1 (0x00007ffe8c0b6000)
	libxdp.so.1 => /lib/x86_64-linux-gnu/libxdp.so.1 (0x00007f76c4c95000)
	libbpf.so.1 => /lib/x86_64-linux-gnu/libbpf.so.1 (0x00007f76c4c32000)
	libnl-3.so.200 => /lib/x86_64-linux-gnu/libnl-3.so.200 (0x00007f76c4c10000)
	libnl-route-3.so.200 => /lib/x86_64-linux-gnu/libnl-route-3.so.200 (0x00007f76c4b80000)
	libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007f76c4624000)
	libnuma.so.1 => /lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f76c4614000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f76c43f4000)
	libelf.so.1 => /lib/x86_64-linux-gnu/libelf.so.1 (0x00007f76c43d5000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f76c43b7000)
	libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007f76c42f9000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f76c4e32000)

there are (at least) two ways how to fix it:

1. is shimming the lib crypto and either implementing the missing functions or stubbing them out so linking works. While this is harder to maintain long term this gives most flexibility.

2. providing separate binaries for distributions with OpenSSL 3.3+ like Ubuntu 24.10. The easiest way would probably be to treat this is separate backend - it would require minimal build changes. At the end you would have yet another binary build and correct binaries would be published to proper distributions. I don't think you would need any msquic changes.

Please let me know if that makes sense. I think you can build this in container or updated VM image if you want to see perf numbers.

[nibanks]

@wfurt thanks for looking into this and explaining it so clearly. I think the fundamental problem on our side is that we assume Ubuntu 20.* (and earlier) map to openssl v1.* and that 22.* and 24.* map to openssl v3.*. This does not seem to be correct. It seems the different Ubuntu minor versions map to different openssl minor versions.

This seems to force us to either (a) drop support of previous minor versions of Ubuntu or (b) we need to support more than just 1 version of openssl v1.* and v3.* and maintain the mapping to the various Ubuntu versions out there. I don't see (a) as a practical option. And to support (b) there is going to be non-trivial work. We need to figure out a more dynamic way to build against the right openssl version than having a dedicated submodule for every version we support.

[wfurt]

I can probably look at it more this week. If there are only few missing functions some form of shim may be reasonable option. 1.2 is dead and 3.9-3.2 were close enough. I suspect her breaking changes will happen somewhat regularly but infrequently.