OpenSSL in 1.6 runtime missing symbols

[directhex]

directhex@flame:~/Projects/dotnet$ nm -D libssl-1.4 | grep OPENSSL
0000000000000000 A OPENSSL_1.0.0
0000000000000000 A OPENSSL_1.0.1
0000000000000000 A OPENSSL_1.0.1d
0000000000000000 A OPENSSL_1.0.2
0000000000000000 A OPENSSL_1.0.2g
                 U OPENSSL_cleanse
                 U OPENSSL_DIR_end
                 U OPENSSL_DIR_read
directhex@flame:~/Projects/dotnet$ nm -D libssl-1.6 | grep OPENSSL
0000000000000000 A OPENSSL_1.0.2d
0000000000000000 A OPENSSL_1.0.2g
                 U OPENSSL_cleanse
                 U OPENSSL_DIR_end
                 U OPENSSL_DIR_read

Similar for libcrypto.

This means any binaries built against libssl with 1.0.1x symbols don't work on the 1.6 runtime, but do on the 1.4 runtime (even if the actual version built against was 1.0.2.x)

./shared/Microsoft.NETCore.App/1.0.1/System.Security.Cryptography.Native.so: /lib/libssl.so.1.0.0: version 'OPENSSL_1.0.1' not found (required by ./shared/Microsoft.NETCore.App/1.0.1/System.Security.Cryptography.Native.so)

[alexlarsson]

It seems like this yocto change just replaced the version script with a new one:
http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/recipes-connectivity/openssl?id=73a43fc15e0463c39baaadecab78fb3ef51b8cd0

Maybe there was an ABI break or something?

[alexlarsson]

So, this seems like a shitshow. It goes like this:
The openssl 1.0.2g upstream release changed the default ABI by removing the sslv2 symbols.
Debian when they imported this bumped soname to 1.0.2 and replaced the version script with a new one
that changed all symbol versions.
The version symbol change made it into yocto, but apparently not the soname change.
Additionally, it seems Ubuntu did not make a soname change, nor a version script change, yet still updated to 1.0.2g, so things built against ubuntu will not work on debian, etc.

Of course, with yocto having the 1.0.0 soname and the 1.0.2 ABI it will not run binaries built on either debian or ubuntu, which seem like the worst possible alternative. So, i'm going to bump the soname in our ssl builds and rebuild everything again.

[alexlarsson]

This seems like the debian soname bump patch:
https://anonscm.debian.org/viewvc/pkg-openssl/openssl/trunk/debian/patches/soname.patch?view=log&pathrev=747

[alexlarsson]

Filed a yocto bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=11396

[smcv]

So, this seems like a shitshow.

OpenSSL's idea of ABI management has historically been fairly disastrous, leading to distros working around it downstream. 1.1.0 seems a lot better in this respect.

A large part of the problem is that it had functions that were documented and hard-coded to use a specific version of SSL - SSLv2_method() will only ever use SSL v2, and SSLv3_method() will only ever use SSL v3, although confusingly, SSLv23_method() will use the best available SSL/TLS version (and these days it will not accept v2 by default). Historically, library users called SSLv3_method() as a way to not get SSL v2 because that version was considered insecure, but that meant they needed source code changes when SSL v3 was also discovered to be insecure...

The openssl 1.0.2g upstream release changed the default ABI by removing the sslv2 symbols.

If symbols are removed, then that's an incompatible change, and the SONAME needs to change. Setting the SONAME is normally meant to be upstream's job, but it seems upstream didn't do this, so Debian did it instead.

If those symbols are conditionally included in the library, then there is no correct SONAME that an upstream can use that will be right for everyone. I'd argue that's an ABI design flaw: a particular version of a library should have a single, known ABI.

The reason Ubuntu didn't bump the SONAME seems to be that, instead of doing the upstream-supported thing and dropping SSLv3 symbols altogether, they seem to have patched those functions to make them exist but fail (analogous to how GLib behaves if you call TLS functions but you don't have glib-networking). The origin of that change seems to be here:

+openssl (1.0.2d-0ubuntu2) xenial; urgency=medium
+
+  * debian/patches/no-sslv3.patch: Disable SSLv3 without using the
+    no-ssl3-method option, as that changes ABI and we don't want to break
+    compatibility with third party applications and applications built for
+    older versions of Ubuntu, especially for an LTS release.
+
+ -- Marc Deslauriers <marc.deslauriers@ubuntu.com>  Mon, 09 Nov 2015 17:37:38 -0500

If I'm understanding correctly, the best solution would be for that change to get upstreamed so everyone used it. The second-best solution is for an orthogonal SONAME bump to result in everyone consistently disabling the same things, which I think may have been what happened in 1.1.0.

Debian when they imported this bumped soname to 1.0.2 and replaced the version script with a new one that changed all symbol versions.

That sounds appropriate. The purpose of the version script is to make sure that if you somehow load libssl.so.1.0.0 and libssl.so.1.0.2 into the same address space (for instance if you link to both libcurl and libruby or something, at a time when libcurl still uses 1.0.0 but libruby has been relinked against 1.0.2), then they don't clash.

The fact that libssl < 1.1 has versioned symbols at all is a distro thing - OpenSSL upstream assume that an entire distro is compiled against a single version, but that's just not feasible to ensure at all times in something the size of Debian, where there might be a thousand packages that need recompiling, some of which fail to rebuild because of related or unrelated bugs. The versioned symbols seem to have finally gone upstream in 1.1 though.

I didn't know Yocto also did this. It's reasonable for Yocto to pull in changes from Debian, but if they do, then the Yocto package maintainer should make sure they understand what Debian is doing, and not set up a situation where they only get half of a group of changes that need to be all-or-nothing.

[alexlarsson]

latest builds use the debian 1.0.2 soname, not much more we can do about this.