Detect when included into AppImage; root CA and engines path

[asashnov]

This is a proposal to discuss a new feature.

OpenSSL should detect whether it is run from AppImage (Linux app bundle, http://appimage.org/) to use an appropriate location for root CA certificates and crypto engines.

When a user downloads some AppImage, for example, Moolticute:
https://github.com/mooltipass/moolticute/releases/download/v0.21.0/Moolticute-x86_64.AppImage
they run it as follow:

~/Downloads $ chmod +x Moolticute-x86_64.AppImage
~/Downloads $ ./Moolticute-x86_64.AppImage

AppImage format is an executable header and compressed filesystem image which will be mounted to a temporary dir. If run with strace you may see following (strace if embedded info launcher script in my case):

$ grep ssl moolticute.strace.log 
13524 open("/tmp/.mount_MooltiH7uOfu/usr/bin/../lib/libssl.so.1.0.2k", O_RDONLY|O_CLOEXEC) = 16
13524 open("/usr/lib/ssl/openssl.cnf", O_RDONLY) = 16
13524 open("/etc/ssl/certs/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 16
13524 stat("/etc/ssl/certs/f060240e.0", {st_mode=S_IFREG|0644, st_size=1298, ...}) = 0
13565 open("/tmp/.mount_MooltiH7uOfu/usr/bin/../lib/libssl.so.1.0.2k", O_RDONLY|O_CLOEXEC) = 13
13565 open("/usr/lib/ssl/openssl.cnf", O_RDONLY) = 13
13565 open("/etc/ssl/certs/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 13
13565 stat("/etc/ssl/certs/f060240e.0", {st_mode=S_IFREG|0644, st_size=1298, ...}) = 0
14334 stat("/etc/ssl/certs//244b5494.0",  <unfinished ...>
14334 open("/etc/ssl/certs//244b5494.0", O_RDONLY) = 33
14334 stat("/etc/ssl/certs//244b5494.1", 0x7fa97a144010) = -1 ENOENT (No such file or directory)

Luckily enough that libssl.so.1.0.2k was borowed from Ubuntu 16.04 and was run on the same distribution, it was able to find root CA certificates. But the location is different across distributions. The similar problem was attemted to be solved for GNU TLS by patch:

https://github.com/darealshinji/vlc-AppImage/issues/1#issuecomment-321041496

So I'm going to fork stable version of OpenSSL 1_0 (branch OpenSSL_1_0_2-stable) and try to create a similar patch.

I little worried about the config is still looked into /usr/lib/ssl/openssl.cnf
And patch to crypto engines is also hardcoded (seems they are not used for SSL connection, but still):

/tmp/.mount_MooltiH7uOfu/usr/lib$ strings libcrypto.so.1.0.2k | grep /engines
/usr/lib/x86_64-linux-gnu/openssl-1.0.0/engines

So as tier-1 I will create a searching for root CA in different locations.

Tier-2 will use APPDIR env var to search related to mount point
(APPDIR=/tmp/.mount_MooltiH7uOfu)

I understand this goes against security considerations but if it can be at least a compile-time option in the upstream that would be great.

[probonopd]

While there is definitely an issue (1.1 breaks 1.0) or two (about every distro places certs in different places) to be solved, I don't think that this is really something AppImage specific, and hence the solution should be a general solution and not AppImage specific either.

References:

• https://media.ccc.de/v/ASG2018-174-2018_desktop_linux_platform_issues
• https://gitlab.com/probono/platformissues#openssl
• https://gitlab.com/probono/platformissues#certificates
• AppImages don't work on ArchLinux with OpenSSL issue subsurface/subsurface#779
• Pre Qt 5.10 app cannot find OpenSSL 1.0 on Arch Linux probonopd/linuxdeployqt#209
• 'std::runtime_error' ... Read-only file system??? probonopd/linuxdeployqt#221
• strange SSL behavior within the AppImage probonopd/linuxdeployqt#110

Non-AppImage examples:

• Linux Cross-Platform Packaging zealdocs/zeal-packaging#34 (comment)
• Qt 5.10 with OpenSSL 1.1 mxe/mxe#2014
• https://jira.atlassian.com/browse/HCPUB-6249?page=com.atlassian.streams.streams-jira-plugin%3Aactivity-stream-issue-tab
• https://forum.emteria.com/discussion/372/problem-with-openssl-compatibility

[probonopd]

This issue is not AppImage specific.

The problem occurs whenever an application needs to ship a private copy of OpenSSL because the application cannot assume that the version of OpenSSL works with the application. For example, if the application is built against OpenSSL 1.0 then it will fail on distributions that ship OpenSSL 1.1, and vice versa, effectively forcing the application to bundle a private copy of OpenSSL.

Now, the private copy of OpenSSL must either ship its own set of certificates (which would be very cumbersome to maintain), or load the certificates from the system. Unfortunately, the different distributions happen to put them in apparently "random" locations (for no apparent good reason other than "distribution policy"):

"/etc/ssl/certs/ca-certificates.crt",     // Debian/Ubuntu/Gentoo etc.
"/etc/pki/tls/certs/ca-bundle.crt",       // Fedora/RHEL
"/etc/ssl/ca-bundle.pem",                 // OpenSUSE
"/etc/pki/tls/cacert.pem",                // OpenELEC
"/etc/ssl/certs",                         // SLES10/SLES11, https://golang.org/issue/12139
"/usr/share/ca-certs/.prebuilt-store/"    // Clear Linux OS; https://github.com/knapsu/plex-media-player-appimage/issues/17#issuecomment-437710032
"/system/etc/security/cacerts"            // Android

Hence, OpenSSL should fall back to loading certificates from all those known locations, and issue a clear recommendation for distributions where to put certificates going forward.

References:

• https://gitlab.com/probono/platformissues/blob/master/README.md#certificates
• https://www.happyassassin.net/2015/01/12/a-note-about-ssltls-trusted-certificate-stores-and-platforms/
• Patch by @darealshinji https://github.com/darealshinji/vlc-AppImage/issues/1#issuecomment-321041496

[richsalz]

No, OpenSSL should not load all certs from all locations. That would be a very poor security practice -- anyone who can install things in one of the locations makes all programs that use OpenSSL trust all of the CA's.

Instead, each application should have its own trust store, or path to CA's. Yes it is a bit harder on everyone, and it would be nice if the various distro's and platforms could unify, but oh well.

I think this should be closed.

[probonopd]

Thanks for joining the discussion @richsalz. What if all locations are within /etc, /usr, and /system - in other words, locations that are only writable by root?

How would you make an end-user desktop application that runs on all distributions, if you don't want to have to care for the certificates yourself?

Or maybe we could make it respect a certain environment variable to set a path to the certificates?

[richsalz]

It's a tough problem, and one I do not think OpenSSL should solve, even if it could solve it.

If I were writing such an application, I would have an ordered list of directories to try, and I'd use the first one that existed as a directory, and had the right ownership and permissions. Yes, ugh.

[probonopd]

https://github.com/Ultimaker/cura-build/blob/28d93cc221b3fea0d91ceef670830c086f6a07de/packaging/cura.sh#L12

According to https://www.microfocus.com/documentation/visual-cobol/VC40/CSWin/HROSSROPEN02.html, OpenSSL configuration files can define

The location of your certificate files.

[ihnorton]

It may be worth noting that Go ships a hard-coded list of only /etc/ paths:

https://github.com/golang/go/blob/master/src/crypto/x509/root_linux.go

As far as I can tell, there's no permission verification, they just try each path until success: 1, 2.

I can understand the disinterest in hard-coding such paths in OpenSSL, but from a systemic perspective it might be useful to do so at the heart of the stack, so to speak, as this could reduce the number of applications shipping their own cert bundles. Perhaps as a default-off build option. Many small, infrequently-updated applications end up shipping certs as a path of least resistance to multi-distro support. (even Python and the JVM both ship their own).

[probonopd]

It may be worth noting that Go ships a hard-coded list of only /etc/ paths:

There's even more locations:

https://gitlab.com/probono/platformissues/blob/master/README.md#certificates

I think one default location should be defined "at the heart of the stack", to increase standardization. The others should be searched as fallback locations.

[Chaz6]

There ought to be a distribution agnostic way to fetch the CTL location. My view is that this could be made part of the XDG specification.

[probonopd]

Can SSL_CERT_FILE and/or SSL_CERT_DIR be used to point to the location of the certificates?

According to the documentation,

Alternatively the SSL_CERT_DIR environment variable can be defined to override this location. The default CA certificates file is called cert.pem in the default OpenSSL directory. Alternatively the SSL_CERT_FILE environment variable can be defined to override this location.

In this case, the AppRun script in the AppImage would need to check all possible locations where the different distributions tend to place those files, and then export the one where they have actually be found, prior to launching the payload executable.

Do you think this would be a good strategy? cc @TheAssassin

[probonopd]

Please see FreeCAD/FreeCAD-Bundle#34 (comment).

[levitte]

A possibility would be to be able to specify paths in the config file. Openssl would then only need to know one location, that of the config file.

[probonopd]

Is there a way to find out, at runtime, from the OpenSSL that is on the system, the path to the certs?

[levitte]

Yes

const char *X509_get_default_private_dir(void);
const char *X509_get_default_cert_area(void);
const char *X509_get_default_cert_dir(void);
const char *X509_get_default_cert_file(void);

If the directory configured with --openssldir is /PATH/TO/OPENSSL, the three functions above will return the strings "/PATH/TO/OPENSSL/private", "/PATH/TO/OPENSSL", "/PATH/TO/OPENSSL/certs" and "/PATH/TO/OPENSSL/cert.pem", respectively.

Furthermore, the following three functions will return the names of the corresponding environment variables that libcrypto try to look at first, before using the above functions:

const char *X509_get_default_cert_dir_env(void);
const char *X509_get_default_cert_file_env(void):

Unfortunately, this is among the lot of under-documented libcrypto functions.

[probonopd]

Thanks @levitte. Do you happen to know a way to find this out without needing a C/C++ program (e.g., from a shell script)?

[bubnikv]

At least for C / C++ applications, it sounds like it may make sense to create an auxiliary library for distro independent binaries, which would

1. Detect the distro
2. Mapped the distro name and version to the path of the certificates.

I suppose such an auxiliary library could fulfill many purposes, I bet many candidates are already mentioned here https://gitlab.com/probono/platformissues/blob/master/README.md