Crash on Fedora 33 due to OpenSSL/Kerberos incompatibility

[renatobellotti]

3D Slicer crashes with the following error message:

Slicer-4.11.20210226-linux-amd64/bin/SlicerApp-real: symbol lookup error: /lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b

Steps to reproduce

Download the latest binary and run it on Fedora.

Expected behavior

The program should start.

Environment

• Slicer version: Slicer-4.11.20210226-linux-amd64
• Operating system: Fedora 33

[lassoan]

How did you start the application?

[renatobellotti]

From the command line: ./Slicer

[lassoan]

Thanks for the information. The launcher sets up the lib paths then.

@RafaelPalomar do you have any suggestion?

[RafaelPalomar]

@lassoan, @renatobellotti It looks a compatibility issue between Kerberos and OpenSSL. The same issue has been found in other software packages (https://forum.ansys.com/discussion/24790/has-anyone-seen-lib64-libk5crypto-so-3-undefined-symbol-evp-kdf-ctrl-version-openssl-1-1-1b)

I'm checking if building Slicer in a system with the same version of Kerberos would detect and adapt OpenSSL to be compatible. Another thing to try could be disabling Kerberos support for OpenSSL in Slicer and see if that solves the problem and does not causes any issue. It might be even good to disable those components we find are not in use by Slicer. I'll get back to you.

[renatobellotti]

Thanks for your diligence! :)

[RafaelPalomar]

I can confirm is a compatibility issue with the OpenSSL shipped with 3D Slicer (v1.1.1g) and the Kerberos version in Fedora 33 (compatible with OpenSSL 1.1.1k).

Older OpenSSL versions had the no-krb5 flag to build without Kerberos, but I can't find that option anymore. So disabling that seems not to be an option.

Bumping the version of OpenSSL to a more recent one is feasible, but it might break compatibility with other Linux distributions. I ignore if there is any plan for moving that (@lassoan, @jcfr, @pieper)?

@renatobellotti you could build Slicer in Fedora 33 on your own, but it is not the most friendly environment for building Slicer. I have managed it by using:

• -DSlicer_USE_SimpleITK=OFF #SimpleITK does not build properlly in Fedora 33
• -DSlicer_USE_SYSTEM_OpenSSL=ON #Using the OpenSSL in Fedora 33 builds ok
• -DSlicer_USE_SYSTEM_LibArchive=ON #GCC11 is not suitable to compile LibArchive at the moment

[lassoan]

If this problem cannot be properly solved then maybe we could choose specific linux distributions that we make Slicer compatible with; and let users use docker or build Slicer from source on others.

[renatobellotti]

Docker sounds good to me! :)

[pieper]

This is also a problem with a centos8 system I am using. The institution discourages docker and I haven't seen that singularity can support a virtual desktop. My workaround is just not to use Slicer on that machine but it is limiting.

[jcfr]

Possible approaches:

1. do not bundle OpenSSL in Linux packages
2. build OpenSSL without Kerberos
3. update OpenSSL

@jcfr currently looking at approach (1)

[jcfr]

OpenSSL and Kerberos

The OpenSSL libraries shipped (1.1.1g) within the latest stable release (4.11.20210226) are not build against Kerberos:

$ pwd
/work/Stable/Slicer-1-build/Slicer-build/_CPack_Packages/linux-amd64/TGZ/Slicer-4.11.20210226-linux-amd64

$ ./Slicer --launch ldd lib/Slicer-4.11/libssl.so
	linux-vdso.so.1 =>  (0x00007ffd77b60000)
	libcrypto.so.1.1 => /work/Stable/Slicer-1-build/Slicer-build/_CPack_Packages/linux-amd64/TGZ/Slicer-4.11.20210226-linux-amd64/bin/../lib/Slicer-4.11/libcrypto.so.1.1 (0x00007f3559a08000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f35597fc000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f35595df000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f3559211000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f355a19d000)

$ ./Slicer --launch ldd lib/Slicer-4.11/libcrypto.so
	linux-vdso.so.1 =>  (0x00007ffd31d83000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f459a782000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f459a566000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f459a197000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f459ae90000)

Libraries depending on libk5crypto.so

To better understand which libraries introduced the dependency, running the following command is helpful:

cd /path/to/Slicer-4.11.20210226-linux-amd64
for lib in $(find . | grep "\.so$"); do (./Slicer --launch ldd $lib | grep libk5crypto 2>&1> /dev/null) && echo $lib; done

When executed on metroplex using the docker image building the release, it returns the following (truncated for convenience):

./lib/Slicer-4.11/CTK[...]PythonQt.so
./lib/Slicer-4.11/libCTK[...].so
./lib/Slicer-4.11/libq[...].so
./lib/Slicer-4.11/qt-loadable-modules/libqSlicer[...].so
./lib/Slicer-4.11/qt-loadable-modules/libvtkSlicer[...].so
./lib/Slicer-4.11/qt-loadable-modules/qSlicer[...]PythonQt.so
./lib/Slicer-4.11/libQt5Multimedia.so
./lib/Slicer-4.11/libQt5Network.so
./lib/Slicer-4.11/libQt5XmlPatterns.so
./lib/Slicer-4.11/libQt5WebEngine.so
./lib/Slicer-4.11/libQt5WebEngineWidgets.so
./lib/Slicer-4.11/libQt5WebChannel.so
./lib/Slicer-4.11/libQt5Qml.so
./lib/Slicer-4.11/libQt5Quick.so
./lib/Slicer-4.11/libQt5QuickWidgets.so
./lib/Slicer-4.11/libQt5WebEngineCore.so
./lib/Slicer-4.11/libQt5QmlModels.so
./lib/Slicer-4.11/libPythonQt.so
./lib/QtPlugins/designer/libCTK[...]WidgetsPlugins.so
./lib/QtPlugins/designer/libq[...]WidgetsPlugins.so
./lib/QtPlugins/designer/libqwebengineview.so
./lib/QtPlugins/iconengines/libqSlicerIconEnginePlugin.so
./lib/QtPlugins/styles/libqSlicerBaseQTGUIStylePlugins.so
./lib/QtPlugins/audio/libqtaudio_alsa.so
./lib/QtPlugins/audio/libqtmedia_pulse.so

Conclusion

The "problem" is that Qt5 libraries have a hard dependency on libk5crypto and since we do not ship this library along side Slicer, this means that Slicer ends up always loading the OpenSSL library found on the system instead of the one provided by Slicer.

Path forward

On Linux, we could require OpenSSL to be installed on the system.

[pieper]

After talking it over with @jcfr @RafaelPalomar and @lassoan on the developer hangout, I tried building kerberos and openssl on the centos machine and it actually worked.

The current master branch of kerberos worked using the developer build.

I also build the OpenSSL_1_1_1-stable tag of openssl.

Then by setting these paths I am able to run Slicer:

export LD_LIBRARY_PATH=/space/alta/1/users/pieper/slicer/openssl:/space/alta/1/users/pieper/slicer/krb5/src/lib

This may not be a great long-term solution but it's an okay workaround for now.