Installation from local path produces linking error

[kursatyurt]

I had installed Julia to /usr/bin/ and preCICE is installed at the location /usr/local/lib/

Following the instructions on the readme, I have installed bindings however the tests are not working as I try test PreCICE I get the following error message:

DUMMY: Running solver dummy with preCICE config file "../solverdummy/precice-config.xml", participant name "SolverOne", and mesh name "MeshOne" 
DUMMY: Running solver dummy with preCICE config file "../solverdummy/precice-config.xml", participant name "SolverTwo", and mesh name "MeshTwo" 
ERROR: ERROR: LoadError: LoadError: could not load library "libprecice"
/usr/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /usr/local/lib/libprecice.so)

It seems Julia only looks at the locally installed folder not looking for the entire system (LD_LIBRARY_PATH). Locally installed versions show the same behavior.

[erikscheurer]

This error occurs after running the solverdummies manually or when doing test PreCICE?

Can you run

$ julia
julia> using PreCICE

without errors?

As this error is similar to the matlab bindings, can you try running

$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 julia
julia> using PreCICE

or directly running the solver dummies with the LD_PRELOAD?

[kursatyurt]

This error occurs after running the solverdummies manually or when doing test PreCICE?

using test PreCICE

Can you run

$ julia
julia> using PreCICE

without errors?

Yes it works without any error.

As this error is similar to the matlab bindings, can you try running

$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 julia
julia> using PreCICE

or directly running the solver dummies with the LD_PRELOAD?

It solves the problem with libstdc++ but then I had another problem with libcurl, and solving libcurl trigger another library...

[erikscheurer]

You may want to try the workaround from here:

An intermediate solution is to replace Julias libstdc++.so.6 with the one from the OS:
works for Ubuntu 19.10 64bit - match your locations accordingly!
cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 $JULIA_HOME/lib/julia/

But this may also only lead to the result with preloading, I am not sure.

If the above fix does not work, you may have to build Julia from source.
For more info on why this occurs you may want to look at JuliaLang/julia#34276 (comment)

[kursatyurt]

You may want to try the workaround from here:

An intermediate solution is to replace Julias libstdc++.so.6 with the one from the OS:
works for Ubuntu 19.10 64bit - match your locations accordingly!
cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 $JULIA_HOME/lib/julia/

But this may also only lead to the result with preloading, I am not sure.

If the above fix does not work, you may have to build Julia from source. For more info on why this occurs you may want to look at JuliaLang/julia#34276 (comment)

Unfortunately, non of them help to solve problems. Preload works for libstdc++ but then arises some other issues about libcurl and preloading does not help.

Pre-compiled binary or building from source also does not make any difference.

BTW, Matlab bindings have a similar issue, but just after preloading libstdc++, it resolved.

[fsimonis]

Just tested this on Arch with julia 1.8.1

I was able to install it from a clone of the repo.
I had to set LD_LIBRARY_PATH to run the solverdummies though or the tests.

[erikscheurer]

I was now able to recreate it in an empty docker containing based on image ubuntu:jammy. I was able to fix it by compiling preCICE myself and also using LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 ./julia.

Maybe someone has an idea on the cause? My best guess is that you need the same version of glibcxx in julia that compiled preCICE and the precompiled preCICE package doesn't use the same one?

[fsimonis]

Can you run readelf -d /usr/lib/libprecice.so.2 | grep NEEDED after installing preCICE to see if the requested version of the stl corresponds with the system stl?

[erikscheurer]

Can you run readelf -d /usr/lib/libprecice.so.2 | grep NEEDED after installing preCICE

After the installation with the package I get

root@899f89aa1265:~/julia-1.8.1/bin# readelf -d /usr/lib/x86_64-linux-gnu/libprecice.so | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libboost_log_setup.so.1.74.0]
 0x0000000000000001 (NEEDED)             Shared library: [libboost_program_options.so.1.74.0]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libxml2.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libpython3.9.so.1.0]
 0x0000000000000001 (NEEDED)             Shared library: [libboost_log.so.1.74.0]
 0x0000000000000001 (NEEDED)             Shared library: [libboost_filesystem.so.1.74.0]
 0x0000000000000001 (NEEDED)             Shared library: [libboost_thread.so.1.74.0]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libmpi_cxx.so.40]
 0x0000000000000001 (NEEDED)             Shared library: [libmpi.so.40]
 0x0000000000000001 (NEEDED)             Shared library: [libpetsc_real.so.3.14]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]

And with compiling myself I get

root@899f89aa1265:~/julia-1.8.1/bin# readelf -d ~/precice/lib/libprecice.so.2 | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libboost_log_setup.so.1.74.0]
 0x0000000000000001 (NEEDED)             Shared library: [libboost_program_options.so.1.74.0]
 0x0000000000000001 (NEEDED)             Shared library: [libxml2.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libboost_log.so.1.74.0]
 0x0000000000000001 (NEEDED)             Shared library: [libboost_filesystem.so.1.74.0]
 0x0000000000000001 (NEEDED)             Shared library: [libboost_thread.so.1.74.0]
 0x0000000000000001 (NEEDED)             Shared library: [libmpi_cxx.so.40]
 0x0000000000000001 (NEEDED)             Shared library: [libmpi.so.40]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]

see if the requested version of the stl corresponds with the system stl

I don't know what the stl is or how to check it. If its the libstdc++.so.6 it only says .6 here, but I found, that the system comes with libstdc++.so.6.0.30 and julia (1.8.1) ships with libstdc++.so.6.0.29

[fsimonis]

Ah I see.

So, julia decides to ship its own libraries for some reason, leading to conflicting dependencies. This is currently a problem with everything in julia attempting to use system libraries.

It's essentially the same problem as we have/had with the Matlab bindings, which also ships its own dependencies.

I just saw that Arch patches julia to use the system libraries, explaining why it runs fine on my system.

I see two ways for us to deal with this long-term:

1. build a version of preCICE for the julia bindings, using the dependencies shipped with julia. This is messy and error-prone, but should work. We can most likely automate parts of the process. Note that participants can use different preCICE builds as long as the version is identical.
2. include a step in the documentation of the julia bindings to patch the build system of julia, essentially taking the same patch as Arch. This shouldn't be that much of an issue as Ubuntu users will most likely compile julia themselves as the versions in the repos are ancient.

I would recommend to try how much of a hassle a separate installation of preCICE is. Ideally, this will be a single shell script in the julia repo that builds preCICE. And a second shellscript that preloads the julian precice and runs julia. Not pretty, but usable.

[erikscheurer]

What I don't understand is why it doesn't just work when preloading the system libstdc++.so.6, this was what fixed it for the matlab bindings right? Is it just because julia also ships other libraries that also throw errors?
How would we patch the build of julia? (I don't think Ubuntu users would compile themselves, there are precompiled julia versions for download)

So you suggest that users have to build preCICE from source if preloading doesn't fix the error? Shouldn't we then just link to the preCICE documentation for building instead of creating our own script in the julia bindings?

Also, I think we should somehow have it that this error occurs when building and not during the first use right?
So maybe something like Libc.Libdl.dlopen("libprecice") in build.jl?

[fsimonis]

What I don't understand is why it doesn't just work when preloading the system libstdc++.so.6, this was what fixed it for the matlab bindings right? Is it just because julia also ships other libraries that also throw errors?

Preloading only works if the libraries are compatible. Preloading different versions is a minefield as doesn't impact parts outside of the shared library, think templates. It's very easy to get into ODR issues here. Thankfully the stl libs do catch such things with errors such as the one we ran into.

How would we patch the build of julia? (I don't think Ubuntu users would compile themselves, there are precompiled julia versions for download)

Good point. The recommended way of installation is to download the binaries, hence, we have a clear target to work with.

So you suggest that users have to build preCICE from source if preloading doesn't fix the error? Shouldn't we then just link to the preCICE documentation for building instead of creating our own script in the julia bindings?

The difficult part is getting all the dependencies right, forcing cmake to use the versions shipped with julia. This is a good candidate for a separate script.

Also, I think we should somehow have it that this error occurs when building and not during the first use right?
So maybe something like Libc.Libdl.dlopen("libprecice") in build.jl?

This should be covered by tests, which we have, right? We should probably tell the user to always run the tests after building the bindings.

[erikscheurer]

Preloading only works if the libraries are compatible.

But building preCICE from source and using the precompiled Julia version worked, why not use this combination instead of building preCICE with the julia library? Wouldn't this be annoying to existing users of preCICE that have an installation of preCICE but want to try Julia?

This should be covered by tests, which we have, right? We should probably tell the user to always run the tests after building the bindings.

Yes okay this should work.

[fsimonis]

But building preCICE from source and using the precompiled Julia version worked, why not use this combination instead of building preCICE with the julia library?

Problem is that it is not guaranteed to work.
A pragmatic approach to resolving this issue is to check if this approach works for our provided debian packages on Ubuntu versions after 20.04 LTS. If everything is fine, then we document the preloading solution prominently and call it a day.

[erikscheurer]

So since it doesn't work with the package on 22.04 and preloading I would still include the documentation for preloading but give compiling with the Julia library as an separate option?
Could you help me with setting up this extra script, I don't think I quite understand what you meant with the description of what it would do. Should it just replace preCICE's CMakeLists.txt or is there a nicer way of telling cmake to compile using this library e.g. through a flag where we can just make a script saying

mkdir build && cd build
cmake --some-flag ..
make -j $(nproc)
make install

or something similar?

[IshaanDesai]

Could you help me with setting up this extra script, I don't think I quite understand what you meant with the description of what it would do.

I think what @fsimonis means here is that we do a source installation of preCICE but instead letting CMake pick the default libraries from the default paths, we point it to pick the libraries from where Julia has them. Have a look at the dependencies section of the preCICE documentation. Here, for every dependency (like boost, PETSc, Eigen) there is a section where it is explained which paths need to be set if the dependency is installed from source. So for us the situation is that the dependencies are not installed from source but are in fact shipped by Julia, so we just set the paths to the appropriate locations. Does this make it clear?

[erikscheurer]

Yes I understand what we try to do, but I don't see how we can add the Julia Libraries to CMake. Or at least going beyond the LD_LIBRARY_PATH since when I try to use this, the error occurs already when configuring:

root@7582e74b6fb6:~/precice/build# export LD_LIBRARY_PATH=/root/julia-1.8.1/lib/julia:$LD_LIBRARY_PATH 
root@7582e74b6fb6:~/precice/build# cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=~/precice -DPRECICE_PETScMapping=OFF -DPRECICE_PythonActions=OFF ..
cmake: /root/julia-1.8.1/lib/julia/libcurl.so.4: version `CURL_OPENSSL_4' not found (required by cmake)
cmake: /root/julia-1.8.1/lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by cmake)

I tried around with the RPATH but couldnt get it to work.