Conflicts with libcurl when installed GMT.jl

[amarux]

Dear community,

I have recently installed Julia 1.8.3 on Ubuntu 22.04 by downloading the tarball from the official web page.
Previous to the installation, I removed the .julia/ directory, and the output of versioninfo() is:

Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × Intel(R) Core(TM) i3-5010U CPU @ 2.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, broadwell)
  Threads: 4 on 4 virtual cores
Environment:
  JULIA_NUM_THREADS = 4

I tried to install the package GMT.jl which is a wrapper to a set of tools for geophysics.
I used add GMT. However, installation of this particular package fails at precompiling, and the error I got was:

ERROR: InitError: could not load library "/lib/libgdal.so.30"
/opt/julia-1.8.5/bin/../lib/julia/libcurl.so: version `CURL_OPENSSL_4' not found (required by /lib/libgdal.so.30)
Stacktrace:
 [1] GDALVersionInfo(a1::String)
   @ GMT.Gdal ~/.julia/packages/GMT/7jXvi/src/gdal.jl:331
 [2] __init__()
   @ GMT.Gdal ~/.julia/packages/GMT/7jXvi/src/gdal.jl:2457
 [3] _include_from_serialized(pkg::Base.PkgId, path::String, depmods::Vector{Any})
   @ Base ./loading.jl:831
 [4] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt64)
   @ Base ./loading.jl:1039
 [5] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1315
 [6] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [7] macro expansion
   @ ./loading.jl:1180 [inlined]
 [8] macro expansion
   @ ./lock.jl:223 [inlined]
 [9] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
during initialization of module Gdal

It seems that there is an incompatibility with the libcurl used by Julia.
I think is important to mention that as a wrapper GMT.jl, depends on the original gmt code. This code can be previously installed on the system, or in its absence, add GMT install it with Conda.

The error with the Julia dependencies arises in the former case, when the original gmt source code is already on the system. In the other case, when gmt is forced to be installed with Conda, there is no problems.

However, the concerns is about GMT.jl or other packages can fail in the future by incompatibilities with the Julia dependencies as libcurl.

@GenericMappingTools/joa-quim, please help to clarify or add more related info about this issue.

Thank you

[joa-quim]

@amarux Thanks, you explained it well but it lacks a bit of info. This conflict started to happen recently in new Linux versions. For example there is no problem with Ubuntu 20.04 but there is with Ubunto 22.04 and ArchLinux that I know of.

[aviks]

So it looks like the gdal.so you are loading expects libcurl to be compiled with OpensSSL. The libcurl supplied with Julia not compiled with OpenSSL, rather it uses MbedTLS on linux, schannel on Windows and SecurTransport on on Mac (the latter two are the native SSL implementations that come with the OS)

This is not official advice, but try replacing the libcurl.so inside your julia installation with your OS provided libcurl, and see if that helps?

Other than that, I'm not sure what else will help here, short of recompiling GMT. I see that GMT does not have a jll, but GDAL does. Could you force GMT to use the gdal.so from the jll rather than the OS provided one it seems to be loading from /lib ?

[joa-quim]

Thanks @aviks for explaining the libcurl's panorama from Julia's view point. But the diagnostic is not promising because this situation arises when one have an installation (GMT in this case but it could be any another program) that relies on system's installed libcurl. Well, maybe system is not here the best word but I mean, that's what package managers install on the system. I don't know what changed in this regard on Ubuntu 22.04 and ArchLinux (and probably others) but the end result is this conflict.

I tried a variation of your suggestion that was to move the libcurl in Julia dir out of the way and then things worked (but don't know if when some Julia process needs curl if it will still work) but this ofc not a solution. But I also wonder why an external (to Julia) program runs it manages to find first the Julia provided lib instead OS provided one. Is it a question of paths? or LD_LIBRARY_DIR's?

Regarding the GDAL jll, I have no idea on how to do that, and specially with install instructions. But I was satisfied to find that now GDAL_JLL already supports netCDF and HDF which it didn't during long time and made it a heavy point against having a GMT_JLL (now the problem is probably still the Windows build).