during build: LoadError: could not load library "libjulia"

[marabelotti]

I just tried to add Oscar but it fails building GAP.jl:

(@v1.4) pkg> status
Status `~/.julia/environments/v1.4/Project.toml`
  [f1435218] Oscar v0.5.0

(@v1.4) pkg> add GAP
  Resolving package versions...
   Updating `~/.julia/environments/v1.4/Project.toml`
  [c863536a] + GAP v0.4.4
   Updating `~/.julia/environments/v1.4/Manifest.toml`
 [no changes]

(@v1.4) pkg> build --verbose GAP
   Building GAP → `~/.julia/packages/GAP/7w7Vl/deps/build.log`
┌ Info: Compiling GAP with
│   gap_src_root = "/home/rara/.julia/artifacts/848f0f318405266896d24d11e4baf5cf7bf3f203/gap-4.11.0"
│   extra_gap_root = "/home/rara/.julia/packages/GAP/7w7Vl/"
└   gap_bin_root = "/home/rara/.julia/packages/GAP/7w7Vl/deps/build/gap-4.11.0-julia-1.4"
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for gcc option to accept ISO C99... none needed
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking ABI flags... 
checking how to run the C preprocessor... gcc  -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking size of void *... 8
checking ABI bit size... 64
checking for a BSD-compatible install... /usr/bin/install -c
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc ... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking for gawk... no
checking for mawk... mawk
checking command to parse /usr/bin/nm -B output from gcc  object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... mt
checking if mt is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc  supports -fno-rtti -fno-exceptions... no
checking for gcc  option to produce PIC... -fPIC -DPIC
checking if gcc  PIC flag -fPIC -DPIC works... yes
checking if gcc  static flag -static works... yes
checking if gcc  supports -c -o file.o... yes
checking if gcc  supports -c -o file.o... (cached) yes
checking whether the gcc  linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for shl_load... no
checking for shl_load in -ldld... no
checking for dlopen... no
checking for dlopen in -ldl... yes
checking whether a program can dlopen itself... yes
checking whether a statically linked program can dlopen itself... no
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking how to run the C++ preprocessor... g++  -E
checking for ld used by g++ ... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++  linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++  option to produce PIC... -fPIC -DPIC
checking if g++  PIC flag -fPIC -DPIC works... yes
checking if g++  static flag -static works... yes
checking if g++  supports -c -o file.o... yes
checking if g++  supports -c -o file.o... (cached) yes
checking whether the g++  linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether byte ordering is bigendian... no
checking for __attribute__((always_inline))... yes
checking for __attribute__((format))... yes
checking for __attribute__((noinline))... yes
checking for __attribute__((noreturn))... yes
checking for __builtin_mul_overflow... no
checking for __builtin_clz... yes
checking for __builtin_clzl... yes
checking for __builtin_clzll... yes
checking for __builtin_popcountl... yes
checking size of int... 4
checking size of long... 8
checking size of long long... 8
checking whether to enable HPC-GAP... no
checking which garbage collector to use... julia
checking whether to enable debug mode... no
checking whether to enable maintainer-mode mode... no
checking whether to enable memory checking... no
checking whether to enable valgrind extensions to GASMAN... no
checking whether to treat C compiler warnings as errors... no
checking whether C compiler accepts -Werror=unknown-warning-option... no
checking whether C compiler accepts -Wno-suggest-attribute=format... yes
checking whether C compiler accepts -fno-strict-aliasing... yes
checking whether C compiler accepts -Wall... yes
checking whether C compiler accepts -Wextra... yes
checking whether C compiler accepts -Warray-bounds... yes
checking whether C compiler accepts -Wno-cast-function-type... yes
checking whether C compiler accepts -Wdouble-promotion... yes
checking whether C compiler accepts -Wduplicated-branches... yes
checking whether C compiler accepts -Wduplicated-cond... yes
checking whether C compiler accepts -Wformat-nonliteral... yes
checking whether C compiler accepts -Wformat-security... yes
checking whether C compiler accepts -Wformat=2... yes
checking whether C compiler accepts -Winit-self... yes
checking whether C compiler accepts -Wno-inline... yes
checking whether C compiler accepts -Wlogical-op... yes
checking whether C compiler accepts -Wmissing-field-initializers... yes
checking whether C compiler accepts -Wmissing-format-attribute... yes
checking whether C compiler accepts -Wmissing-include-dirs... yes
checking whether C compiler accepts -Wmissing-variable-declarations... no
checking whether C compiler accepts -Wno-pragmas... yes
checking whether C compiler accepts -Wpacked... yes
checking whether C compiler accepts -Wpointer-arith... yes
checking whether C compiler accepts -Wredundant-decls... yes
checking whether C compiler accepts -Wredundant-decls... (cached) yes
checking whether C compiler accepts -Wrestrict... yes
checking whether C compiler accepts -Wreturn-type... yes
checking whether C compiler accepts -Wshadow... yes
checking whether C compiler accepts -Wno-sign-compare... yes
checking whether C compiler accepts -Wstrict-aliasing... yes
checking whether C compiler accepts -Wundef... yes
checking whether C compiler accepts -Wunused-but-set-variable... yes
checking whether C compiler accepts -Wno-unused-parameter... yes
checking whether C compiler accepts -Wwrite-strings... yes
checking whether C compiler accepts -Wno-implicit-fallthrough... yes
checking whether C compiler accepts -Wimplicit-function-declaration... yes
checking whether C compiler accepts -Wjump-misses-init... yes
checking whether C compiler accepts -Wnested-externs... yes
checking whether C compiler accepts -Wold-style-definition... yes
checking whether C compiler accepts -Wstrict-prototypes... yes
checking whether C++ compiler accepts -Wextra-semi... yes
checking whether C compiler accepts -Wno-error=cast-function-type... yes
checking whether C compiler accepts -Wno-error=inline... yes
checking whether C compiler accepts -Wno-error=pragmas... yes
checking whether C compiler accepts -Wno-error=sign-compare... yes
checking whether C compiler accepts -Wno-error=unused-parameter... yes
checking whether C compiler accepts -Wno-error=implicit-fallthrough... yes
checking whether to try and use __builtin_popcountl... yes
checking gmp.h usability... yes
checking gmp.h presence... yes
checking for gmp.h... yes
checking for __gmpz_init in -lgmp... yes
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking for inflateEnd in -lz... yes
checking for library containing rl_bind_keyseq... -lreadline
checking readline/readline.h usability... yes
checking readline/readline.h presence... yes
checking for readline/readline.h... yes
checking for julia... /usr/bin/julia
checking for JULIA_CPPFLAGS...  -I/usr/include/julia -fPIC
checking for JULIA_LDFLAGS... ERROR: LoadError: could not load library "libjulia"
libjulia.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at /build/julia-98cBbp/julia-1.4.1+dfsg/usr/share/julia/stdlib/v1.4/Libdl/src/Libdl.jl:109
 [2] dlopen at /build/julia-98cBbp/julia-1.4.1+dfsg/usr/share/julia/stdlib/v1.4/Libdl/src/Libdl.jl:109 [inlined] (repeats 2 times)
 [3] dlpath(::String) at /build/julia-98cBbp/julia-1.4.1+dfsg/usr/share/julia/stdlib/v1.4/Libdl/src/Libdl.jl:232
 [4] libDir() at /usr/share/julia/julia-config.jl:32
 [5] ldflags(::Bool) at /usr/share/julia/julia-config.jl:52
 [6] main() at /usr/share/julia/julia-config.jl:125
 [7] top-level scope at /usr/share/julia/julia-config.jl:136
 [8] include(::Module, ::String) at ./Base.jl:377
 [9] exec_options(::Base.JLOptions) at ./client.jl:288
 [10] _start() at ./client.jl:484
in expression starting at /usr/share/julia/julia-config.jl:136
configure: error: failed to obtain JULIA_LDFLAGS from julia-config.jl
ERROR: LoadError: failed process: Process(`/home/rara/.julia/artifacts/848f0f318405266896d24d11e4baf5cf7bf3f203/gap-4.11.0/configure --with-gc=julia --with-julia=/usr/bin --with-gmp=/home/rara/.julia/packages/GAP/7w7Vl/deps/usr --with-readline=/home/rara/.julia/packages/GAP/7w7Vl/deps/usr --with-zlib=/home/rara/.julia/packages/GAP/7w7Vl/deps/usr --disable-maintainer-mode 'LIBS=-Wl,-rpath -Wl,/home/rara/.julia/packages/GAP/7w7Vl/deps/usr/lib' ARCHEXT=v1.4`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at ./process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at ./process.jl:440
 [3] run at ./process.jl:438 [inlined]
 [4] (::var"#3#4")() at /home/rara/.julia/packages/GAP/7w7Vl/deps/build.jl:56
 [5] cd(::var"#3#4", ::String) at ./file.jl:104
 [6] top-level scope at /home/rara/.julia/packages/GAP/7w7Vl/deps/build.jl:53
 [7] include(::String) at ./client.jl:439
 [8] top-level scope at none:5
in expression starting at /home/rara/.julia/packages/GAP/7w7Vl/deps/build.jl:53
┌ Error: Error building `GAP`: 
└ @ Pkg.Operations /build/julia-98cBbp/julia-1.4.1+dfsg/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:892

[marabelotti]

I just checked and the error is gone on julia-1.5

[fingolfin]

Huh, that's strange. I assume this is on Linux, which distro? How was julia installed? Is there /usr/lib/libjulia.so ?

Thing is, we are invoking julia-config.jl (normally bundled with Julia) to query the compiler flags we need. And this script is producing the error.

[marabelotti]

Yes it is Ubuntu 20.04.1 and I installed julia using the packet manager.
The file is here: /usr/bin/../lib/x86_64-linux-gnu/libjulia.so.1

[fingolfin]

Ahh, of course. <rant>So most likely this is Debian and/or Ubuntu once again messing with a package "because they know best" and w/o involving upstream, and then breaking it in the process.</rant> sigh of course I know the intentions are good, but this has happened far too often to me double sigh.

This really seems like a bug in the Debian package. Not much we can do, other than trying to come up with some crazy Debian specific hack to resolve it, I am afraid. Which we might have to do in the end because a crazy number of people will use that distro sigh

Best would be of course if Debian and/or Ubuntu (not sure who is responsible) fixed this; that is:

julia $(julia -e 'print(joinpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "julia-config.jl"))') --cflags
julia $(julia -e 'print(joinpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "julia-config.jl"))') --ldflags
julia $(julia -e 'print(joinpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "julia-config.jl"))') --ldlibs

all ought to return valid results, with correct & existing paths.

UPDATE: nope, Debian is fine, the problem seems to be that we require libjulia-devel not just libjulia

[ThomasBreuer]

I have also Ubuntu 20.04 on my machine, and it has a julia 1.4.1 installed via apt install julia.
(I think I have never used this julia installation.)
The relevant calls from GAP's configure seem to be

/usr/bin/julia --startup-file=no /usr/share/julia/julia-config.jl --cflags
/usr/bin/julia --startup-file=no /usr/share/julia/julia-config.jl --ldflags

Both work on my machine, but in the above error report only the first of them works.

[benlorenz]

Is there only libjulia.so.1 or is there alibjulia.so symlink as well? If not, maybe the libjulia-dev package needs to be installed to link against libjulia.

[fingolfin]

@benlorenz good point. @ThomasBreuer tells me he has libjulia-devel installed @marabelotti can you verify?

If that's it, I take back all complaints against Debian and redirect them at julia-config.jl ;-) resp. ourselves... Good thing is this will be resolved once we can use GAP_jll

[fingolfin]

@marabelotti can you confirm that installing the system package libjulia-devel resolves the issue?

[marabelotti]

Sorry for the late reply! I've just installed the libjulia-dev package and yes everything works!

[fingolfin]

OK, then I think this can be closed for now. Thank you again for the report and getting back to us!