Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't find solib dependencies imported from system #142

Closed
mboes opened this issue Feb 18, 2018 · 9 comments · Fixed by #144
Closed

Can't find solib dependencies imported from system #142

mboes opened this issue Feb 18, 2018 · 9 comments · Fixed by #144
Assignees
@mrkkrp
Labels
P1 critical: next release type: bug

Comments

@mboes
Copy link
Member

mboes commented Feb 18, 2018

Minimal repro:

$ cd inline-java
# Edit workspace file to point to v0.3 rules_haskell release. 
$ vim WORKSPACE
$ bazel build -s //...
[...]
SUBCOMMAND: # @io_tweag_inline_java//jvm:jvm [action 'Compiling jvm']
(cd /home/mboes/.cache/bazel/_bazel_mboes/6cf48c46fc22333e7461cb52051cf09a/execroot/io_tweag_sparkle && \
  exec env - \
    LD_LIBRARY_PATH=bazel-out/k8-fastbuild/bin/external/io_tweag_inline_java/jni \
    PATH=external/sparkle-toolchain/nix/bin:/home/mboes/.nix-profile/bin:/nix/store/nj7mw0lyx80g7lrxcxy5v9vmnpif5950-gcc-wrapper-6.4.0/bin:/nix/store/nj7mw0lyx80g7lrxcxy5v9vmnpif5950-gcc-wrapper-6.4.0/bin:/home/mboes/.nix-profile/bin:/home/mboes/.nix-profile/bin:/home/mboes/.nix-profile/bin:/nix/store/nj7mw0lyx80g7lrxcxy5v9vmnpif5950-gcc-wrapper-6.4.0/bin:/home/mboes/.nix-profile/bin \
  external/sparkle-toolchain/nix/bin/ghc -c --make -fPIC -hide-all-packages -odir bazel-out/k8-fastbuild/bin/external/io_tweag_inline_java/jvm/objects-jvm-1.0.0 -hidir bazel-out/k8-fastbuild/bin/external/io_tweag_inline_java/jvm/interfaces-jvm-1.0.0 -iexternal/io_tweag_inline_java/jvm/src/Language -package base -package bytestring -package constraints -package choice -package distributed-closure -package exceptions -package singletons -package text -package vector -package jni-1.0.0 -package-db bazel-out/k8-fastbuild/bin/external/io_tweag_inline_java/jni/jni-1.0.0 external/io_tweag_inline_java/jvm/src/Language/Java.hs -package-name jvm-1.0.0 -static -dynamic-too -this-unit-id jvm-1.0.0)
ERROR: /home/mboes/.cache/bazel/_bazel_mboes/6cf48c46fc22333e7461cb52051cf09a/external/io_tweag_inline_java/jvm/BUILD:8:1: Compiling jvm failed (Exit 1)
[1 of 1] Compiling Language.Java    ( external/io_tweag_inline_java/jvm/src/Language/Java.hs, bazel-out/k8-fastbuild/bin/external/io_tweag_inline_java/jvm/objects-jvm-1.0.0/Language/Java.o )
<command line>: can't load .so/.DLL for: bazel-out/k8-fastbuild/bin/external/io_tweag_inline_java/jni/libHSjni-1.0.0-ghc8.2.2.so (libjvm.so: cannot open shared object file: No such file or directory)
INFO: Elapsed time: 1.002s, Critical Path: 0.62s
FAILED: Build did NOT complete successfully

A mangled symlink exists in the folder. But for system imported libraries, the mangling is not useful, because the names of installed system libraries should be globally unique anyways.

I suspect, but haven't checked, that #117 was the cause of this regression.
@mboes mboes added P1 critical: next release type: bug labels Feb 18, 2018
@mrkkrp
Copy link
Member

mrkkrp commented Feb 19, 2018

I investigated this. Initially I was quite confused by the fast that libjvm.so is mentioned at all. Indeed, after mangling we have a mangled symbolic link pointing to lbjvm.so, but the point of mangling is to change the original name of .so library we link to. It looked alarming that despite the fact that linking of libHSjni-1.0.0-ghc8.2.2.so is done against jni_Sopenjdk_libjvm.so:

SUBCOMMAND: # //jni:jni [action 'Linking dynamic library libHSjni-1.0.0-ghc8.2.2.so']
(cd /root/.cache/bazel/_bazel_root/587be86fe6ecbef03a79b0f6efc9d3d8/execroot/io_tweag_inline_java && \
  exec env - \
    PATH=external/inline-java-toolchain/nix/bin:/root/.nix-profile/bin:/nix/store/nj7mw0lyx80g7lrxcxy5v9vmnpif5950-gcc-wrapper-6.4.0/bin:/nix/store/nj7mw0lyx80g7lrxcxy5v9vmnpif5950-gcc-wrapper-6.4.0/bin:/root/.nix-profile/bin:/root/.nix-profile/bin:/root/.nix-profile/bin:/nix/store/nj7mw0lyx80g7lrxcxy5v9vmnpif5950-gcc-wrapper-6.4.0/bin:/root/.nix-profile/bin \
  external/inline-java-toolchain/nix/bin/ghc -shared -dynamic -o bazel-out/k8-fastbuild/bin/jni/libHSjni-1.0.0-ghc8.2.2.so -package base -package bytestring -package choice -package containers -package constraints -package deepseq -package inline-c -package singletons -ljni_Sopenjdk_libjvm -Lbazel-out/k8-fastbuild/bin/jni bazel-out/k8-fastbuild/bin/jni/objects-jni-1.0.0/Foreign/JNI.dyn_o bazel-out/k8-fastbuild/bin/jni/objects-jni-1.0.0/Foreign/JNI/Internal.dyn_o bazel-out/k8-fastbuild/bin/jni/objects-jni-1.0.0/Foreign/JNI/NativeMethod.dyn_o bazel-out/k8-fastbuild/bin/jni/objects-jni-1.0.0/Foreign/JNI/String.dyn_o bazel-out/k8-fastbuild/bin/jni/objects-jni-1.0.0/Foreign/JNI/Types.dyn_o)

...we still are looking for libjvm.so (which is not mentioned anywhere else).

Further, I've confirmed that after linking is done, the symbolic link jni_Sopenjdk_libjvm.so has no importance at all. You can point it to other file or indeed delete it altogether. Why? Because libHSjni-1.0.0-ghc8.2.2.so wants the linking-time target of that symbolic link (the call to ldd is done with the same values of environment variables as during command execution by bazel):

5235b8b51547:~/.cache/bazel/_bazel_root/587be86fe6ecbef03a79b0f6efc9d3d8/execroot/io_tweag_inline_java# ldd bazel-out/
k8-fastbuild/bin/jni/libHSjni-1.0.0-ghc8.2.2.so
        linux-vdso.so.1 (0x00007ffd52b9c000)
        libm.so.6 => /nix/store/glb7dzs11di2cl66rsi8f49wraaj0pqd-glibc-2.26-75/lib/libm.so.6 (0x00007f2568fe2000)
        libjvm.so => not found   # THIS!
        libHSsingletons-2.3.1-GLbBjSX9wPZChwEkzGW0tI-ghc8.2.2.so => /nix/store/cill76iqcr1wf79nz23f8aba6aj0gz62-singletons-2.3.1/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSsingletons-2.3.1-GLbBjSX9wPZChwEkzGW0tI-ghc8.2.2.so (0x00007f25687cd000)
        libHSth-desugar-1.7-ETD1DMEh1fDKd3O9ckx3FL-ghc8.2.2.so => /nix/store/zp84g0pfjpng715kffmn481kp4wa35ac-th-desugar-1.7/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSth-desugar-1.7-ETD1DMEh1fDKd3O9ckx3FL-ghc8.2.2.so (0x00007f2568404000)
        libHSth-orphans-0.13.4-J2yJOX1FJyu28MKQqV4dR6-ghc8.2.2.so => /nix/store/8hymvrxbg9fp6916mp256324n4jvizhc-th-orphans-0.13.4/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSth-orphans-0.13.4-J2yJOX1FJyu28MKQqV4dR6-ghc8.2.2.so (0x00007f25681af000)
        libHSth-lift-instances-0.1.11-FC5rykMSv1V4SUxyFcJrVy-ghc8.2.2.so => /nix/store/fyznahi9gkl27g3w374gg4c7d7l9z5jl-th-lift-instances-0.1.11/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSth-lift-instances-0.1.11-FC5rykMSv1V4SUxyFcJrVy-ghc8.2.2.so (0x00007f2567fa1000)
        libHSth-reify-many-0.1.8-G7DHN9ZzyrxDsCQmxxQ608-ghc8.2.2.so => /nix/store/5z41hcazrvib5sab5l7kpcjhxhbh9bin-th-reify-many-0.1.8/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSth-reify-many-0.1.8-G7DHN9ZzyrxDsCQmxxQ608-ghc8.2.2.so (0x00007f2567d91000)
        libHSth-expand-syns-0.4.3.0-ELNf2VbhI4oI4lVa7rbaZ7-ghc8.2.2.so => /nix/store/ffqxxh255ya7g25kdmw19hll9f4p6b0f-th-expand-syns-0.4.3.0/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSth-expand-syns-0.4.3.0-ELNf2VbhI4oI4lVa7rbaZ7-ghc8.2.2.so (0x00007f2567b78000)
        libHSsafe-0.3.15-Hy54BkCTPx6Bk6Q4GUOJt8-ghc8.2.2.so => /nix/store/hxh2irf5a10b6v9abrp49jzpvs8c3ddk-safe-0.3.15/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSsafe-0.3.15-Hy54BkCTPx6Bk6Q4GUOJt8-ghc8.2.2.so (0x00007f256793e000)
        libHSth-lift-0.7.7-4Fh2JOKcGGMJqOzEylnfxU-ghc8.2.2.so => /nix/store/9p3vzc3jh2k9rjgzrlr2j1s2svbcmn62-th-lift-0.7.7/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSth-lift-0.7.7-4Fh2JOKcGGMJqOzEylnfxU-ghc8.2.2.so (0x00007f2567727000)
        libHSsyb-0.7-KPIRBmt7Nu2JSJko7v2QM4-ghc8.2.2.so => /nix/store/r958q5amwi2aaky7s72b487x50izp3f2-syb-0.7/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSsyb-0.7-KPIRBmt7Nu2JSJko7v2QM4-ghc8.2.2.so (0x00007f25674e6000)
        libHSinline-c-0.6.0.5-10pmocMfjnz3vvXf3ri6nz-ghc8.2.2.so => /nix/store/28d6f2fcf7q1vyfygipz8j36bi9v7bj6-inline-c-0.6.0.5/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSinline-c-0.6.0.5-10pmocMfjnz3vvXf3ri6nz-ghc8.2.2.so (0x00007f256717d000)
        libHSvector-0.12.0.1-IUGn3M9mkBh8CyXcBnfTR4-ghc8.2.2.so => /nix/store/na6r0zfizkg340srp2rw8agkvrw3g0cg-vector-0.12.0.1/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSvector-0.12.0.1-IUGn3M9mkBh8CyXcBnfTR4-ghc8.2.2.so (0x00007f2566ba0000)
        libHStemplate-haskell-2.12.0.0-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/template-haskell-2.12.0.0/libHStemplate-haskell-2.12.0.0-ghc8.2.2.so (0x00007f2566651000)
        libHSpretty-1.1.3.3-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/pretty-1.1.3.3/libHSpretty-1.1.3.3-ghc8.2.2.so (0x00007f25663de000)
        libHSghc-boot-th-8.2.2-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/ghc-boot-th-8.2.2/libHSghc-boot-th-8.2.2-ghc8.2.2.so (0x00007f25661a2000)
        libHSparsers-0.12.7-6gJPrvH9fYC55MCpxbMrOD-ghc8.2.2.so => /nix/store/x7haindlq90zk0vw38k6wj4swdr62rrv-parsers-0.12.7/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSparsers-0.12.7-6gJPrvH9fYC55MCpxbMrOD-ghc8.2.2.so (0x00007f2565ec2000)
        libHSattoparsec-0.13.2.0-Hn159ePSl1wBY8Ps5nsN2o-ghc8.2.2.so => /nix/store/j1pkbbhijaqjz1z29lpzxkc8g72c5klr-attoparsec-0.13.2.0/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSattoparsec-0.13.2.0-Hn159ePSl1wBY8Ps5nsN2o-ghc8.2.2.so (0x00007f2565add000)
        libHSscientific-0.3.5.2-7rBrAvI3wEV6ZzvbQDW5Lt-ghc8.2.2.so => /nix/store/w2zizpf8c3m1kibia9j8gf7vrdxfjmlz-scientific-0.3.5.2/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSscientific-0.3.5.2-7rBrAvI3wEV6ZzvbQDW5Lt-ghc8.2.2.so (0x00007f2565886000)
        libHSprimitive-0.6.2.0-EI3NK1Xfv9zEcRtyXK2EwZ-ghc8.2.2.so => /nix/store/8axksy58pwmfwmz9ycg1shmqj8qg49v7-primitive-0.6.2.0/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSprimitive-0.6.2.0-EI3NK1Xfv9zEcRtyXK2EwZ-ghc8.2.2.so (0x00007f25655ff000)
        libHSinteger-logarithms-1.0.2-5W3scnnO28EGZ7bLb2MgtH-ghc8.2.2.so => /nix/store/nhpc6larww2qk3v9fbszarjkj1y9x5kj-integer-logarithms-1.0.2/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSinteger-logarithms-1.0.2-5W3scnnO28EGZ7bLb2MgtH-ghc8.2.2.so (0x00007f25653e8000)
        libHScharset-0.3.7.1-8XyW23P7e0vC10RPz4i9Gm-ghc8.2.2.so => /nix/store/q5c4qwn24msf2df69m9zxr91im55ydbh-charset-0.3.7.1/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHScharset-0.3.7.1-8XyW23P7e0vC10RPz4i9Gm-ghc8.2.2.so (0x00007f2565082000)
        libHSunordered-containers-0.2.8.0-wMgBBs02mkBVuAqksMsyy-ghc8.2.2.so => /nix/store/w0d8qjdirddjvkii78m15831j3n09p6f-unordered-containers-0.2.8.0/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSunordered-containers-0.2.8.0-wMgBBs02mkBVuAqksMsyy-ghc8.2.2.so (0x00007f2564dfe000)
        libHSsemigroups-0.18.3-3aCIuYG4ID286XaN6MuQu7-ghc8.2.2.so => /nix/store/cf1g5w23hacz2m7ihrjysrprqsmlwsz2-semigroups-0.18.3/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSsemigroups-0.18.3-3aCIuYG4ID286XaN6MuQu7-ghc8.2.2.so (0x00007f2564bf9000)
        libHSbase-orphans-0.6-BvLpQP4NGZV7lCa7V7TFkb-ghc8.2.2.so => /nix/store/gn0gj45g7167hb4j0d8w93v9x7pakz8z-base-orphans-0.6/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSbase-orphans-0.6-BvLpQP4NGZV7lCa7V7TFkb-ghc8.2.2.so (0x00007f25649f6000)
        libHSparsec-3.1.11-EdqzddDfuncJ6FNd9Q11d7-ghc8.2.2.so => /nix/store/a1fsziaayi06l1qqvlc57x0nzgakcmrz-parsec-3.1.11/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSparsec-3.1.11-EdqzddDfuncJ6FNd9Q11d7-ghc8.2.2.so (0x00007f2564745000)
        libHSansi-wl-pprint-0.6.8.1-6pXo22THHmKEAPG4YHprGG-ghc8.2.2.so => /nix/store/lwh3cqqngrnf1zv0zsjkgmxc08n61nzl-ansi-wl-pprint-0.6.8.1/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSansi-wl-pprint-0.6.8.1-6pXo22THHmKEAPG4YHprGG-ghc8.2.2.so (0x00007f2564515000)
        libHSansi-terminal-0.7.1.1-83gYTJ4SUfw3TVZfePHzOW-ghc8.2.2.so => /nix/store/rm5nsvzlgx00c81inm83fhjjx6nas959-ansi-terminal-0.7.1.1/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSansi-terminal-0.7.1.1-83gYTJ4SUfw3TVZfePHzOW-ghc8.2.2.so (0x00007f256429d000)
        libHScolour-2.3.4-3MiiEznBxpfEYrMTHbnSer-ghc8.2.2.so => /nix/store/7jmkygmnnqhbinn8abhcy4647hmyv0fy-colour-2.3.4/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHScolour-2.3.4-3MiiEznBxpfEYrMTHbnSer-ghc8.2.2.so (0x00007f256400f000)
        libHSconstraints-0.9.1-820ZUkkHpxdFebnmkCH9ba-ghc8.2.2.so => /nix/store/k2s5k6scfc13ivq1m3lsl7jwj8i02afz-constraints-0.9.1/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSconstraints-0.9.1-820ZUkkHpxdFebnmkCH9ba-ghc8.2.2.so (0x00007f2563d56000)
        libHStransformers-compat-0.5.1.4-EAh8Ccrgitz4vTZtbTPyCM-ghc8.2.2.so => /nix/store/rh9d6pfm26frfl3ki0zp8ij033a4m7bm-transformers-compat-0.5.1.4/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHStransformers-compat-0.5.1.4-EAh8Ccrgitz4vTZtbTPyCM-ghc8.2.2.so (0x00007f2563b4d000)
        libHSmtl-2.2.1-DscMMmDQUE6GBfOSl4qMUH-ghc8.2.2.so => /nix/store/r5b1pgxryskv9pndzfddkqxjnwk32zs4-mtl-2.2.1/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSmtl-2.2.1-DscMMmDQUE6GBfOSl4qMUH-ghc8.2.2.so (0x00007f2563918000)
        libHStransformers-0.5.2.0-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/transformers-0.5.2.0/libHStransformers-0.5.2.0-ghc8.2.2.so (0x00007f256360f000)
        libHShashable-1.2.6.1-CqLaJxgPIOpH0VY2OuCehB-ghc8.2.2.so => /nix/store/c1fwyxk4f5is2rvxv8qnxp3n15g0adyd-hashable-1.2.6.1/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHShashable-1.2.6.1-CqLaJxgPIOpH0VY2OuCehB-ghc8.2.2.so (0x00007f25633d9000)
        libHStext-1.2.2.2-J0kTo5xltQjAfCz3JZ4Oj0-ghc8.2.2.so => /nix/store/ndn4wy1rd2wzs0mv4g0mn9cvccf7fahp-text-1.2.2.2/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHStext-1.2.2.2-J0kTo5xltQjAfCz3JZ4Oj0-ghc8.2.2.so (0x00007f2562f3a000)
        libHSbinary-0.8.5.1-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/binary-0.8.5.1/libHSbinary-0.8.5.1-ghc8.2.2.so (0x00007f2562c96000)
        libHScontainers-0.5.10.2-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/containers-0.5.10.2/libHScontainers-0.5.10.2-ghc8.2.2.so (0x00007f25627b8000)
        libHSchoice-0.2.2-JCgJvojpuaW6pCIwBxVQX2-ghc8.2.2.so => /nix/store/sr29q72z4flcqvl0xb5dszsvl63mixk8-choice-0.2.2/lib/ghc-8.2.2/x86_64-linux-ghc-8.2.2/libHSchoice-0.2.2-JCgJvojpuaW6pCIwBxVQX2-ghc8.2.2.so (0x00007f25625ab000)
        libHSbytestring-0.10.8.2-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/bytestring-0.10.8.2/libHSbytestring-0.10.8.2-ghc8.2.2.so (0x00007f25622b7000)
        libHSdeepseq-1.4.3.0-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/deepseq-1.4.3.0/libHSdeepseq-1.4.3.0-ghc8.2.2.so (0x00007f256209e000)
        libHSarray-0.5.2.0-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/array-0.5.2.0/libHSarray-0.5.2.0-ghc8.2.2.so (0x00007f2561e1a000)
        libHSbase-4.10.1.0-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/base-4.10.1.0/libHSbase-4.10.1.0-ghc8.2.2.so (0x00007f2561253000)
        libHSinteger-gmp-1.0.1.0-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/integer-gmp-1.0.1.0/libHSinteger-gmp-1.0.1.0-ghc8.2.2.so (0x00007f2561018000)
        libHSghc-prim-0.5.1.1-ghc8.2.2.so => /nix/store/mm3p3w645havyn5kn1fvr8pwg5wvzz0v-ghc-8.2.2/lib/ghc-8.2.2/ghc-prim-0.5.1.1/libHSghc-prim-0.5.1.1-ghc8.2.2.so (0x00007f256097c000)
        libgmp.so.10 => /nix/store/2p7a3yb6hilmjch90p3bw56shn76a0h5-gmp-6.1.2/lib/libgmp.so.10 (0x00007f25606e9000)
        libc.so.6 => /nix/store/glb7dzs11di2cl66rsi8f49wraaj0pqd-glibc-2.26-75/lib/libc.so.6 (0x00007f2560336000)
        /nix/store/27b2hylj1sv682iqijq3g7mbcfiy1rfw-glibc-2.26-131/lib64/ld-linux-x86-64.so.2 (0x00007f256959e000)

It looks for libjvm.so directly. The path to it is not in LD_LIBRARY_PATH, so it cannot find it. If we add path to libjvm.so to LD_LIBRARY_PATH everything works fine.

So mangling does not do what we expected it to do. Names of .so libraries are still names of libraries after resolution of symbolic links. That's somewhat unexpected. And it means that cbits problem is not solved: by the same logic, every library with cbits will be looking for the same libcbits.so file and they will be confused. I tried to find an option for ld that would prevent symbolic link resolution but found nothing so far.

About avoiding mangling for system libs: I don't see a reliable way to tell if a .so library coming to us from a dependency is system-level library or some local library. Anyway this is not important because mangling in its current form doesn't appear to work.

One solution is not resurrect my older PR that adds special attributes for cbits. If this doesn't look good still, then we could do mangling by doing actual copying I guess, but that's not nice at all.

@mboes Your opinion?

@mboes
Copy link
Member Author

mboes commented Feb 19, 2018

So mangling does not do what we expected it to do. Names of .so libraries are still names of libraries after resolution of symbolic links. That's somewhat unexpected. And it means that cbits problem is not solved: by the same logic, every library with cbits will be looking for the same libcbits.so file and they will be confused. I tried to find an option for ld that would prevent symbolic link resolution but found nothing so far.

I suspect you're overstating the issue. Without mangling, we couldn't build aeson and its dependencies. With mangling, we now can. Also, the mangling we do is no invention of our own. It's what the cc_* rules of Bazel do internally. That's not to say there are no bugs in the implementation (hence this ticket). But I doubt the approach is fundamentally flawed.

No need to rehash the discussion about adding special support for building C code here. The problem mangling is meant to solve is present with or without that.

@mboes
Copy link
Member Author

mboes commented Feb 19, 2018

About avoiding mangling for system libs: I don't see a reliable way to tell if a .so library coming to us from a dependency is system-level library or some local library.

Well, one is "imported" using cc_import and the like. The other is not. It's two different rules. It should be easy to pass along the information of which rule was used in a provider.

@mrkkrp
Copy link
Member

mrkkrp commented Feb 19, 2018

But these rules (cc_haskell_import and haskell_cc_import) are temporary and will be deprecated in the future. At least that's what we say in the docs.

@mrkkrp
Copy link
Member

mrkkrp commented Feb 19, 2018

There is -no-canonical-prefixes in gcc:

       -no-canonical-prefixes
           Do not expand any symbolic links, resolve references to /../ or
           /./, or make the path absolute when generating a relative
           prefix.

I'll try to use that and see what happens.

@mrkkrp
Copy link
Member

mrkkrp commented Feb 19, 2018 •
edited
Loading

Update. The name that is looked up when something that depends on dynamic library is run is determined:

  • By SONAME field in library itself. That field can be set when the library is compiled.
  • By filename, if SONAME is not set.

We don't compile system libraries, so we cannot set SONAME. Even worse (?) libjvm.so has SONAME field in it:

[mark@arch try_network]$ readelf -d /usr/lib/jvm/java-8-openjdk/jre/lib/amd64/server/libjvm.so | grep SONAME
 0x000000000000000e (SONAME)             Library soname: [libjvm.so]

This means no matter what you do things that depend on it will search for libjvm.so.

@mrkkrp
Copy link
Member

mrkkrp commented Feb 19, 2018

Perhaps in cc_* rules they can actually control SONAMEs, so mangling works for them... Can you please show me where you took the mangling function from? I searched bazel repo like so:

https://github.com/bazelbuild/bazel/search?utf8=%E2%9C%93&q=mangle&type=

But it finds nothing.

@mrkkrp
Copy link
Member

mrkkrp commented Feb 19, 2018

The best compromise I can think of: during mangling, detect if SONAME field is set. In that case pass the lib as is.

@mrkkrp
Copy link
Member

mrkkrp commented Feb 19, 2018

OK, half of the fix is done, but it's more complicated than it seemed because sonames often include numeric suffixes like .so.5 and so our code should handle these properly and also figure out that when we have libfoo.so and libfoo.so.2 it's the same thing and we only need a single -lfoo for it.

@mrkkrp mrkkrp mentioned this issue Feb 20, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mrkkrp mrkkrp

Labels
P1 critical: next release type: bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Do not mangle system-level solibs tweag/rules_haskell
2 participants
@mboes @mrkkrp