libxkbcommon.so.0 could not be located when cross-compiling

[shiro]

Hello,

I've been successfully using maturin to compile x86 architectures for a while, and it works amazing, even though my library includes libxkbcommon, a native dependency.

Now I'm trying to add arm support, which turned out to be complicated. I'm using ghcr.io/rust-cross/manylinux2014-cross:aarch64 (which itself is an x86 container) and installed the arm version of libxkbcommon by adding a foreign architecture through dpkg and adding arm PPAs.

After doing so, and telling rust where the libs are (export RUSTFLAGS='-L /usr/lib/aarch64-linux-gnu'), it compiled! The final issue I couldn't resolve was the final step - auditwheel, not passing, it always throws the following error:

...
    Finished `release` profile [optimized] target(s) in 24.31s
💥 maturin failed
  Caused by: Cannot repair wheel, because required library libxkbcommon.so.0 could not be located.

It seems to not find the lib, even though the linker could (during compilation). I tried moving the .so files to a bunch of places, but nothing changed. Using the standalone auditwheel show also didn't help, just throws an error (I assume the wheel is still incomplete as maturin isn't done with it).

Edit:

To reproduce:

git clone -b proto/arm https://github.com/shiro/map2.git .
docker run --rm -it -v `pwd`:/mnt -w /mnt ghcr.io/rust-cross/manylinux2014-cross:aarch64 bash
source arm-test.sh
maturin build -f

[messense]

Can you provide a repro? Otherwise it's hard to investigate. Thanks.

[shiro]

Thanks for looking into it, I really appreciate it!
I added the repro steps to the post, hope it helps.

[shiro]

Any pointers as to why it isn't being found?

[messense]

I'm unable to reproduce your issue, it fails before that:

error: failed to run custom build command for `evdev-sys v0.2.5 (/mnt/map2/evdev-rs/evdev-sys)`

Caused by:
  process didn't exit successfully: `/mnt/map2/target/debug/build/evdev-sys-cb31a9d0323d09cd/build-script-build` (exit status: 101)
  --- stdout
  running: "git" "submodule" "update" "--init" "--depth" "1"
  running: cd "/mnt/map2/evdev-rs/evdev-sys" && "cp" "-r" "/mnt/map2/evdev-rs/evdev-sys/libevdev/" "/mnt/map2/target/aarch64-unknown-linux-gnu/debug/build/evdev-sys-e58e9dd176095430/out"
  cargo:rustc-link-search=/mnt/map2/target/aarch64-unknown-linux-gnu/debug/build/evdev-sys-e58e9dd176095430/out/lib
  cargo:root=/mnt/map2/target/aarch64-unknown-linux-gnu/debug/build/evdev-sys-e58e9dd176095430/out
  cargo:include=/mnt/map2/target/aarch64-unknown-linux-gnu/debug/build/evdev-sys-e58e9dd176095430/out/include
  cargo:rerun-if-changed=libevdev
  cargo:rustc-link-lib=static=evdev
  OPT_LEVEL = Some("0")
  TARGET = Some("aarch64-unknown-linux-gnu")
  HOST = Some("x86_64-unknown-linux-gnu")
  cargo:rerun-if-env-changed=CC_aarch64-unknown-linux-gnu
  CC_aarch64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_aarch64_unknown_linux_gnu
  CC_aarch64_unknown_linux_gnu = Some("aarch64-unknown-linux-gnu-gcc")
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("neon")
  cargo:rerun-if-env-changed=CFLAGS_aarch64-unknown-linux-gnu
  CFLAGS_aarch64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_aarch64_unknown_linux_gnu
  CFLAGS_aarch64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=TARGET_CFLAGS
  TARGET_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  running: cd "/mnt/map2/target/aarch64-unknown-linux-gnu/debug/build/evdev-sys-e58e9dd176095430/out/build" && CC="aarch64-unknown-linux-gnu-gcc" CFLAGS="-O0 -ffunction-sections -fdata-sections -fPIC -gdwarf-4 -fno-omit-frame-pointer -Wall -Wextra " "sh" "/mnt/map2/target/aarch64-unknown-linux-gnu/debug/build/evdev-sys-e58e9dd176095430/out/libevdev/autogen.sh" "--host=x86_64-unknown-linux-gnu" "--target=aarch64-unknown-linux-gnu" "--prefix=/mnt/map2/target/aarch64-unknown-linux-gnu/debug/build/evdev-sys-e58e9dd176095430/out"

  --- stderr
  fatal: detected dubious ownership in repository at '/mnt/map2'
  To add an exception for this directory, call:

  	git config --global --add safe.directory /mnt/map2
  sh: 0: cannot open /mnt/map2/target/aarch64-unknown-linux-gnu/debug/build/evdev-sys-e58e9dd176095430/out/libevdev/autogen.sh: No such file
  thread 'main' panicked at evdev-rs/evdev-sys/build.rs:121:5:
  assertion failed: cmd.status()?.success()
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
💥 maturin failed

[shiro]

Interesting, I guess I already had the submodules checked out, so the container didn't throw in my case.
Have you tried running git config --global --add safe.directory /mnt/map2 in the container before building?

[messense]

I got a different error:

/usr/aarch64-unknown-linux-gnu/lib/gcc/aarch64-unknown-linux-gnu/4.8.5/../../../../aarch64-unknown-linux-gnu/bin/ld.bfd: cannot find -lxkbcommon
collect2: error: ld returned 1 exit status

[messense]

Probably because when repair the wheel maturin only tries paths from <target>-gcc --print-sysroot and linked_paths of cargo build script, it does not take -L options in rust flags into consideration yet.

[shiro]

I see, however running:

root@e0947353027d:/mnt# aarch64-unknown-linux-gnu-gcc --print-sysroot
/usr/aarch64-unknown-linux-gnu/aarch64-unknown-linux-gnu/sysroot

shows the same directory I already tried copying the lib to in the sample arm-test.sh script, still doesn't find it. Is there any place that I can copy it to as a temporary solution just to get it to compile for now?

[shiro]

Nevermind, I missed the sysroot part of the path, the following command fixes the issue:

cp /usr/lib/aarch64-linux-gnu/libxkbcommon.so.0.0.0 /usr/aarch64-unknown-linux-gnu/aarch64-unknown-linux-gnu/sysroot/lib64/libxkbcommon.so.0

It's still a hack though, would be great if the -L flag got picked up automatically. Thanks for the help 🙇

[messense]

Opened #2096 to track this.