Problem to statically link bpftool

[huang195]

Following the instructions in README.md, I used the command

 CFLAGS=--static make

hoping to compile and statically link the bpftool binary, however, the error I got it

/usr/bin/ld: cannot find -lelf
/usr/bin/ld: cannot find -lz
/usr/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status

I'm using a fedora:32 docker container to build. I'm guessing I need to install statically linked versions of libelf, libz and libc? I see libelf.so, libz.so and libc.so are all there in /usr/lib64. Where do I get libelf.a, libz.a, and libc.a? Not very familiar with statically linked libraries. Any help is appreciated!

[huang195]

I tried this on a debian docker image, and it still doesn't link, and I'm getting a whole bunch of errors like:

/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libbfd.a(plugin.o): in function `try_load_plugin':
/build/binutils-lybw70/binutils-2.31.1/builddir-single/bfd/../../bfd/plugin.c:228: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libbfd.a(elf64-x86-64.o): in function `elf_x86_64_output_arch_local_syms':
/build/binutils-lybw70/binutils-2.31.1/builddir-single/bfd/../../bfd/elf64-x86-64.c:4551: undefined reference to `htab_traverse'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libbfd.a(elf.o): in function `_bfd_elf_is_local_label_name':
/build/binutils-lybw70/binutils-2.31.1/builddir-single/bfd/../../bfd/elf.c:8682: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libbfd.a(elflink.o): in function `elf_link_add_object_symbols':
/build/binutils-lybw70/binutils-2.31.1/builddir-single/bfd/../../bfd/elflink.c:5211: undefined reference to `objalloc_free_block'
/usr/bin/ld: /build/binutils-lybw70/binutils-2.31.1/builddir-single/bfd/../../bfd/elflink.c:5462: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libbfd.a(elflink.o): in function `bfd_elf_size_dynamic_sections':
/build/binutils-lybw70/binutils-2.31.1/builddir-single/bfd/../../bfd/elflink.c:6655: undefined reference to `lbasename'
/usr/bin/ld: /build/binutils-lybw70/binutils-2.31.1/builddir-single/bfd/../../bfd/elflink.c:6428: undefined reference to `lbasename'

I'm trying to build/link it similar to the instructions here: https://github.com/projectcalico/bpftool, but instead of building it from the Linux kernel, I thought it would be faster to git clone this repo instead, but it doesn't seem to work when trying to statically link it.

P.S. Without the CFLAGS=--static part, it works, but I need the binary statically linked

[qmonnet]

Hi, and thanks for the report! The README specifies that note that this does not work out-of-the-box when linking with libbfd, and this is what you're facing (at least on Debian).

I haven't found the time to look into the details for compiling statically with libbfd or llvm. I've been told that it works with LLVM, provided that the statically compiled version of the library is installed. For libbfd, it seems the libbfd.a library on the system is not enough, but I need to dig more to figure out the cause of these messages. Eventually we probably want to try static builds in the CI, too.

In the meantime, a quick workaround would be to disable linking against libbfd/llvm; then CFLAGS=--static make works for me (tested on Ubuntu). At the moment you have to edit the Makefile to disable libbfd/llvm, though.

[huang195]

@qmonnet oh hey thanks for the quick reply. Can I ask what did you modify in the Makefile to disable libbfd/llvm? I can try it on my Debian box, and if it doesn't work, I can certainly try it on Ubuntu as well.

[huang195]

Ah I figured it out, just putting it here in case others might need a workaround:

root@eb84dabef474:/bpftool/src# ldd bpftool
	not a dynamic executable
root@eb84dabef474:/bpftool/src# git diff Makefile
diff --git a/src/Makefile b/src/Makefile
index c2b9200..b12f82d 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -20,6 +20,9 @@ else
 endif
 BOOTSTRAP_OUTPUT := $(_OUTPUT)bootstrap/
 
+feature-llvm := 0
+feature-libbfd := 0
+
 LIBBPF_OUTPUT := $(_OUTPUT)libbpf/
 LIBBPF_DESTDIR := $(LIBBPF_OUTPUT)
 LIBBPF_INCLUDE := $(LIBBPF_DESTDIR)include
@@ -143,11 +146,11 @@ ifeq ($(feature-llvm),1)
 else
   # Fall back on libbfd
   ifeq ($(feature-libbfd),1)
-    LIBS += -lbfd -ldl -lopcodes
+    LIBS += -ldl -lopcodes
   else ifeq ($(feature-libbfd-liberty),1)
-    LIBS += -lbfd -ldl -lopcodes -liberty
+    LIBS += -ldl -lopcodes -liberty
   else ifeq ($(feature-libbfd-liberty-z),1)
-    LIBS += -lbfd -ldl -lopcodes -liberty -lz
+    LIBS += -ldl -lopcodes -liberty -lz
   endif
 
   # If one of the above feature combinations is set, we support libbfd

[qmonnet]

Yes, disabling them should be doable just by changing FEATURE_TESTS, but I realise the Makefile.feature I did for this repo ignores FEATURE_TESTS and always run the detection. I'll fix that soon.

[huang195]

OK, seems like I used brute force that did the trick, but there are nicer ways. Thanks a lot for your help on finding a workaround @qmonnet . Closing

[qmonnet]

It was a useful report, I opened another issue as a follow-up.

[qmonnet]

I also fixed the usage of FEATURE_TESTS in the Makefile, so it's easier to tweak it to disable some features.