Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libbpf-tools not building because clang attribute error #2855

Open
rtkaratekid opened this issue Apr 2, 2020 · 9 comments
Open

libbpf-tools not building because clang attribute error #2855

rtkaratekid opened this issue Apr 2, 2020 · 9 comments

Comments

@rtkaratekid
Copy link

rtkaratekid commented Apr 2, 2020 •
edited
Loading

Hi all, I'm taking a stab at the new CO-RE API. If these are successfully compiled, they should be able to run on kernels without BTF support? Is that correct?

Regardless, here's the issue. Running Ubuntu 18.04 I just compiled and installed kernel 5.4 with BTF support (which was successful according to bpftool). I then built LLVM/Clang from source using:

apt-get install -y cmake gcc g++
git clone --depth 1 http://llvm.org/git/llvm.git
cd llvm/tools
git clone --depth 1 http://llvm.org/git/clang.git
cd ..; mkdir build; cd build;
cmake -DLLVM_TARGETS_TO_BUILD="X86;BPF" \
  -DLLVM_BUILD_LLVM_DYLIB=ON \
  -DLLVM_ENABLE_RTTI=ON \
  -DCMAKE_BUILD_TYPE=Release ..

make -j$(getconf _NPROCESSORS_ONLN)
make install

Demonstrating successful installation:

$ clang --version
clang version 10.0.0 (http://llvm.org/git/clang.git 65acf43270ea2894dffa0d0b292b92402f80c8cb) (http://llvm.org/git/llvm.git 2c4ca6832fa6b306ee6a7010bfb80a3f2596f824)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin

$ llc --version
LLVM (http://llvm.org/):
  LLVM version 10.0.0svn
  Optimized build.
  Default target: x86_64-unknown-linux-gnu
  Host CPU: skylake

  Registered Targets:
    bpf    - BPF (host endian)
    bpfeb  - BPF (big endian)
    bpfel  - BPF (little endian)
    x86    - 32-bit X86: Pentium-Pro and above
    x86-64 - 64-bit X86: EM64T and AMD64

I clone the BCC repo and then:

$ cd libbpf-tools/
$ make

And got this error:

In file included from drsnoop.bpf.c:3:
./vmlinux.h:5:15: error: attribute 'preserve_access_index' is not supported by '#pragma clang attribute'
#pragma clang attribute push (__attribute__((preserve_access_index)), apply_to = record)
              ^
./vmlinux.h:127438:15: error: '#pragma clang attribute pop' with no matching '#pragma clang attribute push'
#pragma clang attribute pop
              ^
2 errors generated.
Makefile:50: recipe for target '.output/drsnoop.bpf.o' failed
make: *** [.output/drsnoop.bpf.o] Error 1

I think the pop attribute issue will be solved when the push is solved. So, pulling on that thread, I went to the Clang docs since I thought the attribute issue was weird, and it looks like Clang10 does actually include preserve_access_index. Since this is an issue with the vmlinux_505.h file, I'm not really sure where to proceed from here. I don't quite understand BTF yet although today or tomorrow I'm hoping to shore up my ignorance by looking at the Facebook microsite.

Have I done something wrong, or is this an issue with the vmlinux.h generation? What can I do to fix this error?

Edit: I just realized that I'm trying to use the libbpf-tools vmlinux_505.h for a 5.4 kernel. I'll don't know how to generate my own vmlinux_504.h file yet, but I'll see if I can figure it out. That could be the issue?

Edit (again, sorry): just tried using bpftool for vmlinux generation

$ bpftool btf dump file /sys/kernel/btf/vmlinux format c

#ifndef __VMLINUX_H__
#define __VMLINUX_H__

#ifndef BPF_NO_PRESERVE_ACCESS_INDEX
#pragma clang attribute push (__attribute__((preserve_access_index)), apply_to = record)
#endif

#ifndef BPF_NO_PRESERVE_ACCESS_INDEX
#pragma clang attribute pop
#endif

#endif /* __VMLINUX_H__ */

But I'm not sure how to populate the body of the vmlinux.h file.
@anakryiko
Copy link
Contributor

Hi all, I'm taking a stab at the new CO-RE API. If these are successfully compiled, they should be able to run on kernels without BTF support? Is that correct?

No, to run they need target kernels to have BTF, otherwise it's impossible to relocate and adjust BPF program itself correctly.

Regardless, here's the issue. Running Ubuntu 18.04 I just compiled and installed kernel 5.4 with BTF support (which was successful according to bpftool). I then built LLVM/Clang from source using:

apt-get install -y cmake gcc g++
git clone --depth 1 http://llvm.org/git/llvm.git
cd llvm/tools
git clone --depth 1 http://llvm.org/git/clang.git

It's a real disaster the way that LLVM/Clang handled transition to single repo. Those two git repositories are stale and few people reported problems trying to use version compiled from them. Please use (now canonical) single repo: https://github.com/llvm/llvm-project
Or just install Clang/LLVM 10 from packages.

Edit: I just realized that I'm trying to use the libbpf-tools vmlinux_505.h for a 5.4 kernel. I'll don't know how to generate my own vmlinux_504.h file yet, but I'll see if I can figure it out. That could be the issue?

Using 5.5 vmlinux.h with 5.4 kernel should be fine in most cases. vmlinux.h is there to provide type definitions, they don't have to be exactly the same as your target kernel. Only types that you are actually using should be compatible. If there are some new fields on 5.5, which are not present on 5.4, but your program is not accessing them - it should be ok. Which is the case almost in all cases.

Edit (again, sorry): just tried using bpftool for vmlinux generation

$ bpftool btf dump file /sys/kernel/btf/vmlinux format c

But I'm not sure how to populate the body of the vmlinux.h file.

Hm... Are you sure /sys/kernel/btf/vmlinux exists and is not empty? Check that your kernel is compiled with CONFIG_DEBUG_INFO_BTF=y.

@rtkaratekid
Copy link
Author

@anakryiko thanks for the helpful reply.

No, to run they need target kernels to have BTF, otherwise it's impossible to relocate and adjust BPF program itself correctly.

I think that makes sense. Is this because the libbpf loader needs to be able to refer to the kernel datatypes? Even if, say, the BTF kernel is compiled for 5.5, that BTF dependent program couldn't run on a non-BTF 5.5 kernel? Sorry, I'm trying to write programs that can be easily run across one kernel version. I've tried with non-CORE BPF, but have had a lot of trouble trying to understand how to extract meaningful data from specifically tracepoints. So I'm now examining CORE to see if that's easier, but it would be a bit of a hurdle if it couldn't generalize to non-BTF supported kernels.

Is there much of a chance of distros including BTF in their default kernel build in the future?

It's a real disaster the way that LLVM/Clang handled transition to single repo. Those two git repositories are stale and few people reported problems trying to use version compiled from them. Please use (now canonical) single repo: https://github.com/llvm/llvm-project
Or just install Clang/LLVM 10 from packages

This is a life saver! I had no idea that this repo was the single resource. I just compiled and installed Clang/LLVM 10 from this source. However I'm still getting the same weird attribute error:

In file included from drsnoop.bpf.c:3:
./vmlinux.h:5:15: error: attribute 'preserve_access_index' is not supported by '#pragma clang attribute'
#pragma clang attribute push (__attribute__((preserve_access_index)), apply_to = record)
              ^
./vmlinux.h:127438:15: error: '#pragma clang attribute pop' with no matching '#pragma clang attribute push'
#pragma clang attribute pop
              ^
2 errors generated.

Hm... Are you sure /sys/kernel/btf/vmlinux exists and is not empty? Check that your kernel is compiled with CONFIG_DEBUG_INFO_BTF=y.

While the above compiler error may be one of the issues, I think perhaps my BTF support wasn't configured correctly before I compiled the kernel. As you saw above, I am able to generate a vmlinux file, but it just doesn't have a body for some strange reason. I did a weird thing where I compiled the kernel with all debug options turned off (to see how fast it would be, which was actually pretty fun), and then went back and turned those options back on, as well as BTF. There's a chance I screwed up along the way and turned something off BTF needed? I'm attempting all this in a VM, so I think I'll just revert today and try it all fresh and see if it makes a difference.

As for whether it exists and if BTF support is enabled:

$ cat /sys/kernel/btf/vmlinux
��

$ ./bpftool btf dump file /sys/kernel/btf/vmlinux format c
#ifndef __VMLINUX_H__
#define __VMLINUX_H__

#ifndef BPF_NO_PRESERVE_ACCESS_INDEX
#pragma clang attribute push (__attribute__((preserve_access_index)), apply_to = record)
#endif

#ifndef BPF_NO_PRESERVE_ACCESS_INDEX
#pragma clang attribute pop
#endif

#endif /* __VMLINUX_H__ */

$ cat /boot/config-5.4.0 | grep BTF
CONFIG_VIDEO_SONY_BTF_MPX=m
CONFIG_DEBUG_INFO_BTF=y

If you have any other troubleshooting ideas, I'd love to hear them. I'll definitely come back and report on if the kernel compilation was the issue.

@anakryiko
Copy link
Contributor

@anakryiko thanks for the helpful reply.

No, to run they need target kernels to have BTF, otherwise it's impossible to relocate and adjust BPF program itself correctly.

I think that makes sense. Is this because the libbpf loader needs to be able to refer to the kernel datatypes?

Yes, exactly.

Even if, say, the BTF kernel is compiled for 5.5, that BTF dependent program couldn't run on a non-BTF 5.5 kernel?

Yes. While technically it is possible to run same compiled BPF code on exactly the same kernel version and kernel configuration, it's too brittle to allow w/ CO-RE. This is due to any tiny config or code change can lead to a different data type memory layout.

Sorry, I'm trying to write programs that can be easily run across one kernel version. I've tried with non-CORE BPF, but have had a lot of trouble trying to understand how to extract meaningful data from specifically tracepoints.

Tracepoints for the same kernel version should be a stable interface, so if you never need to read anything from task_struct or any other internal kernel data structure, you shouldn't need to use CO-RE actually. Start with looking at existing tracepoint-using BPF programs and how they get their data (though most of them will be using BCC, given before BPF CO-RE it was generally very hard to have non-BCC pre-compiled BPF program).

Anyways, if you can share a minimal example of code using some tracepoint and explain what you hope to achieve, people here might be able to help figure out issues you had.

So I'm now examining CORE to see if that's easier, but it would be a bit of a hurdle if it couldn't generalize to non-BTF supported kernels.

It seems like it is going to be a blocker for you.

Is there much of a chance of distros including BTF in their default kernel build in the future?

Well, it's up to users like you demanding this from distros, actually. So keep pushing :) Right now big users of FB (e.g., Facebook, and, I presume, Google, etc), are building kernels with custom configurations that enable CONFIG_DEBUG_INFO_BTF=y explicitly to be able to use it. For Linux distributions w/ stock kernel configs, there will need to be a good push from users to make BTF available by default, I'd imagine.

It's a real disaster the way that LLVM/Clang handled transition to single repo. Those two git repositories are stale and few people reported problems trying to use version compiled from them. Please use (now canonical) single repo: https://github.com/llvm/llvm-project
Or just install Clang/LLVM 10 from packages

This is a life saver! I had no idea that this repo was the single resource. I just compiled and installed Clang/LLVM 10 from this source. However I'm still getting the same weird attribute error:

In file included from drsnoop.bpf.c:3:
./vmlinux.h:5:15: error: attribute 'preserve_access_index' is not supported by '#pragma clang attribute'
#pragma clang attribute push (__attribute__((preserve_access_index)), apply_to = record)
              ^
./vmlinux.h:127438:15: error: '#pragma clang attribute pop' with no matching '#pragma clang attribute push'
#pragma clang attribute pop
              ^
2 errors generated.

This is strange and probably the version of Clang that you built is not the default clang that's resolved when running just clang. Check that which clang and clang --version return point to and return version you are expecting. E.g.,:

$ clang --version
clang version 10.0.0 (https://github.com/llvm/llvm-project.git d32170dbd5b0d54436537b6b75beaf44324e0c28)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/andriin/bin

I just built llvmorg-10.0.0 tag from llvm-project and it successfully compiled libbpf-tools from BCC. Alternatively, you can explicitly specify path to clang binary to use when building libbpf-tools:

$ pwd
/home/andriin/local/bcc/libbpf-tools
$ make clean && make CLANG=/path/to/your/clang-10 -j30
... <success> ...

Hm... Are you sure /sys/kernel/btf/vmlinux exists and is not empty? Check that your kernel is compiled with CONFIG_DEBUG_INFO_BTF=y.

While the above compiler error may be one of the issues, I think perhaps my BTF support wasn't configured correctly before I compiled the kernel. As you saw above, I am able to generate a vmlinux file, but it just doesn't have a body for some strange reason. I did a weird thing where I compiled the kernel with all debug options turned off (to see how fast it would be, which was actually pretty fun), and then went back and turned those options back on, as well as BTF. There's a chance I screwed up along the way and turned something off BTF needed? I'm attempting all this in a VM, so I think I'll just revert today and try it all fresh and see if it makes a difference.

I don't know and it's hard to tell. Verify that /sys/kernel/btf/vmlinux is non-empty file. You should also see a lot of output when running in non-"format c" mode with bpftool:

$ ls -la /sys/kernel/btf/vmlinux
-r--r--r--. 1 root root 3490473 Apr  5 06:41 /sys/kernel/btf/vmlinux
$ bpftool btf dump file /sys/kernel/btf/vmlinux | head -n10
[1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
[2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
[3] CONST '(anon)' type_id=2
[4] VOLATILE '(anon)' type_id=2
[5] ARRAY '(anon)' type_id=2 index_type_id=1 nr_elems=2
[6] PTR '(anon)' type_id=9
[7] CONST '(anon)' type_id=6
[8] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=(none)
[9] CONST '(anon)' type_id=8
[10] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)
$ bpftool btf dump file /sys/kernel/btf/vmlinux format c | head -n20
#ifndef __VMLINUX_H__
#define __VMLINUX_H__

#ifndef BPF_NO_PRESERVE_ACCESS_INDEX
#pragma clang attribute push (__attribute__((preserve_access_index)), apply_to = record)
#endif

typedef signed char __s8;

typedef unsigned char __u8;

typedef short int __s16;

typedef short unsigned int __u16;

typedef int __s32;

typedef unsigned int __u32;

typedef long long int __s64;

$ cat /boot/config-5.4.0 | grep BTF
CONFIG_VIDEO_SONY_BTF_MPX=m
CONFIG_DEBUG_INFO_BTF=y


If you have any other troubleshooting ideas, I'd love to hear them. I'll definitely come back and report on if the kernel compilation was the issue.

Not really, just make sure you have recent enough bpftool, but given it outputs correct preamble, I think it's recent enough already... So just double-check all the configuration. BTW, to check config, I'd zcat procfs entry for Kconfig instead (just having config file in /boot doesn't mean that you booted correct kernel). Here's what I have:

$ zcat /proc/config.gz | grep DEBUG_INFO
CONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_INFO_REDUCED is not set
# CONFIG_DEBUG_INFO_SPLIT is not set
CONFIG_DEBUG_INFO_DWARF4=y
CONFIG_DEBUG_INFO_BTF=y

@rtkaratekid
Copy link
Author

Okay took a while, but I finally got back to this.

Turns out the BTF issue above was due to some weird kernel compilation stuff. I'm doing all this in a VM so when I just destroyed and rebuilt the VM and compiled the kernel with BTF support, those issues figured themselves out. I was able to generate what looks like a valid vmlinux.h file for kernel 5.4.

At that point I apt install clang (I'm on Ubuntu 18.04) and got version 6.0. Tried building libbpf-tools and predictably got the attribute error also mentioned above.

Went to the LLVM site and ran their install script # bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" to get clang-10. Reset the sym link in /usr/bin to point to clang-10.

Went right back to libbpf-tools. Ran make, and didn't get the attribute error or something about BTF (yet) but I found a new error:

vagrant@ubuntu1804:~/bcc/libbpf-tools$ make clean && make
  CLEAN
  MKDIR    .output
  MKDIR    libbpf
  LIB      libbpf.a
mkdir -p /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c bpf.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/bpf.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c btf.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/btf.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c libbpf.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/libbpf.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c libbpf_errno.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/libbpf_errno.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c netlink.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/netlink.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c nlattr.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/nlattr.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c str_error.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/str_error.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c libbpf_probes.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/libbpf_probes.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c bpf_prog_linfo.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/bpf_prog_linfo.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c xsk.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/xsk.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c btf_dump.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/btf_dump.o
cc -I. -I../include -I../include/uapi -g -O2 -Werror -Wall -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64   -c hashmap.c -o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/hashmap.o
ar rcs /home/vagrant/bcc/libbpf-tools/.output//libbpf/libbpf.a /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/bpf.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/btf.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/libbpf.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/libbpf_errno.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/netlink.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/nlattr.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/str_error.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/libbpf_probes.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/bpf_prog_linfo.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/xsk.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/btf_dump.o /home/vagrant/bcc/libbpf-tools/.output//libbpf/staticobjs/hashmap.o
sed -e "s|@PREFIX@|/usr|" \
	-e "s|@LIBDIR@||" \
	-e "s|@VERSION@|0.0.8|" \
	< libbpf.pc.template > /home/vagrant/bcc/libbpf-tools/.output//libbpf/libbpf.pc
if [ ! -d '/home/vagrant/bcc/libbpf-tools/.output//bpf' ]; then install -d -m 755 '/home/vagrant/bcc/libbpf-tools/.output//bpf'; fi; install bpf.h libbpf.h btf.h xsk.h libbpf_util.h bpf_helpers.h bpf_helper_defs.h bpf_tracing.h bpf_endian.h bpf_core_read.h libbpf_common.h -m 644 '/home/vagrant/bcc/libbpf-tools/.output//bpf'
if [ ! -d '/home/vagrant/bcc/libbpf-tools/.output//pkgconfig' ]; then install -d -m 755 '/home/vagrant/bcc/libbpf-tools/.output//pkgconfig'; fi; install /home/vagrant/bcc/libbpf-tools/.output//libbpf/libbpf.pc -m 644 '/home/vagrant/bcc/libbpf-tools/.output//pkgconfig'
if [ ! -d '/home/vagrant/bcc/libbpf-tools/.output/' ]; then install -d -m 755 '/home/vagrant/bcc/libbpf-tools/.output/'; fi; cp -fpR /home/vagrant/bcc/libbpf-tools/.output//libbpf/libbpf.a  '/home/vagrant/bcc/libbpf-tools/.output/'
  BPF      drsnoop.bpf.o
drsnoop.bpf.c:85:38: error: use of undeclared identifier 'BPF_F_CURRENT_CPU'
        bpf_perf_event_output(ctx, &events, BPF_F_CURRENT_CPU,
                                            ^
1 error generated.
Makefile:50: recipe for target '.output/drsnoop.bpf.o' failed
make: *** [.output/drsnoop.bpf.o] Error 1

I'm not sure what's going on here because it looks like, in the makefile, we are being sure to include libbpf/uapi/linux/bpf.h which is where it looks like BPF_F_CURRENT_CPU lives. Has this issue come up before? Is there some sort of configuration step that I'm missing here?

Still excited about contributing to the new API, just need to get it building.

@anakryiko
Copy link
Contributor

anakryiko commented May 1, 2020 via email

@rtkaratekid
Copy link
Author

Thanks for the reply @anakryiko!

I just want to come clean for a second here, and confess to you I'm also the annoying guy who's been sending questions on the iovisor-dev mailing list. Hopefully I'm not as annoying as I feel, but I'm determined to get this stuff working and you've been really great and gracious with your time to help me out! So again, thanks!
(I am still also learning how to do non-CO-RE BPF as well, with very small degrees of success. Still working on getting data from tracepoints and kprobes and coming up with a sure-fire build system.)

One more thing, just in case, try replacing #include "vmlinux.h" with
#include "vmlinux_505.h", see if that might help. Again, it shouldn't
matter, but who knows.

Okay, since that was the lowest effort thing to do (sorry, it's just friday), I tried that first. It failed at first because llvm-strip was llvm-strip-9. I symlinked to that and the build was successful using vmlinux_505.h!

So that's great, but now, when running any of the tools, e.g. # ./opensnoop I get these errors:

libbpf: Error in bpf_object__probe_name():Function not implemented(38). Couldn't load basic 'r0 = 0' BPF program.
libbpf: Error in bpf_object__probe_global_data():Function not implemented(38). Couldn't create simple array map.
libbpf: Error loading BTF: Function not implemented(38)
libbpf: Error loading .BTF into kernel: -38.
libbpf: kernel doesn't support global data
libbpf: failed to load object 'opensnoop_bpf'
libbpf: failed to load BPF skeleton 'opensnoop_bpf': -95
failed to load BPF object: -95

This makes me think that BTF is actually not supported by my kernel... So I decided to check:

This puts out what looks like a full and valid vmlinux file
$ bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h

The config here looks fine
$ cat /boot/config-5.4.0 | grep BTF
CONFIG_DEBUG_INFO_BTF=y

$ zcat /proc/config.gz | grep DEBUG_INFO

Unfortunately I didn't have CONFIG_IKCONFIG or CONFIG_IKCONFIG_PROC set so the last option wasn't helpful.

At this point, I'm hesitant to say, but it looks like kernel 5.4 isn't having it. The kernel seems to be built correctly, but doesn't support running the libbpf tools in the repo. Does this mean that these tools will only work on Linux 5.5+?

Kernel 5.4 seems to not generate the macro BPF_F_CURRENT_CPU (I've tried a few times) and it can build the tools with vmlinux_505.h, but it won't run them. Hopefully this is helpful. I'm scratching my head over this as well. Perhaps I'll try this over again with Linux 5.5 and see how it goes.

@anakryiko
Copy link
Contributor

anakryiko commented May 2, 2020 via email

@CentUser
Copy link

I run into this problem too. After reading all your posts, seems like it is caused by the clang compiler. I used centos 8 app-stream clang version:8.0.1, still get these errors.
I'll try to install the latest llvm.

@navarrothiago navarrothiago mentioned this issue Jul 31, 2021
Closed
@inevity
Copy link

inevity commented Feb 15, 2022

Ubuntu Jammy Jellyfish (development branch), use clang 14 from apt.llvm.org, build ok. Maybe use need use the bpftool from linux-common-tools , not the libtools/bin/bpftool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@anakryiko @inevity @CentUser @rtkaratekid