kernel 'asm goto' statement breaks bcc #2119
Comments
|
Could you add to the beginning of the bpf program to see whether it helps or not? In the past, the fix likes below (in arch/x86/include/asm/cpufeature.h) Find this https://reviews.llvm.org/D53765 in llvm phabricator. Looks like llvm asm-goto support still not there. |
|
The following bcc change can fix the problem: I will do a little more analysis to make sure I cover all cases before proposing a formal patch. |
|
yonghong-song's patch fixed the problem. Could you consider upstreaming it? thanks. |
|
Sure. Will upstream this or a similar version in the next couple of days. |
|
Just a thought... wouldn't it be safer to "disable" the kernel config options that require asm goto? Something like this: Thanks! |
|
The above change does work for The above approach won't work. Do you know whether there is a good way to disable |
|
Yeah, unfortunately there's not an easy way to disable the usage of But, in general, I'd prefer to fail with an explicit error, rather than silently suppress some |
|
From bcc perspective, suppressing In general, undefining config options is not a good idea either. Some config options have impact on kernel data structure size/layout. Undefining them could make bcc's view structure different from kernel. We can trigger an explicit error even with suppressing |
Fix issue #2119. Kernel 5.0 added more usages of asm goto and llvm does not support asm goto yet. This resulted in compilation error for virtually any bcc scripts. The workaround here is to redefine asm_volatile_goto to asm volatile("invalid use of asm_volatile_goto") which can pass clang. If bpf program does not use asm_volatile_goto, nothing bad will happen. The functions using asm_volatile_goto will be thrown away. If bpf program accidentally uses asm_volatile_goto, a compilation error like below will be printed out: <inline asm>:1:2: error: invalid register/token name invalid use of asm_volatile_goto ^ LLVM ERROR: Error parsing inline asm Signed-off-by: Yonghong Song <yhs@fb.com>
Fix issue #2119. Kernel 5.0 added more usages of asm goto and llvm does not support asm goto yet. This resulted in compilation error for virtually any bcc scripts. The workaround here is to redefine asm_volatile_goto to asm volatile("invalid use of asm_volatile_goto") which can pass clang. If bpf program does not use asm_volatile_goto, nothing bad will happen. The functions using asm_volatile_goto will be thrown away. If bpf program accidentally uses asm_volatile_goto, a compilation error like below will be printed out: <inline asm>:1:2: error: invalid register/token name invalid use of asm_volatile_goto ^ LLVM ERROR: Error parsing inline asm Signed-off-by: Yonghong Song <yhs@fb.com>
Fix issue #2119. Kernel 5.0 added more usages of asm goto and llvm does not support asm goto yet. This resulted in compilation error for virtually any bcc scripts. The workaround here is to redefine asm_volatile_goto to asm volatile("invalid use of asm_volatile_goto") which can pass clang. If bpf program does not use asm_volatile_goto, nothing bad will happen. The functions using asm_volatile_goto will be thrown away. If bpf program accidentally uses asm_volatile_goto, a compilation error like below will be printed out: <inline asm>:1:2: error: invalid register/token name invalid use of asm_volatile_goto ^ LLVM ERROR: Error parsing inline asm Signed-off-by: Yonghong Song <yhs@fb.com>
True.
Also true.
Honestly I think this is the best solution (workaround) so far, at least it's a safe approach and we can report an explicit error, instead of giving, potentially, a different "view" of the kernel because a particular statement isn't supported by the compiler. Thanks for merging the patch! |
Fix issue iovisor#2119. Kernel 5.0 added more usages of asm goto and llvm does not support asm goto yet. This resulted in compilation error for virtually any bcc scripts. The workaround here is to redefine asm_volatile_goto to asm volatile("invalid use of asm_volatile_goto") which can pass clang. If bpf program does not use asm_volatile_goto, nothing bad will happen. The functions using asm_volatile_goto will be thrown away. If bpf program accidentally uses asm_volatile_goto, a compilation error like below will be printed out: <inline asm>:1:2: error: invalid register/token name invalid use of asm_volatile_goto ^ LLVM ERROR: Error parsing inline asm Signed-off-by: Yonghong Song <yhs@fb.com>
|
I have same issue. I'm using CentOS 7.6, bcc-tools-0.6.1-2.el7.x86_64 and kernel 5.0.13 that I compiled by myself. /usr/share/bcc/tools/opensnoop -T |
|
Could you try this one? |
Unfortunately it didn't work either. |
|
I am using 5.0.0-1018-azure, and trying to use opensnoop-bpfcc which I installed as instructed by the readme, I get the error: error: 'asm goto' constructs are not supported yet Do I need to use the upstream package or should this be present already? |
|
@ibotheperfect could you show me your source change with a74c042 and the error message? I would like to see why it does not work? |
|
@gowenpotato this probably needs change like a74c042. That is, the |
|
I had the same issue on CentOS 7.6 with 5.3 kernel from elrepo. Then I built LLVM 9, built bcc from git and things work. |
|
installation steps mentioned here helped me |
Attempt to compile `system-probe` with Linux kernel headers 5.0 and LLVM 8
generates compilation error due to `asm goto`:
```
vagrant@ci-lenaic:~/go/src/github.com/DataDog/datadog-agent$ inv system-probe.build
In file included from ./pkg/ebpf/c/tracer-ebpf.c:17:
In file included from /usr/src/linux-headers-5.0.0-36-generic/include/linux/ptrace.h:6:
In file included from /usr/src/linux-headers-5.0.0-36-generic/include/linux/sched.h:12:
In file included from /usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/current.h:6:
In file included from /usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/percpu.h:45:
In file included from /usr/src/linux-headers-5.0.0-36-generic/include/linux/kernel.h:11:
In file included from /usr/src/linux-headers-5.0.0-36-generic/include/linux/bitops.h:19:
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/bitops.h:209:9: error: 'asm goto' constructs are not supported yet
return GEN_BINARY_RMWcc(LOCK_PREFIX __ASM_SIZE(bts), *addr, c, "Ir", nr);
^
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/rmwcc.h:60:32: note: expanded from macro 'GEN_BINARY_RMWcc'
^
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/rmwcc.h:10:28: note: expanded from macro 'RMWcc_CONCAT'
^
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/rmwcc.h:9:30: note: expanded from macro '__RMWcc_CONCAT'
^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/rmwcc.h:54:2: note: expanded from macro 'GEN_BINARY_RMWcc_6'
__GEN_RMWcc(op " %[val], " arg0, var, cc, \
^
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/rmwcc.h:21:2: note: expanded from macro '__GEN_RMWcc'
asm_volatile_goto (fullop "; j" #cc " %l[cc_label]" \
^
/usr/src/linux-headers-5.0.0-36-generic/include/linux/compiler_types.h:188:37: note: expanded from macro 'asm_volatile_goto'
^
```
`asm goto` have been required for x86 since Linux 4.17 but their use is spreading over versions.
So, with old enough Linux kernel headers, we wouldn’t have the above mentioned errors.
And support for `asm goto` has been added in LLVM 9:
https://prereleases.llvm.org/9.0.0/rc3/tools/clang/docs/ReleaseNotes.html#c-language-changes-in-clang
So, the above mentioned errors don’t happen with LLVM 9 or above.
A discussion about these compilation errors can be found there:
iovisor/bcc#2119
As a workaround, if we have the “bad” combination of Linux kernel headers and LLVM,
we can apply the same trick as the one described in iovisor/bcc#2119
in order to make the `asm goto` block compile if not used but make the compilation abort if used.
|
Thanks @palashkulsh, that helped! For anyone who's looking for a quick fix that works not only for xenial but for other Ubuntu versions, the universal snippet is: Note that for some reason tools don't appear in the PATH, you'll have to find them in |
|
Thanks @kirs , it works for me on ubuntu 18.04
|
Attempt to compile `system-probe` with Linux kernel headers 5.0 and LLVM 8
generates compilation error due to `asm goto`:
```
vagrant@ci-lenaic:~/go/src/github.com/DataDog/datadog-agent$ inv system-probe.build
In file included from ./pkg/ebpf/c/tracer-ebpf.c:17:
In file included from /usr/src/linux-headers-5.0.0-36-generic/include/linux/ptrace.h:6:
In file included from /usr/src/linux-headers-5.0.0-36-generic/include/linux/sched.h:12:
In file included from /usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/current.h:6:
In file included from /usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/percpu.h:45:
In file included from /usr/src/linux-headers-5.0.0-36-generic/include/linux/kernel.h:11:
In file included from /usr/src/linux-headers-5.0.0-36-generic/include/linux/bitops.h:19:
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/bitops.h:209:9: error: 'asm goto' constructs are not supported yet
return GEN_BINARY_RMWcc(LOCK_PREFIX __ASM_SIZE(bts), *addr, c, "Ir", nr);
^
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/rmwcc.h:60:32: note: expanded from macro 'GEN_BINARY_RMWcc'
^
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/rmwcc.h:10:28: note: expanded from macro 'RMWcc_CONCAT'
^
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/rmwcc.h:9:30: note: expanded from macro '__RMWcc_CONCAT'
^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/rmwcc.h:54:2: note: expanded from macro 'GEN_BINARY_RMWcc_6'
__GEN_RMWcc(op " %[val], " arg0, var, cc, \
^
/usr/src/linux-headers-5.0.0-36-generic/arch/x86/include/asm/rmwcc.h:21:2: note: expanded from macro '__GEN_RMWcc'
asm_volatile_goto (fullop "; j" #cc " %l[cc_label]" \
^
/usr/src/linux-headers-5.0.0-36-generic/include/linux/compiler_types.h:188:37: note: expanded from macro 'asm_volatile_goto'
^
```
`asm goto` have been required for x86 since Linux 4.17 but their use is spreading over versions.
So, with old enough Linux kernel headers, we wouldn’t have the above mentioned errors.
And support for `asm goto` has been added in LLVM 9:
https://prereleases.llvm.org/9.0.0/rc3/tools/clang/docs/ReleaseNotes.html#c-language-changes-in-clang
So, the above mentioned errors don’t happen with LLVM 9 or above.
A discussion about these compilation errors can be found there:
iovisor/bcc#2119
As a workaround, if we have the “bad” combination of Linux kernel headers and LLVM,
we can apply the same trick as the one described in iovisor/bcc#2119
in order to make the `asm goto` block compile if not used but make the compilation abort if used.
|
The workaround listed above - installing from the iovisor repo, does not work for me. Relevant system info: |
Either disable 'asm goto' in kernel or add support for llvm?
$ sudo argdist -C 't:irq:irq_handler_entry():int:args->irq'
The text was updated successfully, but these errors were encountered: