Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ABI0 support to Go crypto/tls eBPF tracer #1169

Merged
merged 44 commits into from Jul 5, 2022
Merged

Conversation

mertyildiran
Copy link
Member

@mertyildiran mertyildiran commented Jun 27, 2022

Adds vmlinux.h for x86 and ARM64 which is generated by the command below in the machines with the corresponding architecture:

bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux_x86.h
bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux_arm64.h

that contains the declaration of struct task_struct. The struct is needed to get the Goroutine ID from thread-local storage. It's described in the ABI0 specs.

Go ABI0 (<=1.16) specification
https://go.dev/doc/asm

task_struct, CO-RE and cilium/ebpf

The heap-like area for struct task_struct requires ring buffer. It means that; the PR increases the minimum supported Linux kernel version from 4.10 to 5.8.

We read task_struct using BPF_CORE_READ. Its member thread is a thread_struct and thread_struct is different in kernel versions older than or equal to 4.6.

But cilium/ebpf does not support .kconfig:

extern int LINUX_KERNEL_VERSION __kconfig;

Therefore; for now, we build object files per kernel version and load them accordingly.

Possible TODOs for this subject;

  • cilium/cilium tags images per kernel version. So, do the same thing.
  • Wait cilium/ebpf to add .kconfig support. (uncertain)
  • Don't use cilium/ebpf or fork it.

Check Lists

Tested

  • OpenSSL works on x86_64
  • OpenSSL works on ARM64
  • Go ABI0 works on x86_64
  • Go ABI0 works on ARM64
  • Go ABIInternal works on x86_64
  • Go ABIInternal works on ARM64

Build Strategies

  • BUILDARCH=amd64 TARGETARCH=amd64
  • BUILDARCH=arm64v8 TARGETARCH=arm64v8
  • BUILDARCH=amd64 TARGETARCH=arm64v8
  • BUILDARCH=arm64v8 TARGETARCH=amd64

Test Reference

kubeshark/sock-shop-demo#23

Screenshot from 2022-06-30 21-34-21

Screenshot from 2022-07-01 00-26-59

@mertyildiran mertyildiran added worker Indicates an issue or PR is related to the worker. eBPF Indicates an issue or PR is related to the Extended Berkeley Packet Filter (eBPF). labels Jun 27, 2022
@mertyildiran mertyildiran self-assigned this Jun 27, 2022
@codecov
Copy link

codecov bot commented Jun 27, 2022

Codecov Report

Merging #1169 (17c53ae) into develop (f3a6b3a) will decrease coverage by 17.01%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##           develop    #1169       +/-   ##
============================================
- Coverage    45.17%   28.16%   -17.02%     
============================================
  Files          137       64       -73     
  Lines        13255     5252     -8003     
============================================
- Hits          5988     1479     -4509     
+ Misses        6839     3686     -3153     
+ Partials       428       87      -341     
Impacted Files Coverage Δ
cli/cmd/checkRunner.go
tap/extensions/amqp/spec091.go
tap/extensions/redis/errors.go
cli/mizu/version/versionCheck.go
cli/config/configStructs/logsConfig.go
cli/cmd/check/kubernetesVersion.go
tap/extensions/kafka/request.go
tap/extensions/kafka/main.go
tap/extensions/kafka/reflect.go
tap/extensions/amqp/read.go
... and 64 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f3a6b3a...17c53ae. Read the comment docs.

@mertyildiran mertyildiran marked this pull request as ready for review July 1, 2022 18:52
tap/tlstapper/bpf/go_uprobes.c Outdated Show resolved Hide resolved
tap/tlstapper/bpf/go_uprobes.c Outdated Show resolved Hide resolved
tap/tlstapper/go_hooks.go Outdated Show resolved Hide resolved
M. Mert Yildiran added 6 commits July 4, 2022 20:00
… kernel versions older than 5.8

Also;
Add legacy version of thread_struct: thread_struct___v46
Build an additional object file for the kernel versions older than or equal to 4.6 and load them accordingly.
Add github.com/moby/moby
@mertyildiran
Copy link
Member Author

PR description is updated.

Copy link
Contributor

@nimrod-up9 nimrod-up9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, except maybe remove support for kernel 4.6. See slack discussion.

nimrod-up9
nimrod-up9 previously approved these changes Jul 5, 2022
@mertyildiran mertyildiran merged commit 52c9251 into develop Jul 5, 2022
@mertyildiran mertyildiran deleted the feat/ebpf-go-abi0 branch July 5, 2022 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
eBPF Indicates an issue or PR is related to the Extended Berkeley Packet Filter (eBPF). worker Indicates an issue or PR is related to the worker.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants