Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tetragon: Add support for kprobe multi interface #79

Merged
merged 12 commits into from
Oct 10, 2022

Conversation

olsajiri
Copy link
Contributor

@olsajiri olsajiri commented May 25, 2022

Adding support to attach kprobes with multi kprobe link if
there's more than one kprobe defined and if there's kernel
support for multi kprobe link in kernel.

The multi kprobe interface attaches multiple kprobes much
faster than doing it one by one. Also we safe verifier time,
because the generic kprobe program is loaded just once.

perf stat -n for tracing tests:

kprobe multi:

 Performance counter stats for 'go test -count=1 -run . ./pkg/sensors/tracing/ -test.v':

     143.409080255 seconds time elapsed

     143.082559000 seconds user
      69.401370000 seconds sys

base:

 Performance counter stats for 'go test -count=1 -run . ./pkg/sensors/tracing/ -test.v':

     164.496162582 seconds time elapsed

     142.647112000 seconds user
      82.115091000 seconds sys

There's no change to spec config, the multi kprobe is used
once we detect there's more then one kprobe defined.

Signed-off-by: Jiri Olsa jolsa@kernel.org

@olsajiri olsajiri force-pushed the kprobe_multi branch 3 times, most recently from 3d0f8f9 to 066c678 Compare May 30, 2022 14:17
@olsajiri olsajiri force-pushed the kprobe_multi branch 7 times, most recently from 1a44654 to 064be48 Compare August 17, 2022 13:14
@olsajiri olsajiri force-pushed the kprobe_multi branch 5 times, most recently from 893f390 to e79870f Compare September 29, 2022 07:17
@olsajiri olsajiri marked this pull request as ready for review September 29, 2022 07:55
@olsajiri olsajiri requested a review from a team as a code owner September 29, 2022 07:55
Copy link
Contributor

@kkourt kkourt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! I have a question about supporting multiple tracing policies, but other than that looks great!

bpf/process/types/basic.h Show resolved Hide resolved
pkg/sensors/tracing/generickprobe.go Show resolved Hide resolved
pkg/sensors/tracing/generickprobe.go Show resolved Hide resolved
pkg/sensors/tracing/generickprobe.go Outdated Show resolved Hide resolved
@olsajiri olsajiri force-pushed the kprobe_multi branch 2 times, most recently from de59a42 to 9680437 Compare October 3, 2022 08:28
@olsajiri
Copy link
Contributor Author

olsajiri commented Oct 3, 2022

@kkourt I added the review changes and also added --disable-kprobe-multi option ;-)

Copy link
Contributor

@kkourt kkourt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Add support to load data via MapLoad interfae into specific index.
This way we can set up specific the array map elements.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Adding support to use specific element of config and filter map.
Keeping the index initialized to zero for now, it will be set up
by multi kprobe init code in following patches to read specific
config and filter data.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
When we use single kprobe program instead of having one for each
kprobe attachment, we need to make sure the retprobe_map id stays
unique otherwise we can get mixed data.

Adding extra id - function id - to the retprobe_map key to ensure
that.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Moving global variables init to the top, so it's easier to
add multi kprobe code in following changes.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Adding get_attach_cookie helper support, so it can be
used in following changes.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Adding multi kprobe objects built from generic with __MULTI_KPROBE
defined.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Loading proper index in multi kprobe objects from config map provided
by user space configuration process. Once it's initialized it will read
specific config/filter map element.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
The multi kprobe program needs to be placed in kprobe.multi section
so it is loaded with special multi kprobe attach type.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Adding HasKprobeMulti that detects the support for multi
kprobe link.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Adding support to attach kprobes with multi kprobe link if
there's more than one kprobe dfined and if there's kernel
support for multi kprobe link in kernel.

Adding both spec handler and loading code changes together.

There are following benhaviour changes to standard kprobes:

  - only one program is loaded for all kprobes and it's
    pinned under 'multi_kprobe' pin file
  - only one program is loaded for all return kprobes and
    pinned under 'multi_retkprobe' pin file
  - there's single retprobe_map map instance

Each config needs to have extra element in 'config_map' and
'filter_map'. Limiting size of these to 100 at the moment,
this can be made bigger when we have wildcard support in.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Adding --disable-kprobe-multi option to allow user disable kprobe
multi interface and have a choice to go with generic kprobes for
whatever reason.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Multi kprobe support is still not widely around, skipping the
object for now.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
@kkourt kkourt merged commit a2a3fd4 into cilium:main Oct 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants