Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fprobe: Introduce fprobe function entry/exit probe #2700

Closed
wants to merge 13 commits into from

Conversation

kernel-patches-bot
Copy link

Pull request for series with
subject: fprobe: Introduce fprobe function entry/exit probe
version: 11
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=622705

@kernel-patches-bot
Copy link
Author

Master branch: d3b351f
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=622705
version: 11

@kernel-patches-bot
Copy link
Author

Master branch: d3b351f
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: 2486ab4
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: 8fa42d7
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: f98d6dd
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: cbdaf71
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: 6585abe
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: aaccdf9
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: ad13baf
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: 7cda76d
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: a50cbac
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: a50cbac
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: e0999c8
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

Nobody and others added 13 commits March 17, 2022 20:39
Adding ftrace_set_filter_ips function to be able to set filter on
multiple ip addresses at once.

With the kprobe multi attach interface we have cases where we need to
initialize ftrace_ops object with thousands of functions, so having
single function diving into ftrace_hash_move_and_update_ops with
ftrace_lock is faster.

The functions ips are passed as unsigned long array with count.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
The fprobe is a wrapper API for ftrace function tracer.
Unlike kprobes, this probes only supports the function entry, but this
can probe multiple functions by one fprobe. The usage is similar, user
will set their callback to fprobe::entry_handler and call
register_fprobe*() with probed functions.
There are 3 registration interfaces,

 - register_fprobe() takes filtering patterns of the functin names.
 - register_fprobe_ips() takes an array of ftrace-location addresses.
 - register_fprobe_syms() takes an array of function names.

The registered fprobes can be unregistered with unregister_fprobe().
e.g.

struct fprobe fp = { .entry_handler = user_handler };
const char *targets[] = { "func1", "func2", "func3"};
...

ret = register_fprobe_syms(&fp, targets, ARRAY_SIZE(targets));

...

unregister_fprobe(&fp);

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Add a return hook framework which hooks the function return. Most of the
logic came from the kretprobe, but this is independent from kretprobe.

Note that this is expected to be used with other function entry hooking
feature, like ftrace, fprobe, adn kprobes. Eventually this will replace
the kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment,
this is just an additional hook.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Add rethook for x86 implementation. Most of the code has been copied from
kretprobes on x86.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Add rethook arm64 implementation. Most of the code has been copied from
kretprobes on arm64.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Add rethook powerpc64 implementation. Most of the code has been copied from
kretprobes on powerpc64.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Add rethook arm implementation. Most of the code has been copied from
kretprobes on arm.
Since the arm's ftrace implementation is a bit special, this needs a
special care using from fprobe.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Add exit_handler to fprobe. fprobe + rethook allows us to hook the kernel
function return. The rethook will be enabled only if the
fprobe::exit_handler is set.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Add a sample program for the fprobe. The sample_fprobe puts a fprobe on
kernel_clone() by default. This dump stack and some called address info
at the function entry and exit.

The sample_fprobe.ko gets 2 parameters.
- symbol: you can specify the comma separated symbols or wildcard symbol
  pattern (in this case you can not use comma)
- stackdump: a bool value to enable or disable stack dump in the fprobe
  handler.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Introduce FPROBE_FL_KPROBE_SHARED flag for sharing fprobe callback with
kprobes safely from the viewpoint of recursion.

Since the recursion safety of the fprobe (and ftrace) is a bit different
from the kprobes, this may cause an issue if user wants to run the same
code from the fprobe and the kprobes.

The kprobes has per-cpu 'current_kprobe' variable which protects the
kprobe handler from recursion in any case. On the other hand, the fprobe
uses only ftrace_test_recursion_trylock(), which will allow interrupt
context calls another (or same) fprobe during the fprobe user handler is
running.

This is not a matter in cases if the common callback shared among the
kprobes and the fprobe has its own recursion detection, or it can handle
the recursion in the different contexts (normal/interrupt/NMI.)
But if it relies on the 'current_kprobe' recursion lock, it has to check
kprobe_running() and use kprobe_busy_*() APIs.

Fprobe has FPROBE_FL_KPROBE_SHARED flag to do this. If your common callback
code will be shared with kprobes, please set FPROBE_FL_KPROBE_SHARED
*before* registering the fprobe, like;

 fprobe.flags = FPROBE_FL_KPROBE_SHARED;

 register_fprobe(&fprobe, "func*", NULL);

This will protect your common callback from the nested call.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Add a documentation of fprobe for the user who needs
this interface.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Add a KUnit based selftest for fprobe interface.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
@kernel-patches-bot
Copy link
Author

Master branch: e0999c8
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

@kernel-patches-bot
Copy link
Author

Master branch: 5a5c11e
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
version: 12

Pull request is NOT updated. Failed to apply https://patchwork.kernel.org/project/netdevbpf/list/?series=623535
error message:

Cmd('git') failed due to: exit code(128)
  cmdline: git am -3
  stdout: 'Applying: ftrace: Add ftrace_set_filter_ips function
Using index info to reconstruct a base tree...
M	include/linux/ftrace.h
M	kernel/trace/ftrace.c
Falling back to patching base and 3-way merge...
No changes -- Patch already applied.
Applying: fprobe: Add ftrace based probe APIs
Using index info to reconstruct a base tree...
M	kernel/trace/Kconfig
M	kernel/trace/Makefile
Falling back to patching base and 3-way merge...
CONFLICT (add/add): Merge conflict in kernel/trace/fprobe.c
Auto-merging kernel/trace/fprobe.c
Auto-merging kernel/trace/Makefile
CONFLICT (content): Merge conflict in kernel/trace/Makefile
Auto-merging kernel/trace/Kconfig
CONFLICT (content): Merge conflict in kernel/trace/Kconfig
CONFLICT (add/add): Merge conflict in include/linux/fprobe.h
Auto-merging include/linux/fprobe.h
Patch failed at 0002 fprobe: Add ftrace based probe APIs
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".'
  stderr: 'error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch'

conflict:

diff --cc include/linux/fprobe.h
index 1c2bde0ead73,2ba099aff041..000000000000
--- a/include/linux/fprobe.h
+++ b/include/linux/fprobe.h
@@@ -5,16 -5,13 +5,23 @@@
  
  #include <linux/compiler.h>
  #include <linux/ftrace.h>
++<<<<<<< HEAD
 +#include <linux/rethook.h>
++=======
++>>>>>>> fprobe: Add ftrace based probe APIs
  
  /**
   * struct fprobe - ftrace based probe.
   * @ops: The ftrace_ops.
   * @nmissed: The counter for missing events.
   * @flags: The status flag.
++<<<<<<< HEAD
 + * @rethook: The rethook data structure. (internal data)
   * @entry_handler: The callback function for function entry.
 + * @exit_handler: The callback function for function exit.
++=======
++ * @entry_handler: The callback function for function entry.
++>>>>>>> fprobe: Add ftrace based probe APIs
   */
  struct fprobe {
  #ifdef CONFIG_FUNCTION_TRACER
@@@ -28,31 -25,16 +35,42 @@@
  #endif
  	unsigned long		nmissed;
  	unsigned int		flags;
++<<<<<<< HEAD
 +	struct rethook		*rethook;
 +
 +	void (*entry_handler)(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs);
 +	void (*exit_handler)(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs);
 +};
 +
 +/* This fprobe is soft-disabled. */
 +#define FPROBE_FL_DISABLED	1
 +
 +/*
 + * This fprobe handler will be shared with kprobes.
 + * This flag must be set before registering.
 + */
 +#define FPROBE_FL_KPROBE_SHARED	2
 +
++=======
+ 	void (*entry_handler)(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs);
+ };
+ 
+ #define FPROBE_FL_DISABLED	1
+ 
++>>>>>>> fprobe: Add ftrace based probe APIs
  static inline bool fprobe_disabled(struct fprobe *fp)
  {
  	return (fp) ? fp->flags & FPROBE_FL_DISABLED : false;
  }
  
++<<<<<<< HEAD
 +static inline bool fprobe_shared_with_kprobes(struct fprobe *fp)
 +{
 +	return (fp) ? fp->flags & FPROBE_FL_KPROBE_SHARED : false;
 +}
 +
++=======
++>>>>>>> fprobe: Add ftrace based probe APIs
  #ifdef CONFIG_FPROBE
  int register_fprobe(struct fprobe *fp, const char *filter, const char *notfilter);
  int register_fprobe_ips(struct fprobe *fp, unsigned long *addrs, int num);
diff --cc kernel/trace/Kconfig
index 99dd4ca63d68,7ce31abc542b..000000000000
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@@ -251,14 -240,11 +251,22 @@@ config FPROB
  	bool "Kernel Function Probe (fprobe)"
  	depends on FUNCTION_TRACER
  	depends on DYNAMIC_FTRACE_WITH_REGS
++<<<<<<< HEAD
 +	depends on HAVE_RETHOOK
 +	select RETHOOK
 +	default n
 +	help
 +	  This option enables kernel function probe (fprobe) based on ftrace.
 +	  The fprobe is similar to kprobes, but probes only for kernel function
 +	  entries and exits. This also can probe multiple functions by one
 +	  fprobe.
++=======
+ 	default n
+ 	help
+ 	  This option enables kernel function probe (fprobe) based on ftrace,
+ 	  which is similar to kprobes, but probes only for kernel function
+ 	  entries and it can probe multiple functions by one fprobe.
++>>>>>>> fprobe: Add ftrace based probe APIs
  
  	  If unsure, say N.
  
diff --cc kernel/trace/Makefile
index c6f11a139eac,79255f9de9a4..000000000000
--- a/kernel/trace/Makefile
+++ b/kernel/trace/Makefile
@@@ -98,7 -98,6 +98,10 @@@ obj-$(CONFIG_UPROBE_EVENTS) += trace_up
  obj-$(CONFIG_BOOTTIME_TRACING) += trace_boot.o
  obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
  obj-$(CONFIG_FPROBE) += fprobe.o
++<<<<<<< HEAD
 +obj-$(CONFIG_RETHOOK) += rethook.o
++=======
++>>>>>>> fprobe: Add ftrace based probe APIs
  
  obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
  
diff --cc kernel/trace/fprobe.c
index 8b2dd5b9dcd1,7e8ceee339a0..000000000000
--- a/kernel/trace/fprobe.c
+++ b/kernel/trace/fprobe.c
@@@ -8,22 -8,12 +8,31 @@@
  #include <linux/fprobe.h>
  #include <linux/kallsyms.h>
  #include <linux/kprobes.h>
++<<<<<<< HEAD
 +#include <linux/rethook.h>
 +#include <linux/slab.h>
 +#include <linux/sort.h>
 +
 +#include "trace.h"
 +
 +struct fprobe_rethook_node {
 +	struct rethook_node node;
 +	unsigned long entry_ip;
 +};
 +
 +static void fprobe_handler(unsigned long ip, unsigned long parent_ip,
 +			   struct ftrace_ops *ops, struct ftrace_regs *fregs)
 +{
 +	struct fprobe_rethook_node *fpr;
 +	struct rethook_node *rh;
++=======
+ #include <linux/slab.h>
+ #include <linux/sort.h>
+ 
+ static void fprobe_handler(unsigned long ip, unsigned long parent_ip,
+ 			   struct ftrace_ops *ops, struct ftrace_regs *fregs)
+ {
++>>>>>>> fprobe: Add ftrace based probe APIs
  	struct fprobe *fp;
  	int bit;
  
@@@ -40,51 -30,10 +49,57 @@@
  	if (fp->entry_handler)
  		fp->entry_handler(fp, ip, ftrace_get_regs(fregs));
  
++<<<<<<< HEAD
 +	if (fp->exit_handler) {
 +		rh = rethook_try_get(fp->rethook);
 +		if (!rh) {
 +			fp->nmissed++;
 +			goto out;
 +		}
 +		fpr = container_of(rh, struct fprobe_rethook_node, node);
 +		fpr->entry_ip = ip;
 +		rethook_hook(rh, ftrace_get_regs(fregs), true);
 +	}
 +
 +out:
++=======
++>>>>>>> fprobe: Add ftrace based probe APIs
  	ftrace_test_recursion_unlock(bit);
  }
  NOKPROBE_SYMBOL(fprobe_handler);
  
++<<<<<<< HEAD
 +static void fprobe_kprobe_handler(unsigned long ip, unsigned long parent_ip,
 +				  struct ftrace_ops *ops, struct ftrace_regs *fregs)
 +{
 +	struct fprobe *fp = container_of(ops, struct fprobe, ops);
 +
 +	if (unlikely(kprobe_running())) {
 +		fp->nmissed++;
 +		return;
 +	}
 +	kprobe_busy_begin();
 +	fprobe_handler(ip, parent_ip, ops, fregs);
 +	kprobe_busy_end();
 +}
 +
 +static void fprobe_exit_handler(struct rethook_node *rh, void *data,
 +				struct pt_regs *regs)
 +{
 +	struct fprobe *fp = (struct fprobe *)data;
 +	struct fprobe_rethook_node *fpr;
 +
 +	if (!fp || fprobe_disabled(fp))
 +		return;
 +
 +	fpr = container_of(rh, struct fprobe_rethook_node, node);
 +
 +	fp->exit_handler(fp, fpr->entry_ip, regs);
 +}
 +NOKPROBE_SYMBOL(fprobe_exit_handler);
 +
++=======
++>>>>>>> fprobe: Add ftrace based probe APIs
  /* Convert ftrace location address from symbols */
  static unsigned long *get_ftrace_locations(const char **syms, int num)
  {
@@@ -124,55 -73,10 +139,62 @@@ error
  static void fprobe_init(struct fprobe *fp)
  {
  	fp->nmissed = 0;
++<<<<<<< HEAD
 +	if (fprobe_shared_with_kprobes(fp))
 +		fp->ops.func = fprobe_kprobe_handler;
 +	else
 +		fp->ops.func = fprobe_handler;
 +	fp->ops.flags |= FTRACE_OPS_FL_SAVE_REGS;
 +}
 +
 +static int fprobe_init_rethook(struct fprobe *fp, int num)
 +{
 +	int i, size;
 +
 +	if (num < 0)
 +		return -EINVAL;
 +
 +	if (!fp->exit_handler) {
 +		fp->rethook = NULL;
 +		return 0;
 +	}
 +
 +	/* Initialize rethook if needed */
 +	size = num * num_possible_cpus() * 2;
 +	if (size < 0)
 +		return -E2BIG;
 +
 +	fp->rethook = rethook_alloc((void *)fp, fprobe_exit_handler);
 +	for (i = 0; i < size; i++) {
 +		struct rethook_node *node;
 +
 +		node = kzalloc(sizeof(struct fprobe_rethook_node), GFP_KERNEL);
 +		if (!node) {
 +			rethook_free(fp->rethook);
 +			fp->rethook = NULL;
 +			return -ENOMEM;
 +		}
 +		rethook_add_node(fp->rethook, node);
 +	}
 +	return 0;
 +}
 +
 +static void fprobe_fail_cleanup(struct fprobe *fp)
 +{
 +	if (fp->rethook) {
 +		/* Don't need to cleanup rethook->handler because this is not used. */
 +		rethook_free(fp->rethook);
 +		fp->rethook = NULL;
 +	}
 +	ftrace_free_filter(&fp->ops);
 +}
 +
++=======
+ 	fp->ops.func = fprobe_handler;
+ 	fp->ops.flags |= FTRACE_OPS_FL_SAVE_REGS;
+ }
+ 
++>>>>>>> fprobe: Add ftrace based probe APIs
  /**
   * register_fprobe() - Register fprobe to ftrace by pattern.
   * @fp: A fprobe data structure to be registered.
@@@ -186,7 -90,6 +208,10 @@@
   */
  int register_fprobe(struct fprobe *fp, const char *filter, const char *notfilter)
  {
++<<<<<<< HEAD
 +	struct ftrace_hash *hash;
++=======
++>>>>>>> fprobe: Add ftrace based probe APIs
  	unsigned char *str;
  	int ret, len;
  
@@@ -211,21 -114,10 +236,28 @@@
  			goto out;
  	}
  
++<<<<<<< HEAD
 +	/* TODO:
 +	 * correctly calculate the total number of filtered symbols
 +	 * from both filter and notfilter.
 +	 */
 +	hash = fp->ops.local_hash.filter_hash;
 +	if (WARN_ON_ONCE(!hash))
 +		goto out;
 +
 +	ret = fprobe_init_rethook(fp, (int)hash->count);
 +	if (!ret)
 +		ret = register_ftrace_function(&fp->ops);
 +
 +out:
 +	if (ret)
 +		fprobe_fail_cleanup(fp);
++=======
+ 	ret = register_ftrace_function(&fp->ops);
+ out:
+ 	if (ret)
+ 		ftrace_free_filter(&fp->ops);
++>>>>>>> fprobe: Add ftrace based probe APIs
  	return ret;
  }
  EXPORT_SYMBOL_GPL(register_fprobe);
@@@ -253,15 -145,12 +285,23 @@@ int register_fprobe_ips(struct fprobe *
  	fprobe_init(fp);
  
  	ret = ftrace_set_filter_ips(&fp->ops, addrs, num, 0, 0);
++<<<<<<< HEAD
 +	if (ret)
 +		return ret;
 +
 +	ret = fprobe_init_rethook(fp, num);
++=======
++>>>>>>> fprobe: Add ftrace based probe APIs
  	if (!ret)
  		ret = register_ftrace_function(&fp->ops);
  
  	if (ret)
++<<<<<<< HEAD
 +		fprobe_fail_cleanup(fp);
++=======
+ 		ftrace_free_filter(&fp->ops);
+ 
++>>>>>>> fprobe: Add ftrace based probe APIs
  	return ret;
  }
  EXPORT_SYMBOL_GPL(register_fprobe_ips);
@@@ -312,20 -201,10 +352,27 @@@ int unregister_fprobe(struct fprobe *fp
  	if (!fp || fp->ops.func != fprobe_handler)
  		return -EINVAL;
  
++<<<<<<< HEAD
 +	/*
 +	 * rethook_free() starts disabling the rethook, but the rethook handlers
 +	 * may be running on other processors at this point. To make sure that all
 +	 * current running handlers are finished, call unregister_ftrace_function()
 +	 * after this.
 +	 */
 +	if (fp->rethook)
 +		rethook_free(fp->rethook);
 +
 +	ret = unregister_ftrace_function(&fp->ops);
 +	if (ret < 0)
 +		return ret;
 +
 +	ftrace_free_filter(&fp->ops);
++=======
+ 	ret = unregister_ftrace_function(&fp->ops);
+ 
+ 	if (!ret)
+ 		ftrace_free_filter(&fp->ops);
++>>>>>>> fprobe: Add ftrace based probe APIs
  
  	return ret;
  }

@kernel-patches-bot
Copy link
Author

At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=623535 irrelevant now. Closing PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants