Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpf: Add bpf_rcu_read_lock() support #3976

Closed
wants to merge 7 commits into from
Closed

bpf: Add bpf_rcu_read_lock() support #3976

wants to merge 7 commits into from

Conversation

kernel-patches-bot
Copy link

Pull request for series with
subject: bpf: Add bpf_rcu_read_lock() support
version: 2
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=693112

@kernel-patches-bot
Copy link
Author

Upstream branch: e5659e4
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=693112
version: 2

@kernel-patches-bot
Copy link
Author

At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=693112 expired. Closing PR.

@kernel-patches-bot
Copy link
Author

Upstream branch: e5659e4
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=693799
version: 3

@kernel-patches-bot
Copy link
Author

Upstream branch: 15157d2
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=693799
version: 3

@kernel-patches-bot
Copy link
Author

Upstream branch: c7028aa
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=693799
version: 3

@kernel-patches-bot
Copy link
Author

Upstream branch: c7028aa
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=694190
version: 4

@kernel-patches-bot kernel-patches-bot added V4 and removed V3 labels Nov 10, 2022
@kernel-patches-bot
Copy link
Author

Upstream branch: c7028aa
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=694190
version: 4

@kernel-patches-bot
Copy link
Author

Upstream branch: c7028aa
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=694190
version: 4

@kernel-patches-bot
Copy link
Author

Upstream branch: 5b1d640
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=696243
version: 6

@kernel-patches-bot
Copy link
Author

Upstream branch: c453e64
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=696243
version: 6

@kernel-patches-bot
Copy link
Author

Upstream branch: 383f1a8
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=696243
version: 6

@kernel-patches-bot
Copy link
Author

Upstream branch: 3af43ba
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=696243
version: 6

@yonghong-song
compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))
84f6285 
Currently, without rcu attribute info in BTF, the verifier treats
rcu tagged pointer as a normal pointer. This might be a problem
for sleepable program where rcu_read_lock()/unlock() is not available.
For example, for a sleepable fentry program, if rcu protected memory
access is interleaved with a sleepable helper/kfunc, it is possible
the memory access after the sleepable helper/kfunc might be invalid
since the object might have been freed then. To prevent such cases,
introducing rcu tagging for memory accesses in verifier can help
to reject such programs.

To enable rcu tagging in BTF, during kernel compilation,
define __rcu as attribute btf_type_tag("rcu") so __rcu information can
be preserved in dwarf and btf, and later can be used for bpf prog verification.

Acked-by: KP Singh <kpsingh@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
bpf: Abstract out functions to check sleepable helpers
3e0e09c 
Abstract out two functions to check whether a particular helper
is sleepable or not for bpf_lsm and bpf_trace. These two
functions will be used later to check whether a helper is
sleepable or not in verifier. There is no functionality
change.

Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
bpf: Add kfunc bpf_rcu_read_lock/unlock()
e53c3cd 
Add two kfunc's bpf_rcu_read_lock() and bpf_rcu_read_unlock(). These two kfunc's
can be used for all program types. A new kfunc hook type BTF_KFUNC_HOOK_GENERIC
is added which corresponds to prog type BPF_PROG_TYPE_UNSPEC, indicating the
kfunc intends to be used for all prog types.

Signed-off-by: Yonghong Song <yhs@fb.com>
@kernel-patches-bot
Copy link
Author

Upstream branch: 98b2afc
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=696243
version: 6

@yonghong-song
bpf: Add bpf_rcu_read_lock() verifier support
99e594c 
To simplify the design and support the common practice, no nested
bpf_rcu_read_lock() is allowed. A new bpf_type_flag MEM_RCU is added to
indicate a PTR_TO_BTF_ID object access needing rcu_read_lock protection.
Note that rcu protection is not needed for non-sleepable program, but
it is supported to make cross-sleepable/nonsleepable development easier.
For sleepable program, the following insns can be inside the rcu
lock region:
  - any non call insns except BPF_ABS/BPF_IND
  - non sleepable helpers or kfuncs
The rcu pointer will be invalidated at bpf_rcu_read_unlock() so it
cannot be used outside the current rcu read lock region.
Also, bpf_*_storage_get() helper's 5th hidden argument (for memory
allocation flag) should be GFP_ATOMIC.

If a pointer (PTR_TO_BTF_ID) is marked as rcu, then any use of
this pointer and the load which gets this pointer needs to be
protected by bpf_rcu_read_lock(). The following shows a couple
of examples:
  struct task_struct {
        ...
        struct task_struct __rcu        *real_parent;
        struct css_set __rcu            *cgroups;
        ...
  };
  struct css_set {
        ...
        struct cgroup *dfl_cgrp;
        ...
  }
  ...
  task = bpf_get_current_task_btf();
  cgroups = task->cgroups;
  dfl_cgroup = cgroups->dfl_cgrp;
  ... using dfl_cgroup ...

The bpf_rcu_read_lock/unlock() should be added like below to
avoid verification failures.
  task = bpf_get_current_task_btf();
  bpf_rcu_read_lock();
  cgroups = task->cgroups;
  dfl_cgroup = cgroups->dfl_cgrp;
  bpf_rcu_read_unlock();
  ... using dfl_cgroup ...

The following is another example for task->real_parent.
  task = bpf_get_current_task_btf();
  bpf_rcu_read_lock();
  real_parent = task->real_parent;
  ... bpf_task_storage_get(&map, real_parent, 0, 0);
  bpf_rcu_read_unlock();

There is another case observed in selftest bpf_iter_ipv6_route.c:
  struct fib6_info *rt = ctx->rt;
  ...
  fib6_nh = &rt->fib6_nh[0]; // Not rcu protected
  ...
  if (rt->nh)
    fib6_nh = &nh->nh_info->fib6_nh; // rcu protected
  ...
  ... using fib6_nh ...
Note that the use of fib6_nh is tag with rcu in one path but not in the
other path. Current verification will fail since the same insn cannot
be used with different pointer types. The above use case is a valid
one so the verifier is changed to ignore MEM_RCU type tag
in such cases.

Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
bpf: Enable sleeptable support for cgrp local storage
932a6ae 
With proper bpf_rcu_read_lock() support, sleepable support for cgrp local
storage can be enabled as typical use case task->cgroups->dfl_cgrp
can be protected with bpf_rcu_read_lock().

Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
selftests/bpf: Add tests for bpf_rcu_read_lock()
a63aeca 
Add a few positive/negative tests to test bpf_rcu_read_lock()
and its corresponding verifier support.

Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
selftests/bpf: Add rcu_read_lock test to s390x/aarch64 deny lists
a03dd5f 
The new rcu_read_lock test failed on s390x with the following error message:

  ...
  test_rcu_read_lock:PASS:join_cgroup /rcu_read_lock 0 nsec
  test_local_storage:PASS:skel_open 0 nsec
  libbpf: prog 'cgrp_succ': failed to find kernel BTF type ID of '__s390x_sys_getpgid': -3
  libbpf: prog 'cgrp_succ': failed to prepare load attributes: -3
  libbpf: prog 'cgrp_succ': failed to load: -3
  libbpf: failed to load object 'rcu_read_lock'
  libbpf: failed to load BPF skeleton 'rcu_read_lock': -3
  test_local_storage:FAIL:skel_load unexpected error: -3 (errno 3)
  ...

It failed on aarch64 with the following error message due to inadequate
trampoline support.

  ...
  test_rcu_read_lock:PASS:join_cgroup /rcu_read_lock 0 nsec
  test_local_storage:PASS:skel_open 0 nsec
  test_local_storage:PASS:skel_load 0 nsec
  libbpf: prog 'cgrp_succ': failed to attach: ERROR: strerror_r(-524)=22
  libbpf: prog 'cgrp_succ': failed to auto-attach: -524
  test_local_storage:FAIL:skel_attach unexpected error: -524 (errno 524)
  ...

So add the test to s390x and aarch64 deny lists.

Signed-off-by: Yonghong Song <yhs@fb.com>
@kernel-patches-bot
Copy link
Author

At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=696243 expired. Closing PR.

@kernel-patches-bot kernel-patches-bot deleted the series/691523=>bpf-next branch November 20, 2022 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bpf-next changes-requested V6
Projects
None yet
Milestone
No milestone
2 participants
@kernel-patches-bot @yonghong-song