Implement task_local_storage #271

kernel-patches-bot · 2020-10-27T17:09:56Z

Pull request for series with
subject: Implement task_local_storage
version: 1
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=371615

kernel-patches-bot · 2020-10-27T17:09:57Z

Master branch: 3cb12d2
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=371615
version: 1

kernel-patches-bot · 2020-11-02T18:06:20Z

At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=371615 expired. Closing PR.

kernel-patches-bot · 2020-11-03T20:49:24Z

Master branch: 8aaeed8
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=376633
version: 2

kernel-patches-bot · 2020-11-04T02:07:51Z

Master branch: d0b3d2d
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=376633
version: 2

Usage of spin locks was not allowed for tracing programs due to insufficient preemption checks. The verifier does not currently prevent LSM programs from using spin locks, but the helpers are not exposed via bpf_lsm_func_proto. Based on the discussion in [1], non-sleepable LSM programs should be able to use bpf_spin_{lock, unlock}. Sleepable LSM programs can be preempted which means that allowng spin locks will need more work (disabling preemption and the verifier ensuring that no sleepable helpers are called when a spin lock is held). [1]: https://lore.kernel.org/bpf/20201103153132.2717326-1-kpsingh@chromium.org/T/#md601a053229287659071600d3483523f752cd2fb Signed-off-by: KP Singh <kpsingh@google.com> Acked-by: Song Liu <songliubraving@fb.com>

Similar to bpf_local_storage for sockets and inodes add local storage for task_struct. The life-cycle of storage is managed with the life-cycle of the task_struct. i.e. the storage is destroyed along with the owning task with a callback to the bpf_task_storage_free from the task_free LSM hook. The BPF LSM allocates an __rcu pointer to the bpf_local_storage in the security blob which are now stackable and can co-exist with other LSMs. The userspace map operations can be done by using a pid fd as a key passed to the lookup, update and delete operations. Acked-by: Song Liu <songliubraving@fb.com> Signed-off-by: KP Singh <kpsingh@google.com>

Updates the bpf_probe_map_type API to also support BPF_MAP_TYPE_TASK_STORAGE similar to other local storage maps. Acked-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: KP Singh <kpsingh@google.com>

Updates the binary to handle the BPF_MAP_TYPE_TASK_STORAGE as "task_storage" for printing and parsing. Also updates the documentation and bash completion Acked-by: Song Liu <songliubraving@fb.com> Signed-off-by: KP Singh <kpsingh@google.com>

The currently available bpf_get_current_task returns an unsigned integer which can be used along with BPF_CORE_READ to read data from the task_struct but still cannot be used as an input argument to a helper that accepts an ARG_PTR_TO_BTF_ID of type task_struct. In order to implement this helper a new return type, RET_PTR_TO_BTF_ID, is added. This is similar to RET_PTR_TO_BTF_ID_OR_NULL but does not require checking the nullness of returned pointer. Acked-by: Song Liu <songliubraving@fb.com> Signed-off-by: KP Singh <kpsingh@google.com>

The {inode,sk}_storage_result checking if the correct value was retrieved was being clobbered unconditionally by the return value of the bpf_{inode,sk}_storage_delete call. Also, consistently use the newly added BPF_LOCAL_STORAGE_GET_F_CREATE flag. Acked-by: Song Liu <songliubraving@fb.com> Fixes: cd324d7 ("bpf: Add selftests for local_storage") Signed-off-by: KP Singh <kpsingh@google.com>

With the fixing of BTF pruning of embedded types being fixed, the test can be simplified to use vmlinux.h Acked-by: Song Liu <songliubraving@fb.com> Signed-off-by: KP Singh <kpsingh@google.com>

The test exercises the syscall based map operations by creating a pidfd for the current process. For verifying kernel / LSM functionality, the test implements a simple MAC policy which denies an executable from unlinking itself. The LSM program bprm_committed_creds sets a task_local_storage with a pointer to the inode. This is then used to detect if the task is trying to unlink itself in the inode_unlink LSM hook. The test copies /bin/rm to /tmp and executes it in a child thread with the intention of deleting itself. A successful test should prevent the the running executable from deleting itself. The bpf programs are also updated to call bpf_spin_{lock, unlock} to trigger the verfier checks for spin locks. The temporary file is cleaned up later in the test. Signed-off-by: KP Singh <kpsingh@google.com>

Use the check_syscall_operations added for task_local_storage to exercise syscall operations for other local storage maps: * Check the absence of an element for the given fd. * Create a new element, retrieve and compare its value. * Delete the element and check again for absence. Signed-off-by: KP Singh <kpsingh@google.com>

kernel-patches-bot · 2020-11-05T20:33:42Z

Master branch: d0b3d2d
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=378247
version: 4

kernel-patches-bot · 2020-11-05T23:01:25Z

Master branch: d0b3d2d
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=378545
version: 5

Pull request is NOT updated. Failed to apply https://patchwork.kernel.org/project/netdevbpf/list/?series=378545
error message:

Cmd('git') failed due to: exit code(128)
  cmdline: git am -3
  stdout: 'Applying: bpf: Allow LSM programs to use bpf spin locks
Applying: bpf: Implement task local storage
Applying: libbpf: Add support for task local storage
Applying: bpftool: Add support for task local storage
Applying: bpf: Implement get_current_task_btf and RET_PTR_TO_BTF_ID
Applying: bpf: Update selftests for local_storage to use vmlinux.h
Applying: bpf: Add tests for task_local_storage
Using index info to reconstruct a base tree...
M	tools/testing/selftests/bpf/progs/local_storage.c
Falling back to patching base and 3-way merge...
Auto-merging tools/testing/selftests/bpf/progs/local_storage.c
CONFLICT (content): Merge conflict in tools/testing/selftests/bpf/progs/local_storage.c
Patch failed at 0007 bpf: Add tests for task_local_storage
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".'
  stderr: 'error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch' to see the failed patch'

conflict:

diff --cc tools/testing/selftests/bpf/progs/local_storage.c
index 95fad5aca6af,3e3de130f28f..000000000000
--- a/tools/testing/selftests/bpf/progs/local_storage.c
+++ b/tools/testing/selftests/bpf/progs/local_storage.c
@@@ -39,21 -48,37 +48,44 @@@ SEC("lsm/inode_unlink"
  int BPF_PROG(unlink_hook, struct inode *dir, struct dentry *victim)
  {
  	__u32 pid = bpf_get_current_pid_tgid() >> 32;
++<<<<<<< HEAD
 +	struct dummy_storage *storage;
++=======
+ 	struct local_storage *storage;
+ 	bool is_self_unlink;
+ 	int err;
++>>>>>>> bpf: Add tests for task_local_storage
  
  	if (pid != monitored_pid)
  		return 0;
  
+ 	storage = bpf_task_storage_get(&task_storage_map,
+ 				       bpf_get_current_task_btf(), 0, 0);
+ 	if (storage) {
+ 		/* Don't let an executable delete itself */
+ 		bpf_spin_lock(&storage->lock);
+ 		is_self_unlink = storage->exec_inode == victim->d_inode;
+ 		bpf_spin_unlock(&storage->lock);
+ 		if (is_self_unlink)
+ 			return -EPERM;
+ 	}
+ 
  	storage = bpf_inode_storage_get(&inode_storage_map, victim->d_inode, 0,
 -					BPF_LOCAL_STORAGE_GET_F_CREATE);
 +				     BPF_SK_STORAGE_GET_F_CREATE);
  	if (!storage)
  		return 0;
  
++<<<<<<< HEAD
 +	if (storage->value == DUMMY_STORAGE_VALUE)
++=======
+ 	bpf_spin_lock(&storage->lock);
+ 	if (storage->value != DUMMY_STORAGE_VALUE)
++>>>>>>> bpf: Add tests for task_local_storage
  		inode_storage_result = -1;
+ 	bpf_spin_unlock(&storage->lock);
  
 -	err = bpf_inode_storage_delete(&inode_storage_map, victim->d_inode);
 -	if (!err)
 -		inode_storage_result = err;
 +	inode_storage_result =
 +		bpf_inode_storage_delete(&inode_storage_map, victim->d_inode);
  
  	return 0;
  }
@@@ -63,7 -88,8 +95,12 @@@ int BPF_PROG(socket_bind, struct socke
  	     int addrlen)
  {
  	__u32 pid = bpf_get_current_pid_tgid() >> 32;
++<<<<<<< HEAD
 +	struct dummy_storage *storage;
++=======
+ 	struct local_storage *storage;
+ 	int err;
++>>>>>>> bpf: Add tests for task_local_storage
  
  	if (pid != monitored_pid)
  		return 0;
@@@ -73,10 -99,15 +110,16 @@@
  	if (!storage)
  		return 0;
  
++<<<<<<< HEAD
 +	if (storage->value == DUMMY_STORAGE_VALUE)
++=======
+ 	bpf_spin_lock(&storage->lock);
+ 	if (storage->value != DUMMY_STORAGE_VALUE)
++>>>>>>> bpf: Add tests for task_local_storage
  		sk_storage_result = -1;
+ 	bpf_spin_unlock(&storage->lock);
  
 -	err = bpf_sk_storage_delete(&sk_storage_map, sock->sk);
 -	if (!err)
 -		sk_storage_result = err;
 -
 +	sk_storage_result = bpf_sk_storage_delete(&sk_storage_map, sock->sk);
  	return 0;
  }

kernel-patches-bot · 2020-11-06T02:17:08Z

At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=378545 expired. Closing PR.

restrack should only be attached to a cm_id while the ID has a valid device pointer. It is set up when the device is first loaded, but not cleared when the device is removed. There is also two copies of the device pointer, one private and one in the public API, and these were left out of sync. Make everything go to NULL together and manipulate restrack right around the device assignments. Found by syzcaller: BUG: KASAN: wild-memory-access in __list_del include/linux/list.h:112 [inline] BUG: KASAN: wild-memory-access in __list_del_entry include/linux/list.h:135 [inline] BUG: KASAN: wild-memory-access in list_del include/linux/list.h:146 [inline] BUG: KASAN: wild-memory-access in cma_cancel_listens drivers/infiniband/core/cma.c:1767 [inline] BUG: KASAN: wild-memory-access in cma_cancel_operation drivers/infiniband/core/cma.c:1795 [inline] BUG: KASAN: wild-memory-access in cma_cancel_operation+0x1f4/0x4b0 drivers/infiniband/core/cma.c:1783 Write of size 8 at addr dead000000000108 by task syz-executor716/334 CPU: 0 PID: 334 Comm: syz-executor716 Not tainted 5.11.0+ #271 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 Call Trace: __dump_stack lib/dump_stack.c:79 [inline] dump_stack+0xbe/0xf9 lib/dump_stack.c:120 __kasan_report mm/kasan/report.c:400 [inline] kasan_report.cold+0x5f/0xd5 mm/kasan/report.c:413 __list_del include/linux/list.h:112 [inline] __list_del_entry include/linux/list.h:135 [inline] list_del include/linux/list.h:146 [inline] cma_cancel_listens drivers/infiniband/core/cma.c:1767 [inline] cma_cancel_operation drivers/infiniband/core/cma.c:1795 [inline] cma_cancel_operation+0x1f4/0x4b0 drivers/infiniband/core/cma.c:1783 _destroy_id+0x29/0x460 drivers/infiniband/core/cma.c:1862 ucma_close_id+0x36/0x50 drivers/infiniband/core/ucma.c:185 ucma_destroy_private_ctx+0x58d/0x5b0 drivers/infiniband/core/ucma.c:576 ucma_close+0x91/0xd0 drivers/infiniband/core/ucma.c:1797 __fput+0x169/0x540 fs/file_table.c:280 task_work_run+0xb7/0x100 kernel/task_work.c:140 exit_task_work include/linux/task_work.h:30 [inline] do_exit+0x7da/0x17f0 kernel/exit.c:825 do_group_exit+0x9e/0x190 kernel/exit.c:922 __do_sys_exit_group kernel/exit.c:933 [inline] __se_sys_exit_group kernel/exit.c:931 [inline] __x64_sys_exit_group+0x2d/0x30 kernel/exit.c:931 do_syscall_64+0x2d/0x40 arch/x86/entry/common.c:46 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Fixes: 255d0c1 ("RDMA/cma: rdma_bind_addr() leaks a cma_dev reference count") Link: https://lore.kernel.org/r/3352ee288fe34f2b44220457a29bfc0548686363.1620711734.git.leonro@nvidia.com Signed-off-by: Shay Drory <shayd@nvidia.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

…CKOPT Add verifier ctx test to call bpf_get_netns_cookie from cgroup/setsockopt. #269/p pass ctx or null check, 1: ctx Did not run the program (not supported) OK #270/p pass ctx or null check, 2: null Did not run the program (not supported) OK #271/p pass ctx or null check, 3: 1 OK #272/p pass ctx or null check, 4: ctx - const OK #273/p pass ctx or null check, 5: null (connect) Did not run the program (not supported) OK #274/p pass ctx or null check, 6: null (bind) Did not run the program (not supported) OK #275/p pass ctx or null check, 7: ctx (bind) Did not run the program (not supported) OK #276/p pass ctx or null check, 8: null (bind) OK #277/p pass ctx or null check, 9: ctx (cgroup/setsockopt) Did not run the program (not supported) OK #278/p pass ctx or null check, 10: null (cgroup/setsockopt) Did not run the program (not supported) OK Signed-off-by: Stanislav Fomichev <sdf@google.com>

Add a new test case to query on an empty bpf_mprog and pass the revision directly into expected_revision for attachment to assert that this does succeed. ./test_progs -t tc_opts [ 1.406778] tsc: Refined TSC clocksource calibration: 3407.990 MHz [ 1.408863] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x311fcaf6eb0, max_idle_ns: 440795321766 ns [ 1.412419] clocksource: Switched to clocksource tsc [ 1.428671] bpf_testmod: loading out-of-tree module taints kernel. [ 1.430260] bpf_testmod: module verification failed: signature and/or required key missing - tainting kernel #252 tc_opts_after:OK #253 tc_opts_append:OK #254 tc_opts_basic:OK #255 tc_opts_before:OK #256 tc_opts_chain_classic:OK #257 tc_opts_chain_mixed:OK #258 tc_opts_delete_empty:OK #259 tc_opts_demixed:OK #260 tc_opts_detach:OK #261 tc_opts_detach_after:OK #262 tc_opts_detach_before:OK #263 tc_opts_dev_cleanup:OK #264 tc_opts_invalid:OK #265 tc_opts_max:OK #266 tc_opts_mixed:OK #267 tc_opts_prepend:OK #268 tc_opts_query:OK #269 tc_opts_query_attach:OK <--- (new test) #270 tc_opts_replace:OK #271 tc_opts_revision:OK Summary: 20/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>

Add a new test case to query on an empty bpf_mprog and pass the revision directly into expected_revision for attachment to assert that this does succeed. ./test_progs -t tc_opts [ 1.406778] tsc: Refined TSC clocksource calibration: 3407.990 MHz [ 1.408863] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x311fcaf6eb0, max_idle_ns: 440795321766 ns [ 1.412419] clocksource: Switched to clocksource tsc [ 1.428671] bpf_testmod: loading out-of-tree module taints kernel. [ 1.430260] bpf_testmod: module verification failed: signature and/or required key missing - tainting kernel #252 tc_opts_after:OK #253 tc_opts_append:OK #254 tc_opts_basic:OK #255 tc_opts_before:OK #256 tc_opts_chain_classic:OK #257 tc_opts_chain_mixed:OK #258 tc_opts_delete_empty:OK #259 tc_opts_demixed:OK #260 tc_opts_detach:OK #261 tc_opts_detach_after:OK #262 tc_opts_detach_before:OK #263 tc_opts_dev_cleanup:OK #264 tc_opts_invalid:OK #265 tc_opts_max:OK #266 tc_opts_mixed:OK #267 tc_opts_prepend:OK #268 tc_opts_query:OK #269 tc_opts_query_attach:OK <--- (new test) #270 tc_opts_replace:OK #271 tc_opts_revision:OK Summary: 20/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/r/20231006220655.1653-6-daniel@iogearbox.net Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>

Add several new test cases which assert corner cases on the mprog query mechanism, for example, around passing in a too small or a larger array than the current count. ./test_progs -t tc_opts #252 tc_opts_after:OK #253 tc_opts_append:OK #254 tc_opts_basic:OK #255 tc_opts_before:OK #256 tc_opts_chain_classic:OK #257 tc_opts_chain_mixed:OK #258 tc_opts_delete_empty:OK #259 tc_opts_demixed:OK #260 tc_opts_detach:OK #261 tc_opts_detach_after:OK #262 tc_opts_detach_before:OK #263 tc_opts_dev_cleanup:OK #264 tc_opts_invalid:OK #265 tc_opts_max:OK #266 tc_opts_mixed:OK #267 tc_opts_prepend:OK #268 tc_opts_query:OK #269 tc_opts_query_attach:OK #270 tc_opts_replace:OK #271 tc_opts_revision:OK Summary: 20/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>

Add several new test cases which assert corner cases on the mprog query mechanism, for example, around passing in a too small or a larger array than the current count. ./test_progs -t tc_opts #252 tc_opts_after:OK #253 tc_opts_append:OK #254 tc_opts_basic:OK #255 tc_opts_before:OK #256 tc_opts_chain_classic:OK #257 tc_opts_chain_mixed:OK #258 tc_opts_delete_empty:OK #259 tc_opts_demixed:OK #260 tc_opts_detach:OK #261 tc_opts_detach_after:OK #262 tc_opts_detach_before:OK #263 tc_opts_dev_cleanup:OK #264 tc_opts_invalid:OK #265 tc_opts_max:OK #266 tc_opts_mixed:OK #267 tc_opts_prepend:OK #268 tc_opts_query:OK #269 tc_opts_query_attach:OK #270 tc_opts_replace:OK #271 tc_opts_revision:OK Summary: 20/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Alan Maguire <alan.maguire@oracle.com>

Add several new test cases which assert corner cases on the mprog query mechanism, for example, around passing in a too small or a larger array than the current count. ./test_progs -t tc_opts #252 tc_opts_after:OK #253 tc_opts_append:OK #254 tc_opts_basic:OK #255 tc_opts_before:OK #256 tc_opts_chain_classic:OK #257 tc_opts_chain_mixed:OK #258 tc_opts_delete_empty:OK #259 tc_opts_demixed:OK #260 tc_opts_detach:OK #261 tc_opts_detach_after:OK #262 tc_opts_detach_before:OK #263 tc_opts_dev_cleanup:OK #264 tc_opts_invalid:OK #265 tc_opts_max:OK #266 tc_opts_mixed:OK #267 tc_opts_prepend:OK #268 tc_opts_query:OK #269 tc_opts_query_attach:OK #270 tc_opts_replace:OK #271 tc_opts_revision:OK Summary: 20/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Reviewed-by: Alan Maguire <alan.maguire@oracle.com> Link: https://lore.kernel.org/bpf/20231017081728.24769-1-daniel@iogearbox.net

kernel-patches-bot added bpf-next new V1 labels Oct 27, 2020

kernel-patches-bot added changes-requested and removed new labels Nov 2, 2020

kernel-patches-bot closed this Nov 2, 2020

kernel-patches-bot reopened this Nov 3, 2020

kernel-patches-bot added new V2 and removed V1 changes-requested labels Nov 3, 2020

kernel-patches-bot force-pushed the series/371615=>bpf-next branch from ed51eee to 4e78491 Compare November 3, 2020 20:49

kernel-patches-bot force-pushed the series/371615=>bpf-next branch from 4e78491 to 0a6b4c2 Compare November 4, 2020 02:07

kernel-patches-bot and others added 10 commits November 5, 2020 12:33

adding ci files

3400236

libbpf: Add support for task local storage

af0aaaa

Updates the bpf_probe_map_type API to also support BPF_MAP_TYPE_TASK_STORAGE similar to other local storage maps. Acked-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: KP Singh <kpsingh@google.com>

bpftool: Add support for task local storage

084e8fb

Updates the binary to handle the BPF_MAP_TYPE_TASK_STORAGE as "task_storage" for printing and parsing. Also updates the documentation and bash completion Acked-by: Song Liu <songliubraving@fb.com> Signed-off-by: KP Singh <kpsingh@google.com>

bpf: Update selftests for local_storage to use vmlinux.h

c7fcb6e

With the fixing of BTF pruning of embedded types being fixed, the test can be simplified to use vmlinux.h Acked-by: Song Liu <songliubraving@fb.com> Signed-off-by: KP Singh <kpsingh@google.com>

kernel-patches-bot force-pushed the series/371615=>bpf-next branch from 0a6b4c2 to 8900b90 Compare November 5, 2020 20:33

kernel-patches-bot removed the V2 label Nov 5, 2020

kernel-patches-bot added the V4 label Nov 5, 2020

kernel-patches-bot added merge-conflict V5 and removed V4 labels Nov 5, 2020

kernel-patches-bot added changes-requested and removed merge-conflict new labels Nov 6, 2020

kernel-patches-bot closed this Nov 6, 2020

kernel-patches-bot deleted the series/371615=>bpf-next branch November 10, 2020 22:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement task_local_storage #271

Implement task_local_storage #271

kernel-patches-bot commented Oct 27, 2020

kernel-patches-bot commented Oct 27, 2020

kernel-patches-bot commented Nov 2, 2020

kernel-patches-bot commented Nov 3, 2020

kernel-patches-bot commented Nov 4, 2020

kernel-patches-bot commented Nov 5, 2020

kernel-patches-bot commented Nov 5, 2020

kernel-patches-bot commented Nov 6, 2020

Implement task_local_storage #271

Implement task_local_storage #271

Conversation

kernel-patches-bot commented Oct 27, 2020

kernel-patches-bot commented Oct 27, 2020

kernel-patches-bot commented Nov 2, 2020

kernel-patches-bot commented Nov 3, 2020

kernel-patches-bot commented Nov 4, 2020

kernel-patches-bot commented Nov 5, 2020

kernel-patches-bot commented Nov 5, 2020

kernel-patches-bot commented Nov 6, 2020