Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpf: Add 64bit enum value support #3075

Closed
wants to merge 25 commits into from
Closed

bpf: Add 64bit enum value support #3075

wants to merge 25 commits into from

Conversation

kernel-patches-bot
Copy link

Pull request for series with
subject: bpf: Add 64bit enum value support
version: 3
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=645337

@kernel-patches-bot
Copy link
Author

Master branch: 7e062cd
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=645337
version: 3

@kernel-patches-bot
Copy link
Author

Master branch: 7e062cd
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=645337
version: 3

@kernel-patches-bot
Copy link
Author

Master branch: 1626f57
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=645337
version: 3

@kernel-patches-bot
Copy link
Author

Master branch: 4b4b4f9
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=645337
version: 3

@kernel-patches-bot
Copy link
Author

Master branch: ada6605
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=645337
version: 3

Pull request is NOT updated. Failed to apply https://patchwork.kernel.org/project/netdevbpf/list/?series=645337
error message:

Cmd('git') failed due to: exit code(128)
  cmdline: git am -3
  stdout: 'Applying: bpf: Add btf enum64 support
Applying: libbpf: Permit 64bit relocation value
Applying: libbpf: Fix an error in 64bit relocation value computation
Applying: libbpf: Refactor btf__add_enum() for future code sharing
Applying: libbpf: Add enum64 parsing and new enum64 public API
Using index info to reconstruct a base tree...
M	tools/lib/bpf/libbpf.map
Falling back to patching base and 3-way merge...
Auto-merging tools/lib/bpf/libbpf.map
CONFLICT (content): Merge conflict in tools/lib/bpf/libbpf.map
Patch failed at 0005 libbpf: Add enum64 parsing and new enum64 public API
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".'
  stderr: 'error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch'

conflict:

diff --cc tools/lib/bpf/libbpf.map
index 38e284ff057d,8d2ec50c2307..000000000000
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@@ -462,10 -462,7 +462,15 @@@ LIBBPF_0.8.0 
  
  LIBBPF_1.0.0 {
  	global:
++<<<<<<< HEAD
 +		libbpf_bpf_attach_type_str;
 +		libbpf_bpf_link_type_str;
 +		libbpf_bpf_map_type_str;
 +		libbpf_bpf_prog_type_str;
 +
++=======
+ 		btf__add_enum64;
+ 		btf__add_enum64_value;
++>>>>>>> libbpf: Add enum64 parsing and new enum64 public API
  	local: *;
  };

@kernel-patches-bot
Copy link
Author

At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=645337 expired. Closing PR.

Dan Carpenter and others added 4 commits June 2, 2022 16:27
@anakryiko
bpf: Use safer kvmalloc_array() where possible
7db6299 
The kvmalloc_array() function is safer because it has a check for
integer overflows.  These sizes come from the user and I was not
able to see any bounds checking so an integer overflow seems like a
realistic concern.

Fixes: 0dcac27 ("bpf: Add multi kprobe link")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/Yo9VRVMeHbALyjUH@kili
@d-e-s-o @anakryiko
libbpf: Fix a couple of typos
12947d8 
This change fixes a couple of typos that were encountered while studying
the source code.

Signed-off-by: Daniel Müller <deso@posteo.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/20220601154025.3295035-1-deso@posteo.net
@danobi @anakryiko
bpf, test_run: Remove unnecessary prog type checks
7b711e7 
These checks were effectively noops b/c there's only one way these
functions get called: through prog_ops dispatching. And since there's no
other callers, we can be sure that `prog` is always the correct type.

Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/0a9aaac329f76ddb17df1786b001117823ffefa5.1653855302.git.dxu@dxuuu.xyz
@iamkafai @anakryiko
selftests/bpf: Fix tc_redirect_dtime
f175ece 
tc_redirect_dtime was reported flaky from time to time.  It
always fails at the udp test and complains about the bpf@tc-ingress
got a skb->tstamp when handling udp packet.  It is unexpected
because the skb->tstamp should have been cleared when crossing
different netns.

The most likely cause is that the skb is actually a tcp packet
from the earlier tcp test.  It could be the final TCP_FIN handling.

This patch tightens the skb->tstamp check in the bpf prog.  It ensures
the skb is the current testing traffic.  First, it checks that skb
matches the IPPROTO of the running test (i.e. tcp vs udp).
Second, it checks the server port (dst_ns_port).  The server
port is unique for each test (50000 + test_enum).

Also fixed a typo in test_udp_dtime(): s/P100/P101/

Fixes: c803475 ("bpf: selftests: test skb->tstamp in redirect_neigh")
Reported-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/20220601234050.2572671-1-kafai@fb.com
@kernel-patches-bot
Copy link
Author

Master branch: f175ece
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=647021
version: 4

@anakryiko
libbpf: Fix is_pow_of_2
f913ad6 
Move the correct definition from linker.c into libbpf_internal.h.

Fixes: 0087a68 ("libbpf: Automatically fix up BPF_MAP_TYPE_RINGBUF size, if necessary")
Reported-by: Yuze Chi <chiyuze@google.com>
Signed-off-by: Yuze Chi <chiyuze@google.com>
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20220603055156.2830463-1-irogers@google.com
@kernel-patches-bot
Copy link
Author

Master branch: f913ad6
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=647021
version: 4

liuhangbin and others added 20 commits June 3, 2022 14:41
@liuhangbin @anakryiko
selftests/bpf: Add drv mode testing for xdping
4349680 
As subject, we only test SKB mode for xdping at present.
Now add DRV mode for xdping.

Signed-off-by: Hangbin Liu <liuhangbin@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/20220602032507.464453-1-liuhangbin@gmail.com
adding ci files
7897f4e
@yonghong-song
bpf: Add btf enum64 support
1c19f1c 
Currently, BTF only supports upto 32bit enum value with BTF_KIND_ENUM.
But in kernel, some enum indeed has 64bit values, e.g.,
in uapi bpf.h, we have
  enum {
        BPF_F_INDEX_MASK                = 0xffffffffULL,
        BPF_F_CURRENT_CPU               = BPF_F_INDEX_MASK,
        BPF_F_CTXLEN_MASK               = (0xfffffULL << 32),
  };
In this case, BTF_KIND_ENUM will encode the value of BPF_F_CTXLEN_MASK
as 0, which certainly is incorrect.

This patch added a new btf kind, BTF_KIND_ENUM64, which permits
64bit value to cover the above use case. The BTF_KIND_ENUM64 has
the following three fields followed by the common type:
  struct bpf_enum64 {
    __u32 nume_off;
    __u32 val_lo32;
    __u32 val_hi32;
  };
Currently, btf type section has an alignment of 4 as all element types
are u32. Representing the value with __u64 will introduce a pad
for bpf_enum64 and may also introduce misalignment for the 64bit value.
Hence, two members of val_hi32 and val_lo32 are chosen to avoid these issues.

The kflag is also introduced for BTF_KIND_ENUM and BTF_KIND_ENUM64
to indicate whether the value is signed or unsigned. The kflag intends
to provide consistent output of BTF C fortmat with the original
source code. For example, the original BTF_KIND_ENUM bit value is 0xffffffff.
The format C has two choices, printing out 0xffffffff or -1 and current libbpf
prints out as unsigned value. But if the signedness is preserved in btf,
the value can be printed the same as the original source code.
The kflag value 0 means unsigned values, which is consistent to the default
by libbpf and should also cover most cases as well.

The new BTF_KIND_ENUM64 is intended to support the enum value represented as
64bit value. But it can represent all BTF_KIND_ENUM values as well.
The compiler ([1]) and pahole will generate BTF_KIND_ENUM64 only if the value has
to be represented with 64 bits.

In addition, a static inline function btf_kind_core_compat() is introduced which
will be used later when libbpf relo_core.c changed. Here the kernel shares the
same relo_core.c with libbpf.

  [1] https://reviews.llvm.org/D124641

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
libbpf: Permit 64bit relocation value
5d7557d 
Currently, the libbpf limits the relocation value to be 32bit
since all current relocations have such a limit. But with
BTF_KIND_ENUM64 support, the enum value could be 64bit.
So let us permit 64bit relocation value in libbpf.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
libbpf: Fix an error in 64bit relocation value computation
b7f8b39 
Currently, the 64bit relocation value in the instruction
is computed as follows:
  __u64 imm = insn[0].imm + ((__u64)insn[1].imm << 32)

Suppose insn[0].imm = -1 (0xffffffff) and insn[1].imm = 1.
With the above computation, insn[0].imm will first sign-extend
to 64bit -1 (0xffffffffFFFFFFFF) and then add 0x1FFFFFFFF,
producing incorrect value 0xFFFFFFFF. The correct value
should be 0x1FFFFFFFF.

Changing insn[0].imm to __u32 first will prevent 64bit sign
extension and fix the issue. Merging high and low 32bit values
also changed from '+' to '|' to be consistent with other
similar occurences in kernel and libbpf.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Dave Marchevsky <davemarchevsky@fb.com>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
libbpf: Refactor btf__add_enum() for future code sharing
e18c1c5 
Refactor btf__add_enum() function to create a separate
function btf_add_enum_common() so later the common function
can be used to add enum64 btf type. There is no functionality
change for this patch.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
libbpf: Add enum64 parsing and new enum64 public API
0006b08 
Add enum64 parsing support and two new enum64 public APIs:
  btf__add_enum64
  btf__add_enum64_value

Also add support of signedness for BTF_KIND_ENUM. The
BTF_KIND_ENUM API signatures are not changed. The signedness
will be changed from unsigned to signed if btf__add_enum_value()
finds any negative values.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
libbpf: Add enum64 deduplication support
c534bab 
Add enum64 deduplication support. BTF_KIND_ENUM64 handling
is very similar to BTF_KIND_ENUM.

Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
@yonghong-song
libbpf: Add enum64 support for btf_dump
e83a452 
Add enum64 btf dumping support. For long long and unsigned long long
dump, suffixes 'LL' and 'ULL' are added to avoid compilation errors
in some cases.

Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
@yonghong-song
libbpf: Add enum64 sanitization
6595cd2 
When old kernel does not support enum64 but user space btf
contains non-zero enum kflag or enum64, libbpf needs to
do proper sanitization so modified btf can be accepted
by the kernel.

Sanitization for enum kflag can be achieved by clearing
the kflag bit. For enum64, the type is replaced with an
union of integer member types and the integer member size
must be smaller than enum64 size. If such an integer
type cannot be found, a new type is created and used
for union members.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
libbpf: Add enum64 support for bpf linking
1e2def9 
Add BTF_KIND_ENUM64 support for bpf linking, which is
very similar to BTF_KIND_ENUM.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
libbpf: Add enum64 relocation support
db77624 
The enum64 relocation support is added. The bpf local type
could be either enum or enum64 and the remote type could be
either enum or enum64 too. The all combinations of local enum/enum64
and remote enum/enum64 are supported.

Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
bpftool: Add btf enum64 support
b1a0ff7 
Add BTF_KIND_ENUM64 support.
For example, the following enum is defined in uapi bpf.h.
  $ cat core.c
  enum A {
        BPF_F_INDEX_MASK                = 0xffffffffULL,
        BPF_F_CURRENT_CPU               = BPF_F_INDEX_MASK,
        BPF_F_CTXLEN_MASK               = (0xfffffULL << 32),
  } g;
Compiled with
  clang -target bpf -O2 -g -c core.c
Using bpftool to dump types and generate format C file:
  $ bpftool btf dump file core.o
  ...
  [1] ENUM64 'A' encoding=UNSIGNED size=8 vlen=3
        'BPF_F_INDEX_MASK' val=4294967295ULL
        'BPF_F_CURRENT_CPU' val=4294967295ULL
        'BPF_F_CTXLEN_MASK' val=4503595332403200ULL
  $ bpftool btf dump file core.o format c
  ...
  enum A {
        BPF_F_INDEX_MASK = 4294967295ULL,
        BPF_F_CURRENT_CPU = 4294967295ULL,
        BPF_F_CTXLEN_MASK = 4503595332403200ULL,
  };
  ...

Note that for raw btf output, the encoding (UNSIGNED or SIGNED)
is printed out as well. The 64bit value is also represented properly
in BTF and C dump.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
selftests/bpf: Fix selftests failure
6af64ac 
The kflag is supported now for BTF_KIND_ENUM.
So remove the test which tests verifier failure
due to existence of kflag.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
selftests/bpf: Test new enum kflag and enum64 API functions
6e92fd7 
Add tests to use the new enum kflag and enum64 API functions
in selftest btf_write.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
selftests/bpf: Add BTF_KIND_ENUM64 unit tests
1ade405 
Add unit tests for basic BTF_KIND_ENUM64 encoding.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
selftests/bpf: Test BTF_KIND_ENUM64 for deduplication
e82f973 
Add a few unit tests for BTF_KIND_ENUM64 deduplication.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
selftests/bpf: Add a test for enum64 value relocations
58cc742 
Add a test for enum64 value relocations.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
selftests/bpf: Clarify llvm dependency with possible selftest failures
7ad1533 
Certain subtests in selftests core_reloc and core_reloc_btfgen
requires llvm ENUM64 support in llvm15. If an older compiler
is used, these subtests will fail. Make this requirement clear
in selftests README.rst file.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@yonghong-song
docs/bpf: Update documentation for BTF_KIND_ENUM64 support
c3cf58a 
Add BTF_KIND_ENUM64 documentation in btf.rst.
Also fixed a typo for section number for BTF_KIND_TYPE_TAG
from 2.2.17 to 2.2.18, and fixed a type size issue for
BTF_KIND_ENUM.

Acked-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
@kernel-patches-bot
Copy link
Author

Master branch: 4349680
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=647021
version: 4

@kernel-patches-bot
Copy link
Author

At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=647021 expired. Closing PR.

@kernel-patches-bot kernel-patches-bot deleted the series/637423=>bpf-next branch June 6, 2022 05:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bpf-next changes-requested V4
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@kernel-patches-bot @d-e-s-o @danobi @iamkafai @liuhangbin @yonghong-song