-
Notifications
You must be signed in to change notification settings - Fork 96
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bpf: support resilient split BTF #7016
Commits on May 18, 2024
-
libbpf: add btf__distill_base() creating split BTF with distilled bas…
…e BTF To support more robust split BTF, adding supplemental context for the base BTF type ids that split BTF refers to is required. Without such references, a simple shuffling of base BTF type ids (without any other significant change) invalidates the split BTF. Here the attempt is made to store additional context to make split BTF more robust. This context comes in the form of distilled base BTF providing minimal information (name and - in some cases - size) for base INTs, FLOATs, STRUCTs, UNIONs, ENUMs and ENUM64s along with modified split BTF that points at that base and contains any additional types needed (such as TYPEDEF, PTR and anonymous STRUCT/UNION declarations). This information constitutes the minimal BTF representation needed to disambiguate or remove split BTF references to base BTF. The rules are as follows: - INT, FLOAT are recorded in full. - if a named base BTF STRUCT or UNION is referred to from split BTF, it will be encoded either as a zero-member sized STRUCT/UNION (preserving size for later relocation checks) or as a named FWD. Only base BTF STRUCT/UNIONs that are either embedded in split BTF STRUCT/UNIONs or that have multiple STRUCT/UNION instances of the same name need to preserve size information, so a FWD representation will be used in most cases. - if an ENUM[64] is named, a ENUM forward representation (an ENUM with no values) is used. - in all other cases, the type is added to the new split BTF. Avoiding struct/union/enum/enum64 expansion is important to keep the distilled base BTF representation to a minimum size. When successful, new representations of the distilled base BTF and new split BTF that refers to it are returned. Both need to be freed by the caller. So to take a simple example, with split BTF with a type referring to "struct sk_buff", we will generate distilled base BTF with a FWD struct sk_buff, and the split BTF will refer to it instead. Tools like pahole can utilize such split BTF to populate the .BTF section (split BTF) and an additional .BTF.base section. Then when the split BTF is loaded, the distilled base BTF can be used to relocate split BTF to reference the current (and possibly changed) base BTF. So for example if "struct sk_buff" was id 502 when the split BTF was originally generated, we can use the distilled base BTF to see that id 502 refers to a "struct sk_buff" and replace instances of id 502 with the current (relocated) base BTF sk_buff type id. Distilled base BTF is small; when building a kernel with all modules using distilled base BTF as a test, ovreall module size grew by only 5.3Mb total across ~2700 modules. Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Configuration menu - View commit details
-
Copy full SHA for cc71ec9 - Browse repository at this point
Copy the full SHA cc71ec9View commit details -
selftests/bpf: test distilled base, split BTF generation
Test generation of split+distilled base BTF, ensuring that - base BTF STRUCTs which are embedded in split BTF structs are represented as 0-member sized structs, allowing size checking - base BTF UNION which has a duplicate-named UNION is represented as a 0-member sized union, helping clarify which union is referred to - FWDs are used in place of full named struct/union declarations - FWDs are used in place of full named enum declarations - anonymous struct/unions are represented in full in split BTF - anonymous enums are represented in full in split BTF - types unreferenced from split BTF are not present in distilled base BTF Also test that with vmlinux BTF and split BTF based upon it, we only represent needed base types referenced from split BTF in distilled base. Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Configuration menu - View commit details
-
Copy full SHA for e3b1726 - Browse repository at this point
Copy the full SHA e3b1726View commit details -
libbpf: add btf__parse_opts() API for flexible BTF parsing
Options cover existing parsing scenarios (ELF, raw, retrieving .BTF.ext) and also allow specification of the ELF section name containing BTF. This will allow consumers to retrieve BTF from .BTF.base sections (BTF_BASE_ELF_SEC) also. Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Configuration menu - View commit details
-
Copy full SHA for 0d2aa5a - Browse repository at this point
Copy the full SHA 0d2aa5aView commit details -
bpftool: support displaying raw split BTF using base BTF section as base
If no base BTF can be found, fall back to checking for the .BTF.base section and use it to display split BTF. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Acked-by: Quentin Monnet <qmo@kernel.org>
Configuration menu - View commit details
-
Copy full SHA for 53bd06c - Browse repository at this point
Copy the full SHA 53bd06cView commit details -
resolve_btfids: use .BTF.base ELF section as base BTF if -B option is…
… used When resolving BTF ids, use the BTF in the module .BTF.base section when passed the -B option. Both references to base BTF from split BTF and BTF ids will be relocated for base vmlinux on module load. Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Configuration menu - View commit details
-
Copy full SHA for e2226b8 - Browse repository at this point
Copy the full SHA e2226b8View commit details -
kbuild, bpf: add module-specific pahole/resolve_btfids flags for dist…
…illed base BTF Support creation of module BTF along with distilled base BTF; the latter is stored in a .BTF.base ELF section and supplements split BTF references to base BTF with information about base types, allowing for later relocation of split BTF with a (possibly changed) base. resolve_btfids uses the "-B" option to specify that the BTF.ids section should be populated with split BTF relative to the added .BTF.base section rather than relative to the vmlinux base. Modules will be built with a distilled .BTF.base section for external module build, i.e. make -C. -M=path2/module ...while in-tree module build as part of a normal kernel build will not generate distilled base BTF; this is because in-tree modules change with the kernel and do not require BTF relocation for the running vmlinux. Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Configuration menu - View commit details
-
Copy full SHA for 65ebb9e - Browse repository at this point
Copy the full SHA 65ebb9eView commit details -
Map distilled base BTF type ids referenced in split BTF and their references to the base BTF passed in, and if the mapping succeeds, reparent the split BTF to the base BTF. Relocation is done by first verifying that distilled base BTF only consists of named INT, FLOAT, ENUM, FWD, STRUCT and UNION kinds; then we sort these to speed lookups. Once sorted, the base BTF is iterated, and for each relevant kind we check for an equivalent in distilled base BTF. When found, the mapping from distilled -> base BTF id and string offset is recorded. Once all mappings are established, we can update type ids and string offsets in split BTF and reparent it to the new base. Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Configuration menu - View commit details
-
Copy full SHA for bd4e8cc - Browse repository at this point
Copy the full SHA bd4e8ccView commit details -
selftests/bpf: extend distilled BTF tests to cover BTF relocation
Ensure relocated BTF looks as expected; in this case identical to original split BTF, with a few duplicate anonymous types added to split BTF by the relocation process. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 1c55ef8 - Browse repository at this point
Copy the full SHA 1c55ef8View commit details -
module, bpf: store BTF base pointer in struct module
...as this will allow split BTF modules with a base BTF representation (rather than the full vmlinux BTF at time of BTF encoding) to resolve their references to kernel types in a way that is more resilient to small changes in kernel types. This will allow modules that are not built every time the kernel is to provide more resilient BTF, rather than have it invalidated every time BTF ids for core kernel types change. Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Configuration menu - View commit details
-
Copy full SHA for 72f4be8 - Browse repository at this point
Copy the full SHA 72f4be8View commit details -
libbpf,bpf: share BTF relocate-related code with kernel
Share relocation implementation with the kernel. As part of this, we also need the type/string visitation functions so add them to a btf_common.c file that also gets shared with the kernel. Relocation code in kernel and userspace is identical save for the impementation of the reparenting of split BTF to the relocated base BTF and retrieval of BTF header from "struct btf"; these small functions need separate user-space and kernel implementations. One other wrinkle on the kernel side is we have to map .BTF.ids in modules as they were generated with the type ids used at BTF encoding time. btf_relocate() optionally returns an array mapping from old BTF ids to relocated ids, so we use that to fix up these references where needed for kfuncs. Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Configuration menu - View commit details
-
Copy full SHA for e2a19a1 - Browse repository at this point
Copy the full SHA e2a19a1View commit details -
bpftool: support displaying relocated-with-base split BTF
If the -R <base_btf> option is used, we can display BTF that has been generated with distilled base BTF in its relocated form. For example for bpf_testmod.ko (which is built as an out-of-tree module, so has a distilled .BTF.base section: bpftool btf dump file bpf_testmod.ko Alternatively, we can display content relocated with (a possibly changed) base BTF via bpftool btf dump -R /sys/kernel/btf/vmlinux bpf_testmod.ko The latter mirrors how the kernel will handle such split BTF; it relocates its representation with the running kernel, and if successful, renumbers BTF ids to reference the current vmlinux BTF. Signed-off-by: Alan Maguire <alan.maguire@oracle.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Reviewed-by: Quentin Monnet <qmo@kernel.org>
Configuration menu - View commit details
-
Copy full SHA for 98275e0 - Browse repository at this point
Copy the full SHA 98275e0View commit details