Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

btf: support marshaling Type #641

Closed
wants to merge 6 commits into from
Closed

btf: support marshaling Type #641

wants to merge 6 commits into from

Conversation

lmb
Copy link
Collaborator

@lmb lmb commented Apr 25, 2022

This is a first cut at adding a btf.Builder which allows turning Type into BTF that the kernel understands. The code can round-trip vmlinux BTF and then load the result into the kernel, which is a good indication that we covered most of the bases.

As it stands, the builder is not exported or used in the package, since there is currently no way to add strings necessary for LineInfo to the BTF. This should become a lot easier once #606 lands.

Best reviewed per commit.

btf: add btfInt

Replace intEncoding with a type. This allows adding setters.

btf: add Array.Index

The kernel refuses to load an array type without an index type, so probably
btf.rst is outdated.

btf: move typeDeque to separate file

Split typeDeque into a separate file to reduce the amount of code in types.go.

btf: add stringTableBuilder

A stringTableBuilder helps build a BTF-style string table, which delimits
individual strings with a 0 byte. Each unique string is only added to the
table once.

btf: replace Type.walk with walkType

Type.walk takes a *typeDeque as argument. Go escape analysis doesn't see through
interface boundaries and therefore always allocates such pointers on the heap.
This forces typeDeque to always be heap allocated, which makes traversal more
expensive than it has to be.

Replace the interface function with walkType, which essentially contains a big
type switch.

btf: add builder

Allow building BTF wire format from scratch. We can compare a full vmlinux build to
just re-encoding raw type data as we currently do in Spec.marshal:

    name                    time/op
    BuildVmlinux/builder-4  77.9ms ± 2%
    BuildVmlinux/native-4   23.3ms ± 2%

    name                    alloc/op
    BuildVmlinux/builder-4  33.7MB ± 0%
    BuildVmlinux/native-4   20.1MB ± 0%

    name                    allocs/op
    BuildVmlinux/builder-4    473k ± 0%
    BuildVmlinux/native-4     394k ± 0%

A full rebuild is about 3x slower and uses 1.6x the memory. We can't use the builder
to replace Spec.marshal yet, since we don't have a good way to include LineInfos
in the generated BTF. This will become easier once LineInfos are stored in
Instruction metadata.

Updates #616

@lmb lmb requested a review from ti-mo April 25, 2022 13:14
@lmb lmb marked this pull request as draft April 26, 2022 10:20
@lmb
Copy link
Collaborator Author

lmb commented Apr 26, 2022

As always, needs some more work!

lmb added 4 commits April 26, 2022 17:20
Replace intEncoding with a type. This allows adding setters.
The kernel refuses to load an array type without an index type, so probably
btf.rst is outdated.
Split typeDeque into a separate file to reduce the amount of code in types.go.
A stringTableBuilder helps build a BTF-style string table, which delimits
individual strings with a 0 byte. Each unique string is only added to the
table once.
Type.walk takes a *typeDeque as argument. Go escape analysis doesn't see through
interface boundaries and therefore always allocates such pointers on the heap.
This forces typeDeque to always be heap allocated, which makes traversal more
expensive than it has to be.

Replace the interface function with walkType, which essentially contains a big
type switch.
Allow building BTF wire format from scratch. We can compare a full vmlinux build to
just re-encoding raw type data as we currently do in Spec.marshal:

    name                    time/op
    BuildVmlinux/builder-4  77.9ms ± 2%
    BuildVmlinux/native-4   23.3ms ± 2%

    name                    alloc/op
    BuildVmlinux/builder-4  33.7MB ± 0%
    BuildVmlinux/native-4   20.1MB ± 0%

    name                    allocs/op
    BuildVmlinux/builder-4    473k ± 0%
    BuildVmlinux/native-4     394k ± 0%

A full rebuild is about 3x slower and uses 1.6x the memory. We can't use the builder
to replace Spec.marshal yet, since we don't have a good way to include LineInfos
in the generated BTF. This will become easier once LineInfos are stored in
Instruction metadata.

Updates cilium#616
@lmb lmb marked this pull request as ready for review April 27, 2022 16:03
@lmb
Copy link
Collaborator Author

lmb commented Apr 27, 2022

This works now!

@lmb lmb marked this pull request as draft July 20, 2022 14:27
@lmb lmb closed this Sep 6, 2022
@lmb lmb deleted the btf-builder branch September 6, 2022 08:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant