Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix 5.10+ complexity issue with kubeProxyReplacement=disabled #16084

Merged
merged 3 commits into from
May 12, 2021
Merged

Fix 5.10+ complexity issue with kubeProxyReplacement=disabled #16084

merged 3 commits into from
May 12, 2021

Commits on May 12, 2021

  1. bpf: Fixes to support mattr=+alu32 

    mattr=+alu32, supported since LLVM 7.0 and implied by mcpu=v3, enables
the use of 32-bit registers in BPF bytecode. Enabling this compiler
option can however result in loading issues as illustrated below.

    12: (61) r1 = *(u32 *)(r0 +80) // ctx->data_end
    13: (61) r6 = *(u32 *)(r0 +76) // ctx->data
    14: (bc) w7 = w6 // <- verifier looses track of inferred pkt type here.
    [...]
    38: (71) r1 = *(u8 *)(r7 +20)
    R7 invalid mem access 'inv'

These errors typically happen because the data and data_end pointers are
actually 32-bit registers. Depending on how these pointers are used,
LLVM sometimes makes use of that assumption (e.g., 32-bit assignment on
instruction 14 above). The verifier is however not able to follow and
reject such programs.

We can usually work around those by ensuring these pointers are only
used via 64-bit types. This commit implements this wherever needed to
pass the verifier.

Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Paul Chaignon <paul@cilium.io>
    @pchaigno
    pchaigno committed May 12, 2021
    Configuration menu
    Copy the full SHA
    0467eb8 View commit details
    Browse the repository at this point in the history

  2. loader, bpf: Use mcpu=v3 on kernels 5.10+ 

    Set mcpu=v3 in the compiler on kernels 5.10+ to use all available eBPF
instructions and 32-bit registers. This change fixes the complexity
issue we're hitting on v5.10+ when socket-level load balancing is disabled
(via enable-host-services=false or kube-proxy-replacement=disabled).

Using the third eBPF instruction set doesn't reduce complexity for all
BPF programs but it leads to more standard numbers, with less variations
in complexities. A big part of this improvement is due to the implicit
use of mattr=+alu32 to enable 32-bit eBPF registers.

In addition to the end-to-end test on bpf-next, this change was tested
on kernels 5.10 and 5.11 with the existing verifier-test.sh, compiling
the datapath with both KERNEL=netnext and KERNEL=419.

Signed-off-by: Paul Chaignon <paul@cilium.io>
    @pchaigno
    pchaigno committed May 12, 2021
    Configuration menu
    Copy the full SHA
    ccd5376 View commit details
    Browse the repository at this point in the history

  3. bpf/Makefile: Remove workaround for complexity issue 

    On master and with kernels 5.10+, we have a complexity issue when
ENABLE_HOST_SERVICES_FULL is undefined (i.e., socket-level load balancing
is disabled and additional code compiled in bpf_lxc as a replacement).
Our verifier test included a workaround for that issue, by always
defining ENABLE_HOST_SERVICES_FULL on newer kernels.

This commit removes that workaround since the previous commit fixed the
complexity issue.

Signed-off-by: Paul Chaignon <paul@cilium.io>
    @pchaigno
    pchaigno committed May 12, 2021
    Configuration menu
    Copy the full SHA
    552ec87 View commit details
    Browse the repository at this point in the history