Skip to content

feat: eBPF uSID datapath control-plane packages - #283

Merged
privateip merged 10 commits into
mainfrom
pr2-ebpf-datapath
Aug 6, 2026
Merged

feat: eBPF uSID datapath control-plane packages#283
privateip merged 10 commits into
mainfrom
pr2-ebpf-datapath

Conversation

@privateip

Copy link
Copy Markdown
Contributor

Summary

Third PR in the eBPF uSID datapath stack (on top of #281 usid.c/uformat and #282 codegen toolchain).

Adds the Go control-plane library that will drive the TC-BPF program added in #281:

  • preflight — kernel capability gate (BTF, HASH maps, SCHED_CLS, bpf_fib_lookup with VRF-tbid support), run before attempting to load anything.
  • usidmap — typed read/write/reconcile API for the three kernel maps the program consults (locator_table, function_table, vrf_table).
  • attach — load/pin/attach/detach/watch lifecycle for the TC-BPF ingress hook, including netlink-driven re-attachment on interface or route change. Interface selection is auto-detected but can be overridden via the new GALACTIC_CNI_EBPF_INTERFACES env var (internal/config) for multi-homed nodes.
  • metrics — Prometheus collector plus load/attach event counters.

Every package here is independently unit-testable against fakes (fake kernel tables, a mock kernel prober, fake closers) — nothing outside this tree calls into it yet, so this remains inert until the CNI/GC/installer wiring PRs later in the stack.

Test plan

  • go build ./...
  • go vet ./...
  • go test ./internal/plumbing/ebpf/... ./internal/config/...
  • task lint (0 issues)

Part of the eBPF uSID datapath cutover stack (base: #282).

privateip added a commit that referenced this pull request Aug 5, 2026
Removes the legacy per-route netlink SRv6 ingress mechanism
(srv6.RouteIngressAdd/Del, srv6.go) entirely and replaces it with
registration against the eBPF uSID datapath's pinned maps (attach,
usidmap -- #283): the eBPF/TC-BPF datapath is now the only ingress/decap
path for both veth and tap attachments, so there's no dual-path
coexistence to maintain.

ComputeSID (internal/plumbing/srv6/usid.go) is rewritten onto the
shared uFMT 48+16 bit layout (internal/plumbing/ebpf/uformat -- #281)
instead of its previous ad hoc NodeID/VRFID/Function suffix, so the BGP
control plane and the eBPF dataplane can never drift on bit positions.

The CNI ADD path no longer derives the VRFID straight from the
VPCAttachment identifier (vrfIDFromAttachment); it now allocates a
12-bit uFMT Argument per-node from live BGPVRFInstance CRD state
(allocateArgument), with a collision check (checkArgumentCollision)
covering the allocate-then-create race between two concurrent ADDs.
registerEBPFDatapath/unregisterEBPFDatapath write and roll back the
three eBPF map entries (locator_table, function_table, vrf_table) for
each attachment.

Stacked on #283 (eBPF datapath control-plane packages) and #284
(BGPAdvertisement prefix-merge fix, needed for this PR's
publishBGPStateK8s changes to apply cleanly).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@privateip
privateip marked this pull request as ready for review August 5, 2026 01:58
@privateip
privateip requested a review from a team as a code owner August 5, 2026 01:58
@ecv

ecv commented Aug 5, 2026

Copy link
Copy Markdown

ecv reviewed this through an AI shell, apologies. Questions, not verdicts.

Two things read well first. Re-attaching every resolved interface, rather than only the ones that changed, is the right call, and the FRR story in the comment shows exactly what the diff-only version missed. The generation cutoff scheme is correct as written.

  1. Register writes the whole value struct and leaves Packets, Bytes and LastSeenNs at zero. It runs on every CNI ADD, and again on every retry inside the k8s retry closure, and the comments call that safe because Register overwrites. But those three fields are the datapath's own counters, they reach Prometheus as counters, and they are what R8's gate reads to prove an Argument carried no traffic before cutover. A re-registered entry reads as untouched even after carrying traffic. Should Register read the old value and keep the counters, or should the counters live in a map the control plane never writes?

  2. The filter attaches at priority 1 in direct-action mode, and usid.c returns TC_ACT_OK for traffic it does not claim. In direct-action TC_ACT_OK ends the chain, so no filter at a higher priority number on that device runs. Cilium attaches to native device ingress, and the vmtap docs say pods carry Cilium identity, so both run on these hosts. Does something keep the two off the same device? TC_ACT_UNSPEC is the verdict that hands the packet to the next filter — any reason not to use it on the two fail-open paths?

  3. Load pins every map by name and reuses whatever pin it finds. Change a value struct or a max_entries and the load fails against the old pin. That failure is fatal, and feat: Grant CNI DaemonSet BPF privileges and wire up e2e #288 puts it behind a liveness probe, so the first such change crashloops every node until someone deletes the pins by hand. vrf_value grew egress_kind this cycle, behavior is still unread, and vrf_table sizing is an open question on feat: eBPF vrf_table GC sweep and control-daemon startup #287. What handles the first schema change — a version in the pin path, or delete and recreate on mismatch?

  4. When a netlink subscription channel closes, Watch sets it to nil and keeps looping. Once both close it selects on ctx alone and watches nothing, and nothing logs that. If Watch returns an error, the goroutine in StartWatching logs once and exits, and nothing subscribes again. Health covers the program, the maps and the filter, but not whether the watcher still runs. Should a dead watcher fail health?

  5. Only netlink events drive re-attachment. tc filter del raises no link or route event, so nothing reconciles until some unrelated event arrives. Health notices, the liveness probe fires, the container restarts, and startup re-attaches — so a cleared filter costs a restart. Should a failed health check drive a reconcile before it fails the probe?

  6. Preflight refuses to start without /sys/kernel/btf/vmlinux, but feat: Add TC-BPF uSID program and bit-layout library #281's package doc says the program reads no kernel structs and needs no vmlinux.h. One of the two is wrong. On a kernel that does not expose BTF, does this refuse to run a datapath that would have worked?

  7. Auto-detection lists IPv6 routes from the main table only. A default route in another table stays invisible, and an underlay carrying specific prefixes and no default offers nothing to find. Both end in a fatal startup that only the environment override clears. Is the underlay default always in the main table on these nodes?

  8. ensureClsact lists qdiscs and then adds one. Another agent adding clsact between those two calls turns QdiscAdd into an EEXIST error. Cilium adds clsact to these same devices. Worth treating EEXIST as success?

Smaller: ResolveInterfaces dumps the whole IPv6 route table on every health tick and every debounced event. And CLOCK_MONOTONIC reads right to me — it survives a container restart, the pins die with the node, and the zero fallback leaks rather than misdelivers.

@ecv ecv left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and it LIKED this one 😂

privateip added a commit that referenced this pull request Aug 6, 2026
Removes the legacy per-route netlink SRv6 ingress mechanism
(srv6.RouteIngressAdd/Del, srv6.go) entirely and replaces it with
registration against the eBPF uSID datapath's pinned maps (attach,
usidmap -- #283): the eBPF/TC-BPF datapath is now the only ingress/decap
path for both veth and tap attachments, so there's no dual-path
coexistence to maintain.

ComputeSID (internal/plumbing/srv6/usid.go) is rewritten onto the
shared uFMT 48+16 bit layout (internal/plumbing/ebpf/uformat -- #281)
instead of its previous ad hoc NodeID/VRFID/Function suffix, so the BGP
control plane and the eBPF dataplane can never drift on bit positions.

The CNI ADD path no longer derives the VRFID straight from the
VPCAttachment identifier (vrfIDFromAttachment); it now allocates a
12-bit uFMT Argument per-node from live BGPVRFInstance CRD state
(allocateArgument), with a collision check (checkArgumentCollision)
covering the allocate-then-create race between two concurrent ADDs.
registerEBPFDatapath/unregisterEBPFDatapath write and roll back the
three eBPF map entries (locator_table, function_table, vrf_table) for
each attachment.

Stacked on #283 (eBPF datapath control-plane packages) and #284
(BGPAdvertisement prefix-merge fix, needed for this PR's
publishBGPStateK8s changes to apply cleanly).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
privateip added a commit that referenced this pull request Aug 6, 2026
Adds SweepEBPFVRFTable, a GC pass that reconciles the eBPF uSID
datapath's vrf_table map entries against live BGPVRFInstance CRDs
using a generation-cutoff scheme to avoid a register/sweep race. This
runs from galactic-cni's "run" container rather than
galactic-router's existing GC controller: the pinned vrf_table map
only exists inside that container, which has the /sys/fs/bpf hostPath
mount and CAP_BPF galactic-router's DaemonSet does not need for
anything else. routerNamesForNode gains a fuller sibling,
routersForNode, since the sweep needs each router's full
Spec.SRv6Locator, not just its name.

Wires installer.Run to load/attach/pin the eBPF datapath at startup
(startEBPFDatapath), serve /metrics (Prometheus), report an
"ebpf-datapath" gRPC health sub-service, and run the GC sweep on its
own ticker. Adds the --metrics-port CLI flag to galactic-cni run.

Depends only on #283 (attach/usidmap/metrics/uformat) -- no
dependency on the CNI ADD cutover in #285/#286, since this reconciles
directly against BGPRouter/BGPVRFInstance CRD state rather than
anything registerEBPFDatapath writes to resourceTracker.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
privateip added a commit that referenced this pull request Aug 6, 2026
Removes the legacy per-route netlink SRv6 ingress mechanism
(srv6.RouteIngressAdd/Del, srv6.go) entirely and replaces it with
registration against the eBPF uSID datapath's pinned maps (attach,
usidmap -- #283): the eBPF/TC-BPF datapath is now the only ingress/decap
path for both veth and tap attachments, so there's no dual-path
coexistence to maintain.

ComputeSID (internal/plumbing/srv6/usid.go) is rewritten onto the
shared uFMT 48+16 bit layout (internal/plumbing/ebpf/uformat -- #281)
instead of its previous ad hoc NodeID/VRFID/Function suffix, so the BGP
control plane and the eBPF dataplane can never drift on bit positions.

The CNI ADD path no longer derives the VRFID straight from the
VPCAttachment identifier (vrfIDFromAttachment); it now allocates a
12-bit uFMT Argument per-node from live BGPVRFInstance CRD state
(allocateArgument), with a collision check (checkArgumentCollision)
covering the allocate-then-create race between two concurrent ADDs.
registerEBPFDatapath/unregisterEBPFDatapath write and roll back the
three eBPF map entries (locator_table, function_table, vrf_table) for
each attachment.

Stacked on #283 (eBPF datapath control-plane packages) and #284
(BGPAdvertisement prefix-merge fix, needed for this PR's
publishBGPStateK8s changes to apply cleanly).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
privateip added a commit that referenced this pull request Aug 6, 2026
Register was a blind overwrite: every call, including the CNI ADD
retry path re-registering after a transient k8s-op failure, wrote a
freshly zeroed value struct, resetting Packets/Bytes/LastSeenNs (and
the never-yet-surfaced DroppedPackets) to zero. Those counters are
usid_ingress's own, and R8's make-before-break migration gate reads
them to prove an Argument carried no traffic before cutover -- a
re-registered entry that had actually been carrying live traffic would
read as untouched.

Register now looks the key up first and carries its existing counters
forward, only overwriting VRFTableID/EgressKind and bumping
Generation. A genuinely new key (Lookup returns ebpf.ErrKeyNotExist)
still starts every counter at zero. DroppedPackets is now also decoded
by Get/List, matching Packets/Bytes/LastSeenNs.

ecv's review of #283, point 1.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
privateip added a commit that referenced this pull request Aug 6, 2026
…t EEXIST

Three independent attach-path fixes from ecv's review of #283, bundled
together since all three land in attach.go/usid.c:

1. (point 2) usid_ingress's fail-open paths (not IPv6, too short to
   parse, no locator_table match -- design plan R6) returned
   TC_ACT_OK. This filter attaches direct-action at a fixed tc
   priority, and Cilium attaches its own tc/bpf programs to the same
   native-device ingress hook on these hosts; in direct-action mode
   TC_ACT_OK is a final verdict that ends the qdisc's filter chain, so
   a packet this program doesn't claim would never reach a colocated
   Cilium filter at a later priority. Switched every pre-locator-match
   fail-open path to TC_ACT_UNSPEC, which hands off to the next filter
   instead. Every fail-open path *after* the locator_table match (this
   program has claimed the packet) is unaffected -- those are, and
   remain, TC_ACT_SHOT. Also made the filter's tc priority overridable
   via GALACTIC_CNI_EBPF_FILTER_PRIORITY (default unchanged at 1),
   mirroring the same override internal/vmtap/config.go already
   exposes for its own Cilium-priority-collision risk.

2. (point 3) Load pinned every map by name and treated any schema
   mismatch against an existing pin (ebpf.ErrMapIncompatible -- e.g. a
   changed value struct size or max_entries) as fatal. vrf_value grew
   egress_kind this cycle and vrf_table sizing is an open question on
   #287, so the next such change would crashloop every node until an
   operator manually deleted the stale pins. Load now unpins and
   recreates any incompatible map before retrying the load once --
   every map here is control-plane-owned and reconstructable
   (usidmap.Register/the GC controller repopulate it), so losing its
   contents across a schema change is the correct trade-off against a
   crashloop.

3. (point 8) ensureClsact listed qdiscs then unconditionally added one
   if absent, racy against any other agent (notably Cilium) doing the
   same thing to the same device between the two calls. QdiscAdd
   returning EEXIST in that window is now treated as success, since
   the qdisc ensureClsact wanted to exist now does.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
privateip added a commit that referenced this pull request Aug 6, 2026
…ilure

Watch's netlink-driven reconcile loop had two gaps ecv's review of
#283 flagged (points 4 and 5):

- If both the link and route subscriptions closed, Watch silently
  degraded to reacting only to ctx.Done(), with no log line marking
  the transition. If Watch itself returned an error (its initial
  subscriptions failing), the goroutine logged once and exited for
  good, with no way for anything outside the package to tell the
  watch loop had died -- Health checked the program, maps, and filter,
  but never the watcher.

- Nothing but an unrelated netlink link/route event ever triggered a
  re-evaluation: a `tc filter del` alone raises neither, so a cleared
  filter could sit unfixed until some unrelated event happened to fire
  or the liveness probe restarted the container. A failing health
  check never tried to heal anything itself.

Adds a Watcher type (returned by StartWatching alongside the
objects/interfaces Start already returns): Alive() reports whether the
Watch loop is still actually running, and Reconcile() requests an
out-of-band re-evaluation through the same debounced path a real
netlink event uses. Watch now logs when a subscription closes (and
specifically when both have, since that's the point it can no longer
react to anything on its own) and marks the Watcher dead on any exit.

Handle gained a Watcher field: Healthy() now reports unhealthy if the
watcher has died (it can't self-heal drift anymore), and nudges it via
Reconcile() whenever Health's own checks fail -- so a self-healable gap
gets a chance to heal within one health-check interval instead of
waiting for an unrelated netlink event or a container restart.
internal/installer wires ebpfState's Watcher into the health-check
ticker's Handle.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
privateip added a commit that referenced this pull request Aug 6, 2026
Interface auto-detection only listed IPv6 routes from the main table
(vishvananda/netlink's own default for RouteList), so a default route
living in another table -- e.g. a VRF-scoped underlay -- was invisible
to it, ending in an actionable-but-avoidable fatal startup that only
GALACTIC_CNI_EBPF_INTERFACES could clear. routeListFn now passes
RT_FILTER_TABLE with an unfiltered (RT_TABLE_UNSPEC) Table, which lifts
netlink's non-main-table skip instead of narrowing to one table --
autoDetectInterfaces still just looks for any IPv6 default route,
wherever it lives.

ecv's review of #283, point 7.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
privateip added a commit that referenced this pull request Aug 6, 2026
TestResolveSRv6SID's fixtures were never updated when ComputeSID was
rewritten onto uformat's uFMT 48+16 bit layout (Node-ID 16 bits,
Function 4 bits, Argument 12 bits) instead of its previous ad hoc
NodeID(8)/VRFID(16)/Function(8) suffix: the expected SID string still
reflected the old layout, and the out-of-range NodeID case (255) no
longer exceeds the new, much wider NodeIDMax (0xDFFF), so ComputeSID
no longer errors on it.

Updated the expected SID to the uFMT-correct value (Function
0xE == uformat.FunctionEndDT46 lands in the high nibble of the
Function/Argument group, not 0) and moved the out-of-range case to
0xE000 (one past NodeIDMax).

Pre-existing since the ComputeSID rewrite, unrelated to ecv's review
of #283 -- found while rebasing this branch onto main.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@privateip
privateip requested a review from ecv August 6, 2026 20:37
Base automatically changed from pr1-ebpf-ci-toolchain to main August 6, 2026 21:12
privateip and others added 10 commits August 6, 2026 17:12
Adds the Go control-plane library for the TC-BPF uSID datapath:

- preflight: kernel capability gate (BTF, HASH maps, SCHED_CLS,
  bpf_fib_lookup with VRF-tbid support) run before attempting to load
  the program at all.
- usidmap: typed read/write/reconcile API for the three kernel maps
  (locator_table, function_table, vrf_table) the program consults.
- attach: load/pin/attach/detach/watch lifecycle for the TC-BPF
  ingress hook, including netlink-driven re-attachment on interface or
  route change, gated by the new GALACTIC_CNI_EBPF_INTERFACES env var
  (internal/config) for multi-homed nodes where auto-detection is
  ambiguous.
- metrics: Prometheus collector plus load/attach event counters.

Every package here is independently unit-testable against fakes
(faketable_test.go, a mock kernel prober, fake closers) -- nothing
outside this tree calls any of it yet.

Stacked on #281 (usid.c/uformat) and #282 (codegen toolchain).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds SweepEBPFVRFTable, a GC pass that reconciles the eBPF uSID
datapath's vrf_table map entries against live BGPVRFInstance CRDs
using a generation-cutoff scheme to avoid a register/sweep race. This
runs from galactic-cni's "run" container rather than
galactic-router's existing GC controller: the pinned vrf_table map
only exists inside that container, which has the /sys/fs/bpf hostPath
mount and CAP_BPF galactic-router's DaemonSet does not need for
anything else. routerNamesForNode gains a fuller sibling,
routersForNode, since the sweep needs each router's full
Spec.SRv6Locator, not just its name.

Wires installer.Run to load/attach/pin the eBPF datapath at startup
(startEBPFDatapath), serve /metrics (Prometheus), report an
"ebpf-datapath" gRPC health sub-service, and run the GC sweep on its
own ticker. Adds the --metrics-port CLI flag to galactic-cni run.

Depends only on #283 (attach/usidmap/metrics/uformat) -- no
dependency on the CNI ADD cutover in #285/#286, since this reconciles
directly against BGPRouter/BGPVRFInstance CRD state rather than
anything registerEBPFDatapath writes to resourceTracker.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Removes the legacy per-route netlink SRv6 ingress mechanism
(srv6.RouteIngressAdd/Del, srv6.go) entirely and replaces it with
registration against the eBPF uSID datapath's pinned maps (attach,
usidmap -- #283): the eBPF/TC-BPF datapath is now the only ingress/decap
path for both veth and tap attachments, so there's no dual-path
coexistence to maintain.

ComputeSID (internal/plumbing/srv6/usid.go) is rewritten onto the
shared uFMT 48+16 bit layout (internal/plumbing/ebpf/uformat -- #281)
instead of its previous ad hoc NodeID/VRFID/Function suffix, so the BGP
control plane and the eBPF dataplane can never drift on bit positions.

The CNI ADD path no longer derives the VRFID straight from the
VPCAttachment identifier (vrfIDFromAttachment); it now allocates a
12-bit uFMT Argument per-node from live BGPVRFInstance CRD state
(allocateArgument), with a collision check (checkArgumentCollision)
covering the allocate-then-create race between two concurrent ADDs.
registerEBPFDatapath/unregisterEBPFDatapath write and roll back the
three eBPF map entries (locator_table, function_table, vrf_table) for
each attachment.

Stacked on #283 (eBPF datapath control-plane packages) and #284
(BGPAdvertisement prefix-merge fix, needed for this PR's
publishBGPStateK8s changes to apply cleanly).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The eBPF uSID ingress datapath decapsulates SRv6 traffic and calls
bpf_fib_lookup() to resolve the egress path for the inner packet, then
redirects it straight to the resolved neighbor entirely in-kernel,
never touching the normal forwarding stack. bpf_fib_lookup() does not
itself trigger ARP/NDP resolution the way ordinary kernel packet
forwarding does, so without a pre-existing neighbor table entry it
fails with BPF_FIB_LKUP_RET_NO_NEIGH and the packet is dropped --
every cross-region packet to a pod that had never otherwise triggered
NDP for its own address was silently and permanently blackholed.

installGatewayNeighbor installs a permanent neighbor table entry
mapping the pod's address to its guest veth's own known MAC at CNI
ADD, so this resolution never depends on dynamic ARP/NDP. guestHWAddr
now flows from buildVethResult through publishBGPState into
configureHostGateway; nil for tap attachments, which have no separate
guest-side link in this netns to resolve a MAC from.

Stacked on #285 (eBPF uSID registration cutover), since the neighbor
entry only matters once the eBPF datapath is the one doing the fib
lookup.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Grants the CNI DaemonSet's credential-refresh container CAP_BPF and
CAP_NET_ADMIN plus a bpf-fs hostPath mount (/sys/fs/bpf), and adds an
"ebpf-datapath" gRPC health sub-service to its liveness/readiness
probes -- required unconditionally now that this container also hosts
the eBPF/TC-BPF uSID datapath's load/attach/pin control daemon
(installer.Run, #287); there's no flag left to gate this behind, since
the datapath is the only forwarding path.

Pins GALACTIC_CNI_EBPF_INTERFACES=eth1 for the containerlab topology,
where every lab node is dual-homed and interface auto-detection picks
the wrong (management-bridge) link over the actual transit-fabric one.

Mounts bpffs on Kind nodes in CI (scripts/ci.sh) and updates
TestCNITapInterface to run a privileged e2e pod with its own bpf-fs
mount, starting the eBPF control daemon and waiting for vrf_table to
be pinned before exercising CNI ADD -- the datapath being the only
forwarding path means the e2e pod needs the same maps a production
node's DaemonSet would already have pinned. Adds test-unit-root to
test-e2e's dependency list.

Stacked on #286 (CNI eBPF cutover + gateway fix) and #287 (GC/installer
wiring) -- the visible diff includes both until they merge upstream;
review focuses on the deploy/e2e files listed above.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Register was a blind overwrite: every call, including the CNI ADD
retry path re-registering after a transient k8s-op failure, wrote a
freshly zeroed value struct, resetting Packets/Bytes/LastSeenNs (and
the never-yet-surfaced DroppedPackets) to zero. Those counters are
usid_ingress's own, and R8's make-before-break migration gate reads
them to prove an Argument carried no traffic before cutover -- a
re-registered entry that had actually been carrying live traffic would
read as untouched.

Register now looks the key up first and carries its existing counters
forward, only overwriting VRFTableID/EgressKind and bumping
Generation. A genuinely new key (Lookup returns ebpf.ErrKeyNotExist)
still starts every counter at zero. DroppedPackets is now also decoded
by Get/List, matching Packets/Bytes/LastSeenNs.

ecv's review of #283, point 1.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…t EEXIST

Three independent attach-path fixes from ecv's review of #283, bundled
together since all three land in attach.go/usid.c:

1. (point 2) usid_ingress's fail-open paths (not IPv6, too short to
   parse, no locator_table match -- design plan R6) returned
   TC_ACT_OK. This filter attaches direct-action at a fixed tc
   priority, and Cilium attaches its own tc/bpf programs to the same
   native-device ingress hook on these hosts; in direct-action mode
   TC_ACT_OK is a final verdict that ends the qdisc's filter chain, so
   a packet this program doesn't claim would never reach a colocated
   Cilium filter at a later priority. Switched every pre-locator-match
   fail-open path to TC_ACT_UNSPEC, which hands off to the next filter
   instead. Every fail-open path *after* the locator_table match (this
   program has claimed the packet) is unaffected -- those are, and
   remain, TC_ACT_SHOT. Also made the filter's tc priority overridable
   via GALACTIC_CNI_EBPF_FILTER_PRIORITY (default unchanged at 1),
   mirroring the same override internal/vmtap/config.go already
   exposes for its own Cilium-priority-collision risk.

2. (point 3) Load pinned every map by name and treated any schema
   mismatch against an existing pin (ebpf.ErrMapIncompatible -- e.g. a
   changed value struct size or max_entries) as fatal. vrf_value grew
   egress_kind this cycle and vrf_table sizing is an open question on
   #287, so the next such change would crashloop every node until an
   operator manually deleted the stale pins. Load now unpins and
   recreates any incompatible map before retrying the load once --
   every map here is control-plane-owned and reconstructable
   (usidmap.Register/the GC controller repopulate it), so losing its
   contents across a schema change is the correct trade-off against a
   crashloop.

3. (point 8) ensureClsact listed qdiscs then unconditionally added one
   if absent, racy against any other agent (notably Cilium) doing the
   same thing to the same device between the two calls. QdiscAdd
   returning EEXIST in that window is now treated as success, since
   the qdisc ensureClsact wanted to exist now does.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ilure

Watch's netlink-driven reconcile loop had two gaps ecv's review of
#283 flagged (points 4 and 5):

- If both the link and route subscriptions closed, Watch silently
  degraded to reacting only to ctx.Done(), with no log line marking
  the transition. If Watch itself returned an error (its initial
  subscriptions failing), the goroutine logged once and exited for
  good, with no way for anything outside the package to tell the
  watch loop had died -- Health checked the program, maps, and filter,
  but never the watcher.

- Nothing but an unrelated netlink link/route event ever triggered a
  re-evaluation: a `tc filter del` alone raises neither, so a cleared
  filter could sit unfixed until some unrelated event happened to fire
  or the liveness probe restarted the container. A failing health
  check never tried to heal anything itself.

Adds a Watcher type (returned by StartWatching alongside the
objects/interfaces Start already returns): Alive() reports whether the
Watch loop is still actually running, and Reconcile() requests an
out-of-band re-evaluation through the same debounced path a real
netlink event uses. Watch now logs when a subscription closes (and
specifically when both have, since that's the point it can no longer
react to anything on its own) and marks the Watcher dead on any exit.

Handle gained a Watcher field: Healthy() now reports unhealthy if the
watcher has died (it can't self-heal drift anymore), and nudges it via
Reconcile() whenever Health's own checks fail -- so a self-healable gap
gets a chance to heal within one health-check interval instead of
waiting for an unrelated netlink event or a container restart.
internal/installer wires ebpfState's Watcher into the health-check
ticker's Handle.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Interface auto-detection only listed IPv6 routes from the main table
(vishvananda/netlink's own default for RouteList), so a default route
living in another table -- e.g. a VRF-scoped underlay -- was invisible
to it, ending in an actionable-but-avoidable fatal startup that only
GALACTIC_CNI_EBPF_INTERFACES could clear. routeListFn now passes
RT_FILTER_TABLE with an unfiltered (RT_TABLE_UNSPEC) Table, which lifts
netlink's non-main-table skip instead of narrowing to one table --
autoDetectInterfaces still just looks for any IPv6 default route,
wherever it lives.

ecv's review of #283, point 7.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
TestResolveSRv6SID's fixtures were never updated when ComputeSID was
rewritten onto uformat's uFMT 48+16 bit layout (Node-ID 16 bits,
Function 4 bits, Argument 12 bits) instead of its previous ad hoc
NodeID(8)/VRFID(16)/Function(8) suffix: the expected SID string still
reflected the old layout, and the out-of-range NodeID case (255) no
longer exceeds the new, much wider NodeIDMax (0xDFFF), so ComputeSID
no longer errors on it.

Updated the expected SID to the uFMT-correct value (Function
0xE == uformat.FunctionEndDT46 lands in the high nibble of the
Function/Argument group, not 0) and moved the out-of-range case to
0xE000 (one past NodeIDMax).

Pre-existing since the ComputeSID rewrite, unrelated to ecv's review
of #283 -- found while rebasing this branch onto main.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@privateip
privateip merged commit 7139fc2 into main Aug 6, 2026
@privateip
privateip deleted the pr2-ebpf-datapath branch August 6, 2026 21:37
privateip added a commit that referenced this pull request Aug 7, 2026
…ot tests

Two unrelated pre-existing CI failures surfaced on this PR, the first
one since #283 (feat(ebpf): usid TC-BPF program) actually ran the
"Build"/"Unit Tests (root)" jobs against the merged result of that PR
-- neither job's checks ever recorded against #283's own commits, only
against its "Publish Docker Image" runs, so both gaps below shipped to
main unnoticed:

1. internal/plumbing/ebpf/prog/usid_bpfeb.o and usid_bpfel.o were
   stale relative to usid.c under the Build job's pinned clang-18/
   llvm-18 toolchain (task build's build:ebpf step silently skips
   regeneration whenever clang isn't installed, per its own comment --
   whatever environment last committed these apparently didn't have
   it, or had a different clang version, either of which produces
   different BPF object bytes for the same source per doc.go's own
   BPF2GO_CC rationale). Regenerated both via the exact pinned
   toolchain (clang-18, in an ubuntu:24.04 container matching the
   runner) so they match what the Build job's drift check expects; no
   change to usid.c or the generated .go bindings.

2. internal/cni's bgp_ebpf_test.go creates a real netlink.Vrf link
   (vrf.Add), which fails with "operation not supported" on the
   Actions runner: the kernel doesn't autoload the vrf module via
   request_module() from inside the netlink path there, and nothing in
   the Unit Tests (root) job loaded it explicitly (scripts/ci.sh's
   e2etest case already does `sudo modprobe vrf` for the same
   underlying reason -- this job just never had the equivalent). Added
   the same modprobe step here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants