Skip to content

Net ids are reused before their teardown has run, so a late teardown can delete a live edge #146

Description

@GyulyVGC

send_net_teardown returns the net id to the pool as soon as the teardown messages are enqueued (orchestrator.rs:665-678) — fire-and-forget, no ack.
The pool then hands that id straight back out, and the client tokio::spawns every inbound message (control_channel.rs:71-127), so gRPC arrival order does not imply execution order.
Teardown for generation N and setup for generation N+1 can therefore run against the same id concurrently, or in the wrong order.

Nothing distinguishes one generation from the next: br_<id>_<side>, veth-<id>-s/c, macsec-<id>-s/c, the derived MACs, the SPI (<id>+1000) and the per-net lock path all derive from the id alone.
A stale teardown deletes the new generation's devices, leaving a dead edge that the control plane still reports as established.
Callers hang until their own timeout and surface an opaque 5xx.

Worse case: if the destination client is not connected, send_net_teardown sends nothing at all and frees the id anyway, so the old edge stays live in the kernel and the next allocation collides with it.

Both NetIdPool::allocate and UdpPortPool::allocate reuse the lowest freed value, which maximises how quickly an id comes back round.

Fixes, in order of value:

  • serialize per net id on the client instead of spawning every message (no protocol change; restores the ordering gRPC already delivers)
  • ack the teardown before freeing the id, mirroring the existing ContainerResume pattern
  • FIFO allocation in both pools, so a freed id is the last to come back rather than the first

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Fields

Priority

High

Effort

Medium

Nullnet member

nullnet-client

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions