send_net_teardown returns the net id to the pool as soon as the teardown messages are enqueued (orchestrator.rs:665-678) — fire-and-forget, no ack.
The pool then hands that id straight back out, and the client tokio::spawns every inbound message (control_channel.rs:71-127), so gRPC arrival order does not imply execution order.
Teardown for generation N and setup for generation N+1 can therefore run against the same id concurrently, or in the wrong order.
Nothing distinguishes one generation from the next: br_<id>_<side>, veth-<id>-s/c, macsec-<id>-s/c, the derived MACs, the SPI (<id>+1000) and the per-net lock path all derive from the id alone.
A stale teardown deletes the new generation's devices, leaving a dead edge that the control plane still reports as established.
Callers hang until their own timeout and surface an opaque 5xx.
Worse case: if the destination client is not connected, send_net_teardown sends nothing at all and frees the id anyway, so the old edge stays live in the kernel and the next allocation collides with it.
Both NetIdPool::allocate and UdpPortPool::allocate reuse the lowest freed value, which maximises how quickly an id comes back round.
Fixes, in order of value:
- serialize per net id on the client instead of spawning every message (no protocol change; restores the ordering gRPC already delivers)
- ack the teardown before freeing the id, mirroring the existing
ContainerResume pattern
- FIFO allocation in both pools, so a freed id is the last to come back rather than the first
send_net_teardownreturns the net id to the pool as soon as the teardown messages are enqueued (orchestrator.rs:665-678) — fire-and-forget, no ack.The pool then hands that id straight back out, and the client
tokio::spawns every inbound message (control_channel.rs:71-127), so gRPC arrival order does not imply execution order.Teardown for generation N and setup for generation N+1 can therefore run against the same id concurrently, or in the wrong order.
Nothing distinguishes one generation from the next:
br_<id>_<side>,veth-<id>-s/c,macsec-<id>-s/c, the derived MACs, the SPI (<id>+1000) and the per-net lock path all derive from the id alone.A stale teardown deletes the new generation's devices, leaving a dead edge that the control plane still reports as established.
Callers hang until their own timeout and surface an opaque 5xx.
Worse case: if the destination client is not connected,
send_net_teardownsends nothing at all and frees the id anyway, so the old edge stays live in the kernel and the next allocation collides with it.Both
NetIdPool::allocateandUdpPortPool::allocatereuse the lowest freed value, which maximises how quickly an id comes back round.Fixes, in order of value:
ContainerResumepattern