route reflection

Pre-Alpha. This page describes behavior that may change.

Ze operates as a route server (RFC 7947) through the bgp-rs plugin. The plugin forwards every received route to every other peer, with zero-copy wire forwarding when peers share an encoding context. There is no best-path selection in the route server path: the whole point is that the downstream routers make their own decisions.

Route reflection (the iBGP case, RFC 4456) uses the same plugin configured differently: you add cluster-id to the reflector and treat its iBGP sessions like a route server with ADD-PATH. The operator story is similar but the protocol details differ.

A minimal route server

`bgp-rs` and `bgp-adj-rib-in` are internal plugins loaded automatically.

bgp {
    peer client-a {
        remote { ip 10.0.0.1; as 65001; }
        local  { ip 10.0.0.254; as 65000; }

        family { ipv4/unicast { prefix { maximum 1000000; } } }

        process bgp-rs         { receive [ update ]; send [ update ] }
        process bgp-adj-rib-in { receive [ update state ] }
    }

    peer client-b {
        remote { ip 10.0.0.2; as 65002; }
        local  { ip 10.0.0.254; as 65000; }

        family { ipv4/unicast { prefix { maximum 1000000; } } }

        process bgp-rs         { receive [ update ]; send [ update ] }
        process bgp-adj-rib-in { receive [ update state ] }
    }
}

Every peer gets both plugin bindings. bgp-rs does the forwarding. bgp-adj-rib-in stores the received routes so they can be replayed when a peer reconnects.

The forward-all model

The route server forwards every received route to every other peer. There is no best-path selection. This is RFC 7947 and it is what IXP members expect: the IXP is a transparent bus that propagates routes between members, and the members run their own policy.

The combination of the route server model and ADD-PATH is what lets every downstream make its own best-path decision. Without ADD-PATH, you can only forward one path per prefix to each client. With ADD-PATH, every path is available. See add-path.

Zero-copy forwarding

When two peers negotiate identical capabilities (same ADD-PATH mode, same ASN format, same extended-message support), they share the same encoding context, identified by a ContextID. Routes between peers with matching contexts are forwarded as raw wire bytes: no parse, no rebuild, no allocation. The reactor looks up the cached message, confirms the contexts match, and ships the bytes straight back out.

This is the fast path and the reason a route server with many identically-configured clients runs well on Ze. When contexts differ (for example, one peer without ADD-PATH forwarding to one with ADD-PATH), a progressive build walks the cached attributes and re-encodes for the destination.

Forwarding workers and congestion

Each destination peer has its own long-lived forwarding worker goroutine with a buffered channel. When a destination peer is slower than the update rate, the channel absorbs short bursts (default capacity 4096 items). If the channel fills, items go into a per-worker overflow buffer, and the worker fires a congestion event visible in the logs and in Prometheus.

The overflow uses a two-tier pool. Per-peer pools (64 slots) absorb steady-state traffic. A shared MixedBufMux pool, auto-sized from peer prefix maximums or overridable via ze.fwd.pool.size, bounds overflow memory across every peer. Routes are never dropped: missing a route update is worse than using extra memory. When a peer catches up and the channel drains below 25%, congestion clears.

The Prometheus metrics that show this behaviour:

Metric	Meaning
`ze_bgp_pool_used_ratio`	Global overflow pool utilisation.
`ze_bgp_overflow_items{peer}`	Items in a per-destination overflow buffer.
`ze_bgp_overflow_ratio{source}`	Per-source overflow ratio.

If you see ze_bgp_pool_used_ratio above 0.8 in steady state, you are close to congestion teardown and should investigate which peer is lagging.

Convergent replay

When a peer reconnects, the route server replays every stored route from the other peers. The replay is three phases:

Full snapshot replay from the Adj-RIB-In.
Delta loop that catches routes received during the replay.
End-of-RIB sent when the replay has caught up.

This is what gives RFC 4724-style recovery semantics at the route server layer: a reconnecting peer gets everything the server currently holds for the other peers, in the order that respects RFC convergence.

The plugin bindings in detail

bgp-rs plugin. Needs receive [ update ] (so it sees routes coming in) and send [ update ] (so it can forward routes back out). Every peer that participates in the route server must have these bindings.

bgp-adj-rib-in plugin. Stores received routes for replay on peer reconnect. Needs receive [ update state ]. If you do not bind it, reconnecting peers will not get a replay and convergence will be slower.

Cache commands

Routes in transit are exposed through the cache surface.

cache list
cache forward <id> <peer>
cache release <id>

Use these when you want to inspect or reroute a specific cached message. They are more of a debugging surface than a day-2 operator tool, but they are the mechanism that lets a plugin take control of forwarding decisions for a specific UPDATE.

Without `bgp-rs`

When bgp-rs is not loaded, routes are not forwarded between peers. The bgp-rib plugin (if loaded) stores routes and runs best-path selection, but it does not re-advertise them. Ze becomes a passive route collector in that mode. Load bgp-rs when you want forwarding.