Design

This is a quick attempt to define ExaBGP's internal .. This document should improve over time.

ExaBGP runs a single loop, the loop is managed by the reactor which will share time between the network and the management of the forked process.

As the code needs to be non blocking, extensive use of a feature called co-routine is used. A python co-routine is a resumable function, instead of returning a value a function yields it and at the next call the execution resumes from that yield point. Those functions, when run, create a generator, which is an iterator object that will, when requested, by using the function next() on it, yield the next value. The good thing is that it is possible to generate and consume the value (like for unix pipes) without using massive amounts of memory.

The reactor and networking code use co-routines. The networking code is split in three levels, connection, protocol, and peer :

• connection provides (using co-routine) a non-blocking I/O layer. This is a forever running generator which will return nothing when nothing is there, or data when data is found. It uses unix select(2) on a non-blocking socket.
• protocol wraps connection to provide a BGP message level api, it returns a NOP message when no new messages are available or a Message subclass when new messages can be fully formed ( Open, Update, KeepAlive, etc..)
• each peer gets one instance of Peer which then performs the protocol discussion, again using a co-routine.
• the reactor then manages all the peers and the Process forked and dispatches the messages between them.

The whole code in peer is wrapped in a try / except block. On error an exception is raised and will be intercepted by that block which will send a notification to the other side, tear down the session and re-establish it if required.

The forked process is able to generate a very large amount of data so in order to be fair the reactor runs the cycle for one second, which guarantee that each second KeepAlive (heartbeat between routers) is performed. Up to 0.5 seconds, the reactor consumes data from the PIPE, for the rest of the second, ExaBGP handles the networking I/O, should the networking work not take up to a second more data is consumed from the PIPE. If data should not be there, ExaBGP will then sleep to the end of the second.

When routes to be announced to the peer are generated by the PIPE, the data is parsed and a single NLRI with its attributes, called a Change, is generated. This change is added by the reactor to the outgoing RIB of the peers needing to announce it. The Changes are grouped by attributes, so the Update message created can network efficiently. When the reactor comes to handle each peer, each RIB generates a co-routine which is consumed by the peer networking code, obviously the consumption code is a co-routine itself managed by the peer so this creation is non-blocking too.