Introduce "eventloop" style API to better handle netlink event storms #499
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@celebdor Antoni, pls take a look at the PR |
added 3 commits
May 14, 2018 15:41
Event queue interface is an alternative to "post" callbacks. While "post" callbacks are executed in a separate thread each, event queue interface follows the "eventloop" metaphor, i.e. each netlink event received in the monitoring thread is put in the queue from which it can be subsequently fetched by calling `nextmsg()` generator function in the main thread (or in any other thread(s) started by the user). In the event of packet storm, it is much nicer to the resources than creating a thread for each received message. Signed-off-by: Eugene Crosser <crosser@average.org>
IPDB reinitialization is rather fragile, and sometimes hangs for a long time, and prevents release() from finishing. When event queue is used, errors in the monitor thread are reported to the user via event queue, so the user can take care of cleanup and reinitialization. Signed-off-by: Eugene Crosser <crosser@average.org>
Introduce keyword arguments to IPRoute() and IPDB() constructors to specify netlink socket send and receive buffer sizes. Defaults are 1048576, 1048576 (one megabyte). Useful for defining strategy for handling netlink packet storms: specify smaller size when early bailout is desired. When used in conjunction with eventqueue, exception will be raised in the main thread when netlink events are lost, allowing for user-controlled recovery procedure. Signed-off-by: Eugene Crosser <crosser@average.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please consider this proposal.
In mass network topology updates, thousands of netlink packets lead to excessive resource usage (one thread per event use much memory and results in slow processing) and hanging process (recovery from failed IPDB commit is slow, and monitoring thread is prone to die without the master thread noticing).
This change allows the user to process incoming netlink messages in an event loop (in the main thread, or in any user-initiated thread(s)), and netlink socket receive errors are passed via the same queue, and re-raised in the master thread. This allows the user program to notice that netlink events where lost and take appropriate action.
The last commit of three allows to specify the size of netlink socket send and receive buffers, and thus control the "storm sensitivity" of the system.
Sidenote: on Linux, system wide max size of socket receive and send buffers is controlled with
/proc/sys/net/core/rmem_max, it may be worth to mention in the documentation.