logging - non-blocking writes to socket - v3 #2651
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Writing to a unix socket can cause Suricata to block in the packet path. This could happen if the read-endpoint of the unix socket stays connected, but stops reading, or simply can't read fast enough as part of its event processing. To choose packets over events, do non-blocking socket writes and drop the event if the write would block and update a dropped counter.
If running against a pcap there is no reason to drop events, a blocking socket is fine here. So only do non-blocking writes when running off a live device.
Merged
|
Merged in #2655, thanks Jason! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previous PR: #2649
Changes from last PR:
Previous Description
One way to address sockets that are blocking on write, (which also blocks Suricata in general) is to drop the event if the write to the socket will block.
This isn't as feature-full as for an internal message queue could be which could buffer the packets until the socket no longer blocks, but in many cases buffering just delays having to drop events, especially if the other end of the socket continually can't keep up.
Here we do socket writes with a non-blocking send and up a counter if it failed to send. The second commit is an example of a logger logging the drop count. But we probably want a real stat for this that is in the event log.
Changes from last PR:
I looked at a stats counter, but it was non-trivial to one via the stats API to do the various entry points into the logger (ie: flow). Will revisit though.
Prscript