Feature 1093: stream engine config options v5 #801
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The stream reassembly engine uses a set of pools in which preallocated
segments are stored. There are various pools each with different packet
sizes. The goal is to lower memory presure. Until now, these pools were
hardcoded.
This patch introduces the ability to configure them fully from the yaml.
There can be at max 256 of these pools.
Yaml layout is as follows:
stream:
reassemble:
segments:
- size: 2048
prealloc: 3000
- size: 4
prealloc: 1000
- size: 1024
prealloc: 2000
The size is the packet size. The prealloc value indicates how many
segments are set up at startup.
The pools have no limit wrt how many segments can be used of a certain
size. If the engine needs more than the prealloc size, segments are
malloc'd and free'd. The only limit here is the stream.reassemble.memcap.
If the yaml part if omitted, the default values are the same as before.
Feature #1093
The stream chunk pool contains preallocating stream chunks (StreamMsg). These are used for raw reassembly, used in raw content inspection by the detection engine. The default setting so far has been 250, which was hardcoded. This meant that in setups that needed more, allocs and frees would be happen constantly. This patch introduces a yaml option to set the 'prealloc' value in the pool. The default is still 250. stream.reassembly.chunk-prealloc Related to feature #1093.
TL-Yao
pushed a commit
to scantist-ossops-m2/suricata
that referenced
this pull request
Apr 15, 2024
Bug OISF#801 Flowvars are set from pcre, and lock the flow when being set. However when HTTP buffers were inspected, flow was already locked: deadlock. This patch introduces a post-match list in the detection engine thread ctx, where store candidates are kept. Then a post-match function is used to finalize the storing if the rule matches. Solves the deadlock and brings the handling of flowvars more in line with flowbits and flowints.
TL-Yao
pushed a commit
to scantist-ossops-m2/suricata
that referenced
this pull request
Apr 15, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes since #799:
PRscript: