Issue drop to packet as early as possible.

(cherry picked from commit 71a033a)

Now that flow drop is applied to packets before other processing,
no drop has to be issued on a packet.

(cherry picked from commit 85ddba6)

(cherry picked from commit 77f4966)

Remove logic to apply flow drop, as this is now handled in the
flow engine.

However, keep the logic that frees/cleans the session state.

(cherry picked from commit d91a1e8)

When a flow is in the drop flow state, don't use pseudo packets
when it is timing out. There should be no work left to do at this
point.

(cherry picked from commit 2a95154)

(cherry picked from commit 418cc1f)

(cherry picked from commit 95bf724)

Test broke after recent changes. Functionality is tested in
suricata-verify, so just remove the test.

(cherry picked from commit 8a535a0)

(cherry picked from commit 13fe957)

(cherry picked from commit fd93f00)

So far, we store one variable in state to hold whether we want to
discard a long line till LF irrespective of direction. This means that a
long command to the client followed by a regular command w LF can be
considered as one long line which is incorrect.

Bug 6055

In case LF was found for a long line way outside of the limit, we should
not need to update the delimiter len and current line len because the
line is capped at 4k and the LF was not within these 4k bytes.

Rust 1.70 has introduced some possible issues between LLVM and gcc
causing link errors that are fixed by explicitly adding -lntdll.

Thanks to extendr/rextendr#285 for the fix.

(cherry picked from commit 0360cb6)

Counters depended on availability of flow and tcp session, meaning
that 2 memcaps could affect the counters.

Bug: #5017.
(cherry picked from commit 36f6e05)

(cherry picked from commit 39d2524)

Accept DNS messages with an invalid opcode that are otherwise
valid. Such DNS message will create a parser event.

This is a change of behavior, previously an invalid opcode would cause
the DNS message to not be detected or parsed as DNS.

Issue: #5444
(cherry picked from commit c98c49d)

As UDP streams getting probed, a stream that does not appear to be DNS
at first, may have a single packet that does look close enough to DNS
to be picked up as DNS causing every subsequent packet to result in a
parser error.

To mitigate this, probe every incoming DNS message header for validity
before continuing onto the body.  If the header doesn't validate as
DNS, just ignore the packet so no parse error is registered.

(cherry picked from commit 595700a)

As part of extra header validation, split out DNS body parsing to
avoid the overhead of parsing the header twice.

(cherry picked from commit d720ead)

When adding an element to the dynamic headers table, the oldest
ones may get evicted. When multiple elements get evicted, they
should get evicted all at once with drain, instead of one by one
as there will be a massive move each time.

Ticket: #6103
(cherry picked from commit 6350736)

Allows for more efficient removal from front of the list.

Ticket: #5753
(cherry picked from commit 1d91836)

Allows for more efficient removal from front of the list.

Ticket: #5277
(cherry picked from commit 3189414)

Allows for more efficient removal from front of the list.

Ticket: #5295
(cherry picked from commit 4e0ad5e)

Allows for more efficient removal from front of the list.

Ticket: #5296
(cherry picked from commit 2db8472)

Allows for more efficient removal from front of the list.

Ticket: #5298
(cherry picked from commit e319d31)

Allows for more efficient removal from front of the list.

Ticket: #5271
(cherry picked from commit dfe76bb)

As was done for TCP in dfe76bb and d745d28

Ticket: #5518
(cherry picked from commit bf43011)

Issue: 6070

This commit adds support for the multiplication operator to byte-math.
The regex for parsing the keyword options was missing the `*` character.

Set the IPv6 packet proto before parsing the ext headers, similar to
decode-ipv4, incase of an ext header parsing error. Otherwise
rule decode-events are not triggered for packets encapsulated in IPv6.

Bug: #6086.
(cherry picked from commit 531d99f)

With the move to installing Suricata-Update files from Suricata
Makefile's, the sample configuration files were forgotten.

Ticket: #6132
(cherry picked from commit b8071a9)

Check that the Suricata-Update example configuration files are
installed.

(cherry picked from commit 6d7923c)

Before:
If the input was such that we'd enter DATA mode in the middle, the
entire data would be passed through SMTPGetLine fn and be processed with
line limits etc in place.

After:
Since we don't want any limits to be enforced on DATA, we pass it to
SMTPPreProcessCommands fn to take care of it differently from the
commands.

Bug 5981

If a long line had LF post the limit, it should be considered complete
and not wait for the next line to complete it. However, currently, any
following lines were skipped which could sometimes also be important
commands for the entire transaction.

Fix this by setting a flag in case we're truncating a long line but
after having found the LF character.

Bug 5989

Issue:
Currently, while handling of long lines, if the line exceeded the limit,
we'd set a variable state->discard_till_lf which will be reset in the
later stages based on the data that arrives. However, because there was
one variable per state, this meant that a later stage in the other
direction could also modify it which is incorrect.

Fix:
Use separate variables for each direction.

Bug 6053

To protect against possible supply chain attacks, disable Lua rules by
default. They can be enabled under the "security" section of
suricata.yaml.

Ticket: #6122

For dataset filenames coming from rules, do not allow filenames that
are absolute or contain a directory traversal with "..". This prevents
datasets from escaping the define data-directory which may allow a bad
rule to overwrite any file that Suricata has permission to write to.

Add a new configuration option,
"datasets.rules.allow-absolute-filenames" to allow absolute filenames
in dataset rules. This will be a way to revert back to the pre 6.0.13
behavior where save/state rules could use any filename.

Ticket: #6118

On installation, make sure the data directory is created. This will
usually be /var/lib/suricata/data, but otherwise follows the
autoconf/automake instructions.

This directory is for runtime state information, which for now is
datasets but may be expanded in the future.  Suricata already expects
this directory to exist for "state" and "save" datasets, but it has
been up to the user to create it.

Add a new configuration flag, "datasets.rules.allow-write" to control
if rules can contain "save" or "state" rules which allow write access
to the file system.

Ticket: #6123

Uncomment the datasets configuration for easier editing by users.  The
values are left commented out as their defaults.

This is the start of a security consideration chapter, starting with
directions on how to run Suricata as a non-root user.