Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: add multi buffer matching documentation v4 #9180

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions doc/userguide/rules/dns-keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,8 @@ DNS query on the wire (snippet)::
``dns.query`` buffer::

mail.google.com

Multiple Buffer Matching
~~~~~~~~~~~~~~~~~~~~~~~~

``dns.query`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.
4 changes: 4 additions & 0 deletions doc/userguide/rules/file-keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Example::

filename:"secret";

``file.name`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did you list all these ? (Did you run a git grep ?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did you list all these ? (Did you run a git grep ?)

I looked at the commit history for the patch set, though now I realize I would have missed any later commits. I went back and looked again and did miss something. So I will have an additional commit to this PR for review.

I do have a question about:
src/detect-quic-cyu-hash.c: DetectBufferTypeSupportsMultiInstance(BUFFER_NAME);
src/detect-quic-cyu-string.c: DetectBufferTypeSupportsMultiInstance(BUFFER_NAME);

Should () contain the respective quic buffer names instead of BUFFER_NAME?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There must be a macro #define BUFFER_NAME a few lines up...


fileext
-------

Expand Down Expand Up @@ -47,6 +49,8 @@ Example::
Note: as libmagic versions differ between installations, the returned
information may also slightly change. See also #437.

``file.magic`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.

filestore
---------

Expand Down
5 changes: 5 additions & 0 deletions doc/userguide/rules/http-keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -839,3 +839,8 @@ Notes
pattern '<html' is absent from the first inspected chunk.

- ``file.data`` can also be used with SMTP

Multiple Buffer Matching
~~~~~~~~~~~~~~~~~~~~~~~~

``file.data`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.
1 change: 1 addition & 0 deletions doc/userguide/rules/http2-keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ Examples::

``http2.header_name`` can be used as ``fast_pattern``.

``http2.header_name`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.

Additional information
----------------------
Expand Down
2 changes: 2 additions & 0 deletions doc/userguide/rules/ike-keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ Examples::

ike.vendor:4a131c81070358455c5728f20e95452f;

``ike.vendor`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.


ike.key_exchange_payload
------------------------
Expand Down
1 change: 1 addition & 0 deletions doc/userguide/rules/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,4 @@ Suricata Rules
datasets
lua-detection
differences-from-snort
multi-buffer-matching
4 changes: 4 additions & 0 deletions doc/userguide/rules/kerberos-keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ Signature example::

``krb5_cname`` can be used as ``fast_pattern``.

``krb5.cname`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.

krb5_sname
----------

Expand All @@ -75,6 +77,8 @@ Signature example::

``krb5_sname`` can be used as ``fast_pattern``.

``krb5.sname`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.

krb5_err_code
-------------

Expand Down
4 changes: 4 additions & 0 deletions doc/userguide/rules/mqtt-keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,8 @@ Examples::

``mqtt.subscribe.topic`` is a 'sticky buffer' and can be used as ``fast_pattern``.

``mqtt.subscribe.topic`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.


mqtt.unsubscribe.topic
----------------------
Expand All @@ -249,6 +251,8 @@ Examples::

``mqtt.unsubscribe.topic`` is a 'sticky buffer' and can be used as ``fast_pattern``.

``mqtt.unsubscribe.topic`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.


Additional information
----------------------
Expand Down
92 changes: 92 additions & 0 deletions doc/userguide/rules/multi-buffer-matching.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
Multiple Buffer Matching
========================

Suricata 7 and newer now supports matching contents in multiple
buffers within the same transaction.

For example a single DNS transaction that has two queries in it:

query 1: example.net
query 2: something.com

Example rule:

.. container:: example-rule

`alert dns $HOME_NET any -> $EXTERNAL_NET any (msg:"DNS Multiple Question Example Rule"; dns.query; content:"example"; dns.query; content:".com"; classtype:misc-activity; sid:1; rev:1;)`

Within the single DNS query transaction, there are two queries
and Suricata will set up two instances of a dns.query buffer.

The first ``dns.query`` buffer will look for content:"example";

The second ``dns.query`` buffer will look for content:".com";

The example rule will alert on the example query since all the
content matches are satisfied for the rule.

For matching multiple headers in HTTP2 traffic a rule using the
new functionality would look like:

.. container:: example-rule

`alert http2 any any -> any any (msg:"HTTP2 Multiple Header Buffer Example"; flow:established,to_server; http.request_header; content:"method|3a 20|GET"; http.request_header; content:"authority|3a 20|example.com"; classtype:misc-activity; sid:1; rev:1;)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rule is missing closing `


With HTTP2 there are multiple headers seen in the same flow record.
We now have a way to write a rule in a more efficient way using the
multiple buffer capability.


**Note** Existing behavior when using sticky buffers still applies:

Example rule:

.. container:: example-rule

`alert dns $HOME_NET any -> $EXTERNAL_NET any (msg:"DNS Query Sticky Buffer Classic Example Rule"; dns.query; content:"example"; content:".net"; classtype:misc-activity; sid:1; rev:1;)`

The above rule will alert on a single dns query containing
"example.net" or "example.domain.net" since the rule content
matches are within a single ``dns.query`` buffer and all
content match requirements of the rule are met.


**Note:** This is new behavior, in versions of Suricata prior to
version 7 multiple statements of the same sticky buffer did not
Comment on lines +54 to +55
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
**Note:** This is new behavior, in versions of Suricata prior to
version 7 multiple statements of the same sticky buffer did not
**Note:** This is new behavior. In versions of Suricata prior to
version 7, multiple statements of the same sticky buffer did not

make a second instance of the buffer. For example:

dns.query; content:"example"; dns.query; content:".com";

would be equivalent to:

dns.query; content:"example"; content:".com";

Using our example from above, the first query is for example.net
which matches content:"example"; but does not match content:".com";

The second query is for something.com which would match on the
content:".com"; but not the content:"example";

So with the Suricata behavior prior to Suricata 7, the signature
would not fire in this case since both content conditions will
not be met.

Multiple buffer matching is currently enabled for use with the
following keywords:

* ``dns.query``
* ``file.data``
* ``file.magic``
* ``file.name``
* ``http.request_header``
* ``http.response_header``
* ``http2.header_name``
* ``ike.vendor``
* ``krb5_cname``
* ``krb5_sname``
* ``mqtt.subscribe.topic``
* ``mqtt.unsubscribe.topic``
* ``quic.cyu.hash``
* ``quic.cyu.string``
* ``tls.certs``
* ``tls.cert_subject``
4 changes: 4 additions & 0 deletions doc/userguide/rules/quic-keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Examples::
quic.cyu.hash; content:"7b3ceb1adc974ad360cfa634e8d0a730"; \
sid:1;)

``quic.cyu.hash`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.

quic.cyu.string
---------------

Expand All @@ -29,6 +31,8 @@ Examples::
quic.cyu.string; content:"46,PAD-SNI-VER-CCS-UAID-TCID-PDMD-SMHL-ICSL-NONP-MIDS-SCLS-CSCT-COPT-IRTT-CFCW-SFCW"; \
sid:2;)

``quic.cyu.string`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.

quic.version
------------

Expand Down
4 changes: 4 additions & 0 deletions doc/userguide/rules/tls-keywords.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ Examples::

``tls.cert_subject`` can be used as ``fast_pattern``.

``tls.cert_subject`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.

tls.subject
~~~~~~~~~~~

Expand Down Expand Up @@ -174,6 +176,8 @@ Example::

``tls.certs`` can be used as ``fast_pattern``.

``tls.certs`` supports multiple buffer matching, see :doc:`multi-buffer-matching`.

tls.version
-----------

Expand Down