Generate docs

For speculative service changes: zeek/zeek#565

script-reference/autogenerated-script-index.rst

@@ -477,4 +477,5 @@
    policy/frameworks/control/controller.zeek </scripts/policy/frameworks/control/controller.zeek>
    policy/frameworks/files/extract-all-files.zeek </scripts/policy/frameworks/files/extract-all-files.zeek>
    policy/misc/dump-events.zeek </scripts/policy/misc/dump-events.zeek>
+   policy/protocols/conn/speculative-service.zeek </scripts/policy/protocols/conn/speculative-service.zeek>
    zeekygen/example.zeek </scripts/zeekygen/example.zeek>

scripts/base/bif/event.bif.zeek.rst

@@ -76,6 +76,8 @@ Events
 :zeek:id:`profiling_update`: :zeek:type:`event`                                                        Generated each time Zeek's internal profiling log is updated.
 :zeek:id:`protocol_confirmation`: :zeek:type:`event`                                                   Generated when a protocol analyzer confirms that a connection is indeed
                                                                                                        using that protocol.
+:zeek:id:`protocol_late_match`: :zeek:type:`event`                                                     Generated if a DPD signature matched but the DPD buffer is already exhausted
+                                                                                                       and thus the analyzer could not be attached.
 :zeek:id:`protocol_violation`: :zeek:type:`event`                                                      Generated when a protocol analyzer determines that a connection it is parsing
                                                                                                        is not conforming to the protocol it expects.
 :zeek:id:`raw_packet`: :zeek:type:`event`                                                              Generated for every packet Zeek sees that have a valid link-layer header.
@@ -845,6 +847,25 @@ Events
       of :zeek:type:`Conn::Info`: once confirmed, the protocol will be listed
       there (and thus in ``conn.log``).
 
+.. zeek:id:: protocol_late_match
+
+   :Type: :zeek:type:`event` (c: :zeek:type:`connection`, atype: :zeek:type:`Analyzer::Tag`)
+
+   Generated if a DPD signature matched but the DPD buffer is already exhausted
+   and thus the analyzer could not be attached. While this does not confirm
+   that a protocol is actually used, it allows to retain that information.
+
+
+   :c: The connection.
+
+
+   :atype: The type of the analyzer confirming that its protocol is in
+          use. The value is one of the ``Analyzer::ANALYZER_*`` constants. For example,
+          ``Analyzer::ANALYZER_HTTP`` means the HTTP analyzer determined that it's indeed
+          parsing an HTTP connection.
+
+   .. bro:see:: dpd_buffer_size
+
 .. zeek:id:: protocol_violation
 
    :Type: :zeek:type:`event` (c: :zeek:type:`connection`, atype: :zeek:type:`Analyzer::Tag`, aid: :zeek:type:`count`, reason: :zeek:type:`string`)

scripts/base/init-bare.zeek.rst

@@ -119,6 +119,7 @@ Redefinable Options
 :zeek:id:`dpd_buffer_size`: :zeek:type:`count` :zeek:attr:`&redef`                         Size of per-connection buffer used for dynamic protocol detection.
 :zeek:id:`dpd_ignore_ports`: :zeek:type:`bool` :zeek:attr:`&redef`                         If true, don't consider any ports for deciding which protocol analyzer to
                                                                                            use.
+:zeek:id:`dpd_late_match_stop`: :zeek:type:`bool` :zeek:attr:`&redef`                      If true, stops signature matching after a late match.
 :zeek:id:`dpd_match_only_beginning`: :zeek:type:`bool` :zeek:attr:`&redef`                 If true, stops signature matching if :zeek:see:`dpd_buffer_size` has been
                                                                                            reached.
 :zeek:id:`dpd_reassemble_first_packets`: :zeek:type:`bool` :zeek:attr:`&redef`             Reassemble the beginning of all TCP connections before doing
@@ -1157,11 +1158,40 @@ Redefinable Options
    .. zeek:see:: dpd_reassemble_first_packets dpd_buffer_size
       dpd_match_only_beginning
 
+.. zeek:id:: dpd_late_match_stop
+
+   :Type: :zeek:type:`bool`
+   :Attributes: :zeek:attr:`&redef`
+   :Default: ``F``
+   :Redefinition: from :doc:`/scripts/policy/protocols/conn/speculative-service.zeek`
+
+      ``=``::
+
+         T
+
+
+   If true, stops signature matching after a late match. A late match may occur
+   in case the DPD buffer is exhausted but a protocol signature matched. To
+   allow late matching, :zeek:see:`dpd_match_only_beginning` must be disabled.
+
+   .. zeek:see:: dpd_reassemble_first_packets dpd_buffer_size
+      dpd_match_only_beginning
+
+   .. note:: Despite the name, this option stops *all* signature matching, not
+      only signatures used for dynamic protocol detection but is triggered by
+      DPD signatures only.
+
 .. zeek:id:: dpd_match_only_beginning
 
    :Type: :zeek:type:`bool`
    :Attributes: :zeek:attr:`&redef`
    :Default: ``T``
+   :Redefinition: from :doc:`/scripts/policy/protocols/conn/speculative-service.zeek`
+
+      ``=``::
+
+         F
+
 
    If true, stops signature matching if :zeek:see:`dpd_buffer_size` has been
    reached.
@@ -7081,6 +7111,10 @@ Types
          (present if :doc:`/scripts/policy/protocols/mqtt/main.zeek` is loaded)
 
 
+      speculative_service: :zeek:type:`set` [:zeek:type:`string`] :zeek:attr:`&default` = ``{  }`` :zeek:attr:`&optional`
+         (present if :doc:`/scripts/policy/protocols/conn/speculative-service.zeek` is loaded)
+
+
    A connection. This is Zeek's basic connection type describing IP- and
    transport-layer information about the conversation. Note that Zeek uses a
    liberal interpretation of "connection" and associates instances of this type

scripts/base/protocols/conn/main.zeek.rst

@@ -208,6 +208,13 @@ Types
 
          The inner VLAN for this connection, if applicable.
 
+      speculative_service: :zeek:type:`string` :zeek:attr:`&log` :zeek:attr:`&optional`
+         (present if :doc:`/scripts/policy/protocols/conn/speculative-service.zeek` is loaded)
+
+         Protocol that was determined by a matching signature after the beginning
+         of a connection. In this situation no analyzer can be attached and hence
+         the data cannot be analyzed nor the protocol can be confirmed.
+
    The record type which contains column fields of the connection log.
 
 Events

scripts/policy/protocols/conn/speculative-service.zeek.rst

@@ -0,0 +1,27 @@
+:tocdepth: 3
+
+policy/protocols/conn/speculative-service.zeek
+==============================================
+.. zeek:namespace:: Conn
+
+This script adds information about matched DPD signatures to the connection
+log.
+
+:Namespace: Conn
+:Imports: :doc:`base/protocols/conn </scripts/base/protocols/conn/index>`
+
+Summary
+~~~~~~~
+Redefinitions
+#############
+========================================================================== =
+:zeek:type:`Conn::Info`: :zeek:type:`record`                               
+:zeek:type:`connection`: :zeek:type:`record`                               
+:zeek:id:`dpd_late_match_stop`: :zeek:type:`bool` :zeek:attr:`&redef`      
+:zeek:id:`dpd_match_only_beginning`: :zeek:type:`bool` :zeek:attr:`&redef` 
+========================================================================== =
+
+
+Detailed Interface
+~~~~~~~~~~~~~~~~~~
+

scripts/zeekygen/__load__.zeek.rst

@@ -4,7 +4,7 @@ zeekygen/__load__.zeek
 ======================
 
 
-:Imports: :doc:`policy/frameworks/control/controllee.zeek </scripts/policy/frameworks/control/controllee.zeek>`, :doc:`policy/frameworks/control/controller.zeek </scripts/policy/frameworks/control/controller.zeek>`, :doc:`policy/frameworks/files/extract-all-files.zeek </scripts/policy/frameworks/files/extract-all-files.zeek>`, :doc:`policy/misc/dump-events.zeek </scripts/policy/misc/dump-events.zeek>`, :doc:`policy/protocols/ssl/notary.zeek </scripts/policy/protocols/ssl/notary.zeek>`, :doc:`test-all-policy.zeek </scripts/test-all-policy.zeek>`, :doc:`zeekygen/example.zeek </scripts/zeekygen/example.zeek>`
+:Imports: :doc:`policy/frameworks/control/controllee.zeek </scripts/policy/frameworks/control/controllee.zeek>`, :doc:`policy/frameworks/control/controller.zeek </scripts/policy/frameworks/control/controller.zeek>`, :doc:`policy/frameworks/files/extract-all-files.zeek </scripts/policy/frameworks/files/extract-all-files.zeek>`, :doc:`policy/misc/dump-events.zeek </scripts/policy/misc/dump-events.zeek>`, :doc:`policy/protocols/conn/speculative-service.zeek </scripts/policy/protocols/conn/speculative-service.zeek>`, :doc:`policy/protocols/ssl/notary.zeek </scripts/policy/protocols/ssl/notary.zeek>`, :doc:`test-all-policy.zeek </scripts/test-all-policy.zeek>`, :doc:`zeekygen/example.zeek </scripts/zeekygen/example.zeek>`
 
 Summary
 ~~~~~~~