Pull request #4022: build: generate and tag 3.1.71.0

Merge in SNORT/snort3 from ~PRBG/snort3:build_3.1.71.0 to master

Squashed commit of the following:

commit e1ebf3f63dd0c0c9891e913de83dbde00beca65a
Author: Priyanka Gurudev <prbg@cisco.com>
Date:   Mon Sep 25 07:45:30 2023 -0400

    build: generate and tag 3.1.71.0

CMakeLists.txt

@@ -3,7 +3,7 @@ project (snort CXX C)
 
 set (VERSION_MAJOR 3)
 set (VERSION_MINOR 1)
-set (VERSION_PATCH 70)
+set (VERSION_PATCH 71)
 set (VERSION_SUBLEVEL 0)
 set (VERSION "${VERSION_MAJOR}.${VERSION_MINOR}.${VERSION_PATCH}.${VERSION_SUBLEVEL}")
 

ChangeLog.md

@@ -1,4 +1,23 @@
-2023-08-10: 3.1.70.0
+2023-09-25: 3.1.71.0
+
+* appid, http_inspect, http2_inspect: create appid session if not present in decrypt event handler, add message section as part of StreamFlowIntf for httpx
+* codecs: Add IPv6 Reserved Address to GID:116 Rules
+* detection: avoid multiple fixups of duplicated trees
+* detection: fix of default ips policy switching
+* flow: allow reinspection for blocked icmp flows after reload
+* flow: generate flow setup and established events for ha flows
+* host_cache: cppcheck fix
+* http2_inspect: fix http2 frame length for logging
+* main: fix signals handling after failed started instances
+* main: reset_stats argument type improvement
+* parser: add file_id rule syntax evaluation
+* smtp: add alert for mixed LF and CRLF
+* smtp: process DATA\n (no \r)
+* stream: extend list of arguments for extra data logging
+* stream_tcp: ensure all data segments after a zero window are blocked when NAP is inline
+* stream_tcp: examine whether a segment plugs a hole before blocking due to exceeding queue_limit
+
+2023-09-10: 3.1.70.0
 
 * appid: makes regex error more of a warning
 * detection: fix assert expression

doc/reference/snort_reference.text

@@ -8,7 +8,7 @@ Snort 3 Reference Manual
 The Snort Team
 
 Revision History
-Revision 3.1.70.0 2023-09-10 14:39:41 EDT TST
+Revision 3.1.71.0 2023-09-25 07:40:20 EDT TST
 
 ---------------------------------------------------------------------
 
@@ -1726,7 +1726,9 @@ Commands:
     default policy
   * snort.dump_stats(): show summary statistics
   * snort.dump_heap_stats(): show heap statistics
-  * snort.reset_stats(): clear summary statistics
+  * snort.reset_stats(type): clear summary statistics. Type can be:
+    daq|module|appid|file_id|snort|ha|all. reset_stats() without a
+    parameter clears all statistics.
   * snort.rotate_stats(): roll perfmonitor log files
   * snort.reload_config(filename): load new configuration
   * snort.reload_policy(filename): reload part or all of the default
@@ -2252,6 +2254,8 @@ Rules:
   * 116:461 (ipv6) IPv6 routing type 0 extension header
   * 116:475 (ipv6) IPv6 mobility header includes an invalid value for
     the payload protocol field
+  * 116:476 (ipv6) IPv6 packet from reserved source address
+  * 116:477 (ipv6) IPv6 packet to reserved dest address
 
 
 3.18. llc
@@ -5418,6 +5422,7 @@ Rules:
   * 124:15 (smtp) attempted authentication command buffer overflow
   * 124:16 (smtp) file decompression failed
   * 124:17 (smtp) STARTTLS command injection attempt
+  * 124:18 (smtp) mix of LF and CRLF as end of line
 
 Peg counts:
 
@@ -12869,12 +12874,12 @@ session.
 The TCP packet is invalid because it doesn’t have a SYN, ACK, or RST
 flag set.
 
-116:424 (eth) truncated ethernet header
+116:424 (pbb) truncated ethernet header
 
 The packet length is less than the minimum ethernet header size (14
 bytes)
 
-116:424 (eth) truncated ethernet header
+116:424 (pbb) truncated ethernet header
 
 A truncated ethernet header was detected.
 
@@ -13105,6 +13110,14 @@ payload protocol field
 The IPv6 mobility header includes an invalid value for the payload
 protocol field.
 
+116:476 (ipv6) IPv6 packet from reserved source address
+
+The IPv6 packet has a reserved source address.
+
+116:477 (ipv6) IPv6 packet to reserved dest address
+
+The IPv6 packet has a reserved destination address.
+
 119:1 (http_inspect) URI has percent-encoding of an unreserved
 character
 
@@ -14258,6 +14271,10 @@ File decompression failed.
 
 SMTP STARTTLS command injection attempt.
 
+124:18 (smtp) mix of LF and CRLF as end of line
+
+SMTP traffic has a mix of LF and CRLF as end of line
+
 125:1 (ftp_server) TELNET cmd on FTP command channel
 
 TELNET command is detected on FTP control channel.
@@ -15606,7 +15623,9 @@ alert is raised by the enhanced JavaScript normalizer.
     default policy
   * snort.dump_stats(): show summary statistics
   * snort.dump_heap_stats(): show heap statistics
-  * snort.reset_stats(): clear summary statistics
+  * snort.reset_stats(type): clear summary statistics. Type can be:
+    daq|module|appid|file_id|snort|ha|all. reset_stats() without a
+    parameter clears all statistics.
   * snort.rotate_stats(): roll perfmonitor log files
   * snort.reload_config(filename): load new configuration
   * snort.reload_policy(filename): reload part or all of the default

doc/upgrade/snort_upgrade.text

@@ -8,7 +8,7 @@ Snort 3 Upgrade Manual
 The Snort Team
 
 Revision History
-Revision 3.1.70.0 2023-09-10 14:40:46 EDT TST
+Revision 3.1.71.0 2023-09-25 07:41:06 EDT TST
 
 ---------------------------------------------------------------------
 

doc/user/snort_user.text

@@ -8,7 +8,7 @@ Snort 3 User Manual
 The Snort Team
 
 Revision History
-Revision 3.1.70.0 2023-09-10 14:40:02 EDT TST
+Revision 3.1.71.0 2023-09-25 07:40:35 EDT TST
 
 ---------------------------------------------------------------------
 
@@ -963,11 +963,11 @@ Fast patterns are content strings that have the fast_pattern option
 or which have been selected by Snort automatically to be used as a
 fast pattern. Snort will by default choose the longest pattern in the
 rule since that is likely to be most unique. That is not always the
-case so add fast_pattern to the appropriate content option for best
-performance. The ideal fast pattern is one which, if found, is very
-likely to result in a rule match. Fast patterns that match frequently
-for unrelated traffic will cause Snort to work hard with little to
-show for it.
+case so add fast_pattern to the appropriate content or regex option
+for best performance. The ideal fast pattern is one which, if found,
+is very likely to result in a rule match. Fast patterns that match
+frequently for unrelated traffic will cause Snort to work hard with
+little to show for it.
 
 Certain contents are not eligible to be used as fast patterns.
 Specifically, if a content is negated, then if it is also relative to
@@ -3330,26 +3330,30 @@ There are 3 steps to enable file processing:
 5.7.3. Pre-packaged File Magic Rules
 
 A set of file magic rules is packaged with Snort. They can be located
-at "lua/file_magic.lua". To use this feature, it is recommended that
-these pre-packaged rules are used; doing so requires that you include
-the file in your Snort configuration as such (already in snort.lua):
+at "lua/file_magic.rules". To use this feature, it is recommended
+that these pre-packaged rules are used; doing so requires that you
+include the file in your Snort configuration as such (already in
+snort.lua):
 
-dofile('magic.lua')
+file_id = { rules_file = 'file_magic.rules' }
 
 Example:
 
-{ type = "GIF", id = 62, category = "Graphics", rev = 1,
-  magic = { { content = "| 47 49 46 38 37 61 |",offset = 0 } } },
+file_id (msg:"GIF"; file_meta:type GIF, id 62, category "Graphics";
+  file_data; content:"| 47 49 46 38 37 61 |", depth 6, offset 0; gid:4; sid:56; rev:1;)
 
-{ type = "GIF", id = 63, category = "Graphics", rev = 1,
-  magic = { { content = "| 47 49 46 38 39 61 |",offset = 0 } } },
+file_id (msg:"GIF"; file_meta:type GIF, id 63, category 'Graphics";
+  file_data; content:"| 47 49 46 38 39 61 |", depth 6, offset 0; gid:4; sid:57; rev:1;)
 
 The previous two rules define GIF format, because two file magics are
 different. File magics are specified by content and offset, which
 look at content at particular file offset to identify the file type.
 In this case, two magics look at the beginning of the file. You can
 use character if it is printable or hex value in between "|".
 
+Note that file_meta and a fast-pattern option (content, regex) are
+required for each file_id rule.
+
 5.7.4. File Policy
 
 You can enabled file type, file signature, or file capture by