Packet:parse_options() in nselib/packet.lua allows uncontrolled memory consumption (DoS)

[msuhanov]

Describe the bug
The bug is here:

nmap/nselib/packet.lua

Line 697 in b0293fe

l = self:u8(offset + opt_ptr + 1)

If l is 0, the function goes into the infinite loop, consuming memory until the OOM killer arrives.

To Reproduce

$ cat test.nse 
local stdnse = require('stdnse')
local shortport = require('shortport')
local packet = require('packet')

portrule = shortport.port_or_service({443}, {"https"}, "tcp", "open")

action = function(host, port)
    local pktr_data = stdnse.fromhex("45000044cc154000ee068872aaaaaaaabbbbbbbb01bbfbb9a06db07a41414142c012ffffb6fd0000020405b40402010103030101080a26e09a1c28541468000003001122")
    local pktr = packet.Packet:new(pktr_data, #pktr_data)
    return "good"
end

$ nmap -v --script=test.nse -p443 -d google.com
Starting Nmap 7.99SVN ( https://nmap.org ) at 2026-05-21 11:35 +0300
--------------- Timing report ---------------
  hostgroups: min 1, max 100000
  rtt-timeouts: init 1000, min 100, max 10000
  max-scan-delay: TCP 1000, UDP 1000, SCTP 1000
  parallelism: min 0, max 0
  max-retries: 10, host-timeout: 0
  min-rate: 0, max-rate: 0
---------------------------------------------
NSE: Using Lua 5.4.
NSE: Arguments from CLI: 
NSE: Loaded 1 scripts for scanning.
NSE: Script Pre-scanning.
NSE: Starting runlevel 1 (of 1) scan.
Initiating NSE at 11:35
Completed NSE at 11:35, 0.00s elapsed
mass_dns: Using DNS server 127.0.0.53
Initiating Parallel DNS resolution of 1 host. at 11:35
mass_dns: 0.00s 0/2 [#: 1, OK: 0, NX: 0, DR: 0, SF: 0, TR: 2]
Completed Parallel DNS resolution of 1 host. at 11:35, 0.00s elapsed
DNS resolution of 2 IPs took 0.00s. Mode: Async [#: 1, OK: 2, NX: 0, DR: 0, SF: 0, TR: 2, CN: 0]
Warning: Hostname google.com resolves to 10 IPs. Using 64.233.162.100.
Initiating Ping Scan at 11:35
Scanning google.com (64.233.162.100) [2 ports]
Completed Ping Scan at 11:35, 0.05s elapsed (1 total hosts)
Overall sending rates: 40.99 packets / s.
Initiating Parallel DNS resolution of 1 host. at 11:35
mass_dns: 0.03s 0/1 [#: 1, OK: 0, NX: 0, DR: 0, SF: 0, TR: 1]
Completed Parallel DNS resolution of 1 host. at 11:35, 0.03s elapsed
DNS resolution of 1 IPs took 0.03s. Mode: Async [#: 1, OK: 1, NX: 0, DR: 0, SF: 0, TR: 1, CN: 0]
Initiating Connect Scan at 11:35
Scanning google.com (64.233.162.100) [1 port]
Discovered open port 443/tcp on 64.233.162.100
Completed Connect Scan at 11:35, 0.05s elapsed (1 total ports)
Overall sending rates: 19.69 packets / s.
NSE: Script scanning 64.233.162.100.
NSE: Starting runlevel 1 (of 1) scan.
Initiating NSE at 11:35
NSE: Starting test against google.com (64.233.162.100:443).

# Gigabytes of memory are eaten here!

Expected behavior
Invalid (zero-length) trailing TCP options are either skipped or raise an exception.

Version info (please complete the following information):

• OS: Ubuntu 24.04.4 LTS (Noble Numbat), kernel: 6.17.0-29-generic
• Output of nmap --version:

Nmap version 7.99SVN ( https://nmap.org )
Platform: x86_64-unknown-linux-gnu
Compiled with: nmap-liblua-5.4.8 openssl-3.0.13 nmap-libssh2-1.11.1 libz-1.3 libpcre2-10.42 libpcap-1.10.4 nmap-libdnet-1.18.0 ipv6
Compiled without:
Available nsock engines: epoll poll select

Previous versions (e.g., 7.94) are affected too.

• Output of nmap --iflist: this is not relevant.

Additional context
Some real-world hosts return this garbage, crashing Nmap NSE scans.

A simple patch makes this bug non-fatal:

@@ -701,6 +701,9 @@ function Packet:parse_options(offset, length)
     end
     options[op].len  = l
     options[op].data = d
+    if l==0 then
+      break
+    end
     opt_ptr = opt_ptr + l
     op = op + 1
   end