nmap crashes by exceeding FD_SET of 1024 even without high --min-parallelism/--max-parallelism

[msecrfe]

Describe the bug
nmap -sT crashes because it attempts to create FD_SET fd 1024 above the upper limit of FD_SETSIZE (1024). While this can happen when forcing high parallelism (by setting --min-parallelism or --max-parallelism very high, i.e., close to 1000), this also happens (though rarely) for long target lists without setting parallelism parameters.

To Reproduce
nmap was launched as follows for a long target list of about 20k targets, each of them identified by FQDN:
nmap --privileged -T4 --host-timeout '10m' --script-timeout '1m' --max-retries '3' --open -v --reason --dns-servers 8.8.8.8,8.8.4.4,1.1.1.1 -sTUV -p 'T:20,21,22,23,25,53,67,68,69,80,81,82,88,110,111,123,135,137,138,139,143,161,162,179,222,389,443,444,445,464,465,500,502,513,514,515,530,543,544,546,547,554,587,593,631,636,873,902,903,989,990,993,995,1080,1098,1099,1433,1434,1494,1521,1719,1720,1723,1883,1900,2049,2222,2375,2376,2377,2483,2484,2598,3020,3128,3268,3269,3306,3389,3478,4433,4434,4443,5000,5004,5005,5037,5060,5061,5222,5223,5269,5349,5432,5500,5555,5800,5900,5901,5902,5984,5985,5986,6000,6379,6443,6514,8000,8010,8080,8081,8090,8091,8092,8443,8883,9000,9001,9050,9090,9389,10443,27017,49152,49153,49154,U:53,67,68,69,88,111,123,135,137,138,139,161,162,389,464,500,514,530,546,547,554,631,1194,1434,1494,1701,1719,1720,1900,2049,2598,3391,3478,4500,5004,5005,5060,5349,5351,5353,5355,49152,49153,49154' --version-intensity '1' --script 'default and safe' -Pn -iL /tmp/20k_targets.txt -oA /tmp/nmap_out

The crash happened with about 17k targets being done, which took about 6 days:
Attempt to FD_SET fd 1024, which is not less than FD_SETSIZE (1024). Aborted (core dumped)

Expected behavior
nmap should not crash while using -sT for exceeding FD_SETSIZE when not forcing parallelism close to the limit. Automatic timing template parameters (in this case T4) should ensure to always stay below FD_SETSIZE of 1024.

Version info (please complete the following information):

• OS: Ubuntu 24.04 LTS with kernel 6.11.0
• Output of nmap --version:

Nmap version 7.94SVN ( https://nmap.org )
Platform: x86_64-pc-linux-gnu
Compiled with: liblua-5.4.6 openssl-3.0.13 libssh2-1.11.0 libz-1.3 libpcre2-10.42 libpcap-1.10.4 nmap-libdnet-1.12 ipv6
Compiled without:
Available nsock engines: epoll poll select

• Output of nmap --iflist:

$ nmap --iflist
Starting Nmap 7.94SVN ( https://nmap.org ) at 2025-05-07 12:19 UTC
************************INTERFACES************************
DEV        (SHORT)      IP/MASK                     TYPE     UP MTU   MAC
lo         (lo)         127.0.0.1/8                 loopback up 65536
lo         (lo)         ::1/128                     loopback up 65536
eth0       (eth0)       172.17.0.4/24               ethernet up 1500  00:22:48:F2:BC:95
eth0       (eth0)       fe80::222:48ff:fef2:bc95/64 ethernet up 1500  00:22:48:F2:BC:95
enP21961s1 (enP21961s1) (none)/0                    ethernet up 1500  00:22:48:F2:BC:95
enP21961s1 (enP21961s1) fe80::222:48ff:fef2:bc95/64 ethernet up 1500  00:22:48:F2:BC:95

**************************ROUTES**************************
DST/MASK                     DEV        METRIC GATEWAY
168.63.129.16/32             eth0       100    172.17.0.1
169.254.169.254/32           eth0       100    172.17.0.1
172.17.0.1/32                eth0       100
172.17.0.0/24                eth0       100
0.0.0.0/0                    eth0       100    172.17.0.1
::1/128                      lo         0
fe80::222:48ff:fef2:bc95/128 enP21961s1 0
fe80::222:48ff:fef2:bc95/128 eth0       0
fe80::/64                    eth0       256
fe80::/64                    enP21961s1 256
ff00::/8                     eth0       256
ff00::/8                     enP21961s1 256

Additional context
None

[dmiller-nmap]

This is a very strange issue, since the TCP Connect scan (scan_engine_connect.cc) specifically checks for socket values greater than FD_SETSIZE. Nsock does not, and perhaps should, but that would only affect the select engine, and Nmap uses that only as a last resort. On your system, it would use the epoll engine instead for service scan, NSE, and DNS. Do you have information about what stage of the scan was running when the crash happened?

[msecrfe]

Unfortunately I neither have the terminal output nor the core dump anymore. Maybe my assumption that it was due to -sT is wrong and it may indeed have been during another scan phase.
I have rerun the scan with a manual limit on parallelization in the meantime, with --max-parallelism of 900, and the crash did not occur.

Edit: ... did not occur so far. Please see the two comments below, it indeed happened again.

[msecrfe]

I repeated the scan and was able to reproduce the crash. In fact, it crashed with --max-parallelism '900' - which was a bit unexpected. Had hoped that a distance of 100 (between 900 and 1000) would be enough to prevent the crash.

It seems the crash happens during UDP scanning, not during TCP scanning:

Failed to resolve "<SNIP>".
Failed to resolve "<SNIP>".
Initiating Parallel DNS resolution of 2 hosts. at 14:54
Completed Parallel DNS resolution of 2 hosts. at 14:54, 0.23s elapsed
Initiating System DNS resolution of 1 host. at 14:54
Completed System DNS resolution of 1 host. at 14:54, 0.00s elapsed
Initiating UDP Scan at 14:54
Scanning 9 hosts [44 ports/host]
Attempt to FD_SET fd 1024, which is not less than FD_SETSIZE (1024). Try using a lower parallelism.
./run_tests.sh: Line 100: 689775 Aborted
             nmap --privileged -T4 -Pn --max-parallelism '900' --host-timeout '10m' --script-timeout '1m' --max-retries '1' --open -v --reason --dns-servers 8.8.8.8,8.8.4.4,1.1.1.1 -sTUV -p 'T:20,21,22,23,25,53,67,68,69,80,81,82,88,110,111,123,135,137,138,139,143,161,162,179,222,389,443,444,445,464,465,500,502,513,514,515,530,543,544,546,547,554,587,593,631,636,873,902,903,989,990,993,995,1080,1098,1099,1433,1434,1494,1521,1719,1720,1723,1883,1900,2049,2222,2375,2376,2377,2483,2484,2598,3020,3128,3268,3269,3306,3389,3478,4433,4434,4443,5000,5004,5005,5037,5060,5061,5222,5223,5269,5349,5432,5500,5555,5800,5900,5901,5902,5984,5985,5986,6000,6379,6443,6514,8000,8010,8080,8081,8090,8091,8092,8443,8883,9000,9001,9050,9090,9389,10443,27017,49152,49153,49154,U:53,67,68,69,88,111,123,135,137,138,139,161,162,389,464,500,514,530,546,547,554,631,1194,1434,1494,1701,1719,1720,1900,2049,2598,3391,3478,4500,5004,5005,5060,5349,5351,5353,5355,49152,49153,49154' --version-light --script 'default and safe' -iL ~/visibility_tests/input/fqdns.txt -oA ~/visibility_tests/result/nmap_out

At this point, nmap had been running for about a week:

Starting nmap at 2025-06-04T23:26:18+02:00
Finished nmap at 2025-06-11T14:54:11+02:00

The crash happened at input line 15329 of 17352 input lines - so given the number of targets and the runtime of almost a week, this was very long into the scan.

[msecrfe]

I was running the same scan (same parameters, same input) from a second machine located in Azure. (The above machine from my last comment is located on-prem.)

It also crashed, also seemingly during UDP scanning:

Initiating Parallel DNS resolution of 6 hosts. at 22:52
Completed Parallel DNS resolution of 6 hosts. at 22:52, 0.22s elapsed
Initiating System DNS resolution of 1 host. at 22:52
Completed System DNS resolution of 1 host. at 22:52, 0.57s elapsed
Initiating UDP Scan at 22:52
Scanning 18 hosts [44 ports/host]
Attempt to FD_SET fd 1024, which is not less than FD_SETSIZE (1024). Try using a lower parallelism.
./run_tests.sh: line 59: 696269 Aborted                 (core dumped)
nmap --privileged -T4 -Pn --max-parallelism '900' --host-timeout '10m' --script-timeout '1m' --max-retries '1' --open -v --reason --dns-servers 8.8.8.8,8.8.4.4,1.1.1.1 -sTUV -p 'T:20,21,22,23,25,53,67,68,69,80,81,82,88,110,111,123,135,137,138,139,143,161,162,179,222,389,443,444,445,464,465,500,502,513,514,515,530,543,544,546,547,554,587,593,631,636,873,902,903,989,990,993,995,1080,1098,1099,1433,1434,1494,1521,1719,1720,1723,1883,1900,2049,2222,2375,2376,2377,2483,2484,2598,3020,3128,3268,3269,3306,3389,3478,4433,4434,4443,5000,5004,5005,5037,5060,5061,5222,5223,5269,5349,5432,5500,5555,5800,5900,5901,5902,5984,5985,5986,6000,6379,6443,6514,8000,8010,8080,8081,8090,8091,8092,8443,8883,9000,9001,9050,9090,9389,10443,27017,49152,49153,49154,U:53,67,68,69,88,111,123,135,137,138,139,161,162,389,464,500,514,530,546,547,554,631,1194,1434,1494,1701,1719,1720,1900,2049,2598,3391,3478,4500,5004,5005,5060,5349,5351,5353,5355,49152,49153,49154' --version-light --script 'default and safe' -iL ~/visibility_tests/input/fqdns.txt -oA ~/visibility_tests/result/nmap_out

The duration of the run was pretty similar at a little more than 6 days:

Starting nmap at 2025-06-05T13:13:26+00:00
Finished nmap at 2025-06-11T22:52:34+00:00

The last input line processed before the crash was line 10953 of 17352.

The core dump is 380 MB big:

$ ls -la /var/crash/
total 371148
drwxrwsrwt  2 root root      4096 Jun 11 22:52 .
drwxr-xr-x 13 root root      4096 Apr  9 12:53 ..
-rw-r-----  1 root root 380041277 Jun 11 22:52 _usr_bin_nmap.1000.crash

[dmiller-nmap]

Okay, that's very helpful. I had missed that we use select() (and therefore checked_fd_set() which is the source of the error message) in pcap_select(), which is used in a lot of Nmap parts. I will look into what might be causing such a high file descriptor to be used. It's possible we're leaking pcap handles, so the bug only shows up after we've made 1000+ of them, but I can't say for sure.

[dmiller-nmap]

So it turns out we were leaking a socket for every run of TCP Connect scan (one per hostgroup). This would never crash the connect scan itself, since when we fail to get a socket ID less than FD_SETSIZE, we slow down and wait until an existing socket is closed. However, it did mean that eventually there were no low socket IDs left when UDP scan ran, which does not have such a check since it expects to only open one descriptor at a time. That's where the crash occurred. If you had not had -sU, then Connect scan would simply throttle down by one socket each hostgroup until it was limited to one socket at a time, the same as though --max-parallelism=1 had been used. The upshot of this is that now your Connect scans ought to finish faster.