[issue] Nuclei ( Freeze | Stop sending requests ) in long term execution while providing 44mb list

[osamahamad]

Describe the bug
While providing a big list of hosts I want to scan for specific bug on a tmux session ( Since it will took some time ) . Example:

nuclei -no-color -bs 1000 -t /nuclei-templates/misconfigurations/shell-history.yaml -l activedomains.txt -trace-log log-nuclei.txt | tee -a shell-history.out

Nuclei works fine but after some time ( after one day ) . I can see from the trace log file by checking with tail.

tail log-nuclei.txt 

That is hang on / freeze and stop sending requests on the rest of hosts that meant to check based on that specific template.

I leaved it and come after 24 hours and it seems remain the same ( By checking trace log, the last request remains the same after 24 hour which indicate that nuclei stop sending requests) .

By de-attaching tmux session that is meant to nuclei and checking with htop it seems that nuclei is already running and consume resources.

---

I tried to split my hosts file into list of files each one contains 1000 line of hosts ( which is validated as valid list of http/https URLs ) . and run it with for loop which will let nuclei visit each file and perform its scan.

split activedomains.txt -l 1000 splitted-active-domains/

for i in $(ls splitted-active-domains/); do nuclei -no-color -bs 1000 -t /nuclei-templates/misconfigurations/shell-history.yaml -l splitted-active-domains/$i -trace-log log-nuclei.txt | tee -a shell-history.out`

After some time. It also hangs by seeing trace log tail.

I tried the following on two different servers using Ubuntu 20.04 .
The problem remains nuclei still running but not sending requests. Therefore, My hosts in activedomains.txt is not fully scanned.

Nuclei version
v2.3.8

[geeknik]

I'd first recommend that you update to nuclei 2.4.2 and try again as there have been a lot of bug fixes between 2.3.8 and 2.4.2. 👍🏻

[osamahamad]

Hi,

I tested it again by

for i in $(ls splitted-active-domains/); do nuclei -no-color -bs 1000 -t /nuclei-templates/misconfigurations/shell-history.yaml -l splitted-active-domains/$i -trace-log log-nuclei.txt | tee -a shell-history.out`

using nuclei 2.4.2.

Issue remains the same.

[Mzack9999]

@osamahamad Thanks for reporting this issue. Were you able to identify if the tool hangs on any specific host of the list or sublist group? If you could join our discord server and reach out to pd-team with more details to reproduce the behavior, that would be helpful to nail down the root cause and fix it. Thanks!

[osamahamad]

Hi @Mzack9999 , I confirmed that it is not related to a specific host by checking with tail the trace log with different executions. The result seems all last scanned hosts are different.

I also tried to check if this problem related to tmux so I tried with screen and the result are the same ; hangs on a random host/subgroup from the list and stop sending requests.

Regarding discord server, I'm already in and I will see what I can do about it.

[Regala]

Can confirm similar behavior with lots of hosts.

[rewiaca]

Faced the same issue, consistently interupts on a big list on v.2.5.2
I suppose it happens when -bl and -rl specified more than a server is capable of. Any way to debug?

[minispooner2]

same here. no progress on this in 2 years? this is serious breakage, i can't use this product if this happens (but hey it's free :D). i can't ssh into my servers to check status anymore, presumedly because the scans are hanging and consuming all server resources. again, great scanner tool though. any updates on this issue?

[abd-4fg]

any update on this issue in 2023 ?

[minispooner2]

i troubleshooted and i think it's capping the CPU consumption. assigning more AWS vCPU's seems to be solving it