panic: sync: WaitGroup is reused before previous Wait has returned

[khalilwtf00]

Hello team hope you are good,
I have an issue where nuclei just stop working with this message
Killed

I tried to locate the location and the source I didn't find anything,
Keep in mind please that this happen only in the latest version
Best wishes

[ehsandeep]

@khalilwtf00 yesterday we released new version of nuclei that includes a fix for this issue, can you confirm if you were the latest version of nuclei? i.e v2.8.4?

[khalilwtf00]

@khalilwtf00 yesterday we released new version of nuclei that includes a fix for this issue, can you confirm if you were the latest version of nuclei? i.e v2.8.4?

@ehsandeep Thanks for the reply,
Yes latest version v2.8.4 is used.

[ehsandeep]

@khalilwtf00 please share all the details then, cmd/system config/ target/ template number and anything that can be used to recreate the issue.

[geeknik]

I can confirm this behavior is happening with 2.8.4, but in my case, this only happens when using the fuzzing-templates. My work around is to set -c 1 to limit the RAM usage. I am using go1.18.3 on Ubuntu 20.04.5 LTS.

Workflow:
echo "domain.tld" | subfinder -all | httpx | katana -f qurl | nuclei -t fuzzing-templates -stats

My list of domains is only 8,621. However, there is a huge spike in RAM usage 2-3 minutes after starting which results in the Killed message thanks to the Linux kernel unless the machine has 8+ GB of RAM.

[ehsandeep]

@geeknik @khalilwtf00 using -ss host-spray should resolve the memory issue, reference - https://nuclei.projectdiscovery.io/nuclei/get-started/#which-scan-strategy-to-use

we have follow up issue for doing this automatic - #3076

[khalilwtf00]

@ehsandeep Thanks for the reply,
I'm using a VPS, I will try again and I will share the result next !
Best Wishes

[khalilwtf00]

I got this error after 15 mins of working

[tarunKoyalwar]

@khalilwtf00 , this only seems to happen with host-spray can you verify if it also happens with template-spray strategy and -c 10 -bs 200

[khalilwtf00]

@tarunKoyalwar Thanks for help,
in the same way I get killed again

[marcelo321]

Same happening to me. When using 1gb linodes. I was doing ss host-spray -concurrency 150 -bulk-size 10 as per recommended in the wiki, and lowering it more to see if that resolves the issue.

Does lowering concurrency fix the issue? or what can we try? @khalilwtf00

[khalilwtf00]

@marcelo321 I tried with 16Gb RAM, with -rl 10 and still crash,

[tarunKoyalwar]

@khalilwtf00 , It looks like a memory issue , we are currently in process of fixing this soon for the time being you can also use previous version from releases

[tarunKoyalwar]

@khalilwtf00 , @geeknik the waitgroup issue with host-spray should be fixed with attached PR and v2.8.8 .

about memory killed issue we are currently working on something #3203
can you share if you are facing killed issue with below nuclei

go install -v github.com/projectdiscovery/nuclei/v2/cmd/nuclei@85b45e8

this version of nuclei is linearly scaled so feel free to try it with -c 5000 or even more

Note
-rl i.e ratelimit flag limits max concurrency per sec so if we use -rl 10 then nuclei sends 10 req/s (default is 150 req/s) . so either keep -rl default or increase it depending on target

@marcelo321 I tried with 16Gb RAM, with -rl 10 and still crash,

[khalilwtf00]

@tarunKoyalwar Hello,
No killed issue this time with the same target list.

[tarunKoyalwar]

@khalilwtf00

• just to confirm you installed nuclei using below command

go install -v github.com/projectdiscovery/nuclei/v2/cmd/nuclei@85b45e8

also how many targets are in target list ??

[khalilwtf00]

Yes.
And the targets list was more than 6500

[tarunKoyalwar]

@khalilwtf00 , can you share if you are facing killed issue with attached PR . you can install using

go install -v github.com/projectdiscovery/nuclei/v2/cmd/nuclei@c2a4fd1

[khalilwtf00]

@tarunKoyalwar Yes, the same targets list, with killed issue after 11 hours of running

[tarunKoyalwar]

@khalilwtf00 , thanks for sharing we will look into . In meanwhile since there are no issues with linear:concurrency PR #3140 (comment) . It is experimental but stable so feel free to use it for mass scanning

[rxerium]

Hey team, I'm receiving these wait group errors in v3.2.6. Should I create a new issue?

[tarunKoyalwar]

@rxerium , yes that seems to be a new issue