Interlace hangs with large amount of IPs in a file

[0xtavian]

Get 50,000 random IPs from https://onlinerandomtools.com/generate-random-ip and save to file named ips.txt
The following command hangs and never starts. CPU is maxed out. From what I can tell, this bug only impacts IPs in a list. For example:
interlace -tL ips.txt -c "echo _target_ " -threads 5

Attempting with hostnames instead of IPs works as expected and is successful, example interlace -tL subdomains.txt -c "echo _target_ " -threads 5 (download hostnames from a target with more than 50k subdomains from https://chaos.projectdiscovery.io/#/).
Attempting with CIDR expansion works as expected and is successful, example interlace -t 10.0.0.0/16 -c "echo _target_ " -threads 5

I've tried with Ubuntu 20.04 and Kali 2020.4.

[prodigysml]

Hey @0xtavian , this is a known issue. There is a cheeky workaround for it. Basically using a fake target (like -t a) and then passing a command list (-cL) with all the string replacements already done, will speed up the process significantly! Unfortunately, atm the string replacement is written really slow, which is why interlace starts off a bit slow. The command list workaround works well and I use it myself!

[prodigysml]

After some chit chat with @0xtavian, we think the issue may lie in the usage of the IP object (slows down everything). When we get around to fixing this, it's something we need to take a look at (#reminder)

[pry0cc]

So I did some quick n dirty forking and just removed the IP object piece altogether - ...and it works!

So maybe we just need an option to skip putting them into options? Or do we make it more performant?

pry0cc@64ef017

[0xtavian]

I can confirm this fixes the issue as well.

[prodigysml]

Awesome! I'll try to take a look and push a patch soon! Might be a while as some other things are on fire atm. Thanks for being patient everyone!

[pry0cc]

Any movement on this issue?

[prodigysml]

Hey @pry0cc, tad busy with some work stuff, but I am working on it in the background :) So far my analysis is that we will need to rewrite that section (IP related and all operations around that).

[prodigysml]

I can confirm the patch is working a LOT better than before! It is easily compatible with 50k IPs now. Waiting on the reviewer to approve.