Nmap-friendly output (or output destined to be nmap input)

[Viss]

Hi Rob!
I'd love for masscan to output text in a way I can feed it directly to nmap - by that I mean as though one were going to now scan the findings using nmap, since the heavy lifting of discovering ports is now done...

10.0.0.1 -p 80,443,8080

etc..
so that one could literally pipe the output of masscan directly to the input of nmap - it would greatly improve workflow fluidity! :D

Thanks in advance!
-Viss

[Crypto-Cat]

Can't believe this has been an open issue for 6 years.. It's such a critical feature

[Viss]

I've also asked multiple people at the nmap project about this and their complaint is that for the -iL functionatlity to take something like
ip:port1,port2,port3
would take a full rewrite of their core code. so they've been sat on it as well

having the ability to do such a thing would literally revolutionize portscanning, and make scanning in general way way way less hamfisted.

[Crypto-Cat]

It shouldn't be that complicated for a basic workaround.. If Masscan was able to produce outputs such as:

10.0.0.1 -p U:1337,T:23,80,443
10.0.0.2 -p U:31337,T:22,8080,9000

then I could just output the result to a text file and run:

for item in $(cat output.txt); do nmap -sV -sA -sU $item; done

[Crypto-Cat]

I put together a script to do what I described above, hopefully it will at least provide some automation until Masscan/Nmap implement the functionality :)

gen_nmap.txt

[iknowjason]

It is interesting that you can't supply nmap with targets in the form of 'ip:port1,port2,port3' and you can only supply targets as IP addresses. I've created three scripts that automate masscan and nmap together. Based on how automated you are trying to do it, you can use one of the three. Thanks @Crypto-Cat for the idea, i've modeled one of them after yours. Here they are:

https://gist.github.com/iknowjason/f14f0cce2e5fb0c5171f0bbe77521f41

https://gist.github.com/iknowjason/4dc9c99468503341569ed25deddb51c8

https://gist.github.com/iknowjason/00dfd887ff3c700811d0869ac43c6abf

[tzmcrypto]

@iknowjason I really suggest using the -oJ output format. The greppable format can be used but splitting by space is not enough (your script doesn't currently work). By using the JSON output you can literally read the file and "transform" it into a python obj/dict and go from there

[iknowjason]

@thezoomerhacker Thanks for the suggestion. That can be easily done and I can do it in the next iteration. I had thought to do that with JSON but thought greppable would still be nice for analysis afterward. Question for you though, which script doesn't work for you - v1, v2, or v3? What version of masscan are you running? It might be the version of masscan and outputting a space differently than the version I used in my testbed. All three of my scripts worked with greppable output from masscan. That might explain the difference. Regardless, very good suggestion and thanks!

[tzmcrypto]

hey @iknowjason, I'm on version 1.0.6 and yeah, testing the script I realized the masscan output messes with the spaces. That's also why I suggested JSON in the first place. I managed to quickly edit your script and get it to work using the -oJ output format. I have it in local but here's the main difference

with open('%s' % str(sys.argv[1])) as f:
    loaded_json = json.load(f)
    for x in loaded_json:
        ### Parse the port only if open (if you want TCP ports only - specify here)
        if x["ports"][0]["status"] == "open":
            port = x["ports"][0]["port"]
            ip_addr = x["ip"]
            ### Add the IP address to dictionary if it doesn't already exist
            try:
                hosts[ip_addr]
            except KeyError:
                hosts[ip_addr] = {}

            ### Add the port list to dictionary if it doesn't already exist
            try:
                hosts[ip_addr][ports]
            except KeyError:
                hosts[ip_addr][ports] = []

            ## append the port to the list
            if port in hosts[ip_addr][ports]:
                pass
            else:
                hosts[ip_addr][ports].append(port)

[iknowjason]

@thezoomerhacker I see an issue for sure. I developed the scripts on Mac OS running masscan 1.04. Your JSON adaption doesn't even work with 1.04 outputted JSON file. Getting a JSON error parsing. I switched over to Linux with masscan 1.06 and your code above works just fine.

[tzmcrypto]

@iknowjason I see...it really does depend on the masscan version there's no workaround unfortunately. Good to know, thank you!

[iknowjason]

Hey @thezoomerhacker I've updated and tested all three of my scripts with support for JSON parsing. Thanks again. I've updated the gists and I've also create a new github repo that has all three of the scripts: https://github.com/iknowjason/masscan_nmap