WebRecon2

WebRecon2 efficiently and accurately enumerates subdomains.

Installation

1. Clone the repo:

$ git clone https://github.com/sammooredev/WebRecon2.git

2. cd into folder:

$ cd WebRecon2

3. You have two options; either build the binary, or run with "go run":

$ go build 
$ ./WebRecon

$ go run main.go

Dependencies

Golang 1.21

• tested on go1.21

Tools that must be reachable within your $PATH:

1. amass
2. subfinder
3. dnsgen
4. puredns
   • massdns - binary will also need to be accessible within your $PATH

What does this tool do?

WebRecon2 utilizes the best tools available, each great at their own job, and combines them into a single script to automate a workflow that would typically be followed manually when performing subdomain enumeration.

Step 1: User Input

Input is supplied as a list of domains in the domains.txt file. The domains.txt file must be created using the following directory:

/Programs/<program name>/recon-data/domains.txt

can be named anthing you like. The contents of "domains.txt" should be formatted as follows:

foo.com 
bar.com
foo.bar.com

Once a domains.txt file is created, the tool can be run as so:

./WebRecon2 <program name>

Step 2: Subdomain Enumeration & Generation

This is the first stage of WebRecon2. The domains supplied as input are passed to amass & subfinder, where both tools simulataneously perform subdomain enumeration.

In parallel to subdomain enumeration being performed, a process of subdomain generation also occurs.

Each word in ./wordlists/httparchive_subddomains_2022_12_28.txt is prepended to each domain from domains.txt. This generates millions of potential subdomains. The concept reads weirdly, so consider this graphic to understand the process:

insert graphic

Currently you'll have to edit the code to change the wordlist thats used, but a plan to add an update feature in the future to pull the most recent files from https://wordlists.assetnote.io/.

Upon completion of the subdomain enumeration tools and and subdomain generation algorithm, the results are combined into a single file: all_enumerated_subdomains_combined_unique.txt

4. A new directory is created for each domain in the domains.txt file, within each new directory the subdomains of that domain are placed into a new file (<domain>-subdomains.out).

5. One instance of puredns is run for each entry in domains.txt file, unless a subdomain of a domain already included in the domains.txt file is present. In the case that the domains.txt file includes entries as shown in step 1 (bar.com & foo.bar.com), only one instance of puredns is ran using the higher level domain (bar.com) as input.

6. The output of puredns (a list of subdomains that had DNS records), are passed to dnsgen. This generates a new file containing permutations of puredns' output.

7. Puredns is then ran against the dnsgen output, to unconver even more subdomains.

8. Outputs a directory for each domain defined in domains.txt, containing results. A list of all unqiue subdomains for each domain combined is outputted as "final_list_unique.out"

Each tool generates a file as output and it isnt trashed by WebRecon2 after it's done running.

How to use

To create your own program and run WebRecon2 against it, perform the following:

1. Create the folder structure for the program within the ./Programs directory. Starbucks will be used as an example.

$ mkdir -p ./Programs/Starbucks/recon-data

2. Create a "domains.txt" within the recon-data directory you just created. Define a domain to be tested on each line.

$ vim ./Programs/Starbucks/recon-data/domains.txt

3. Run WebRecon.

$ ./WebRecon Starbucks

Once WebRecon2 has started running, it will create a directory using the current date to store its data. The output folder will ultimately be structured like so:

If you wish to test WebRecon2 with a quickstart, the Starbucks program structure is included in the repo. Just do the following after installing and building. It will test a single domain (starbucks.com):

$ ./WebRecon Starbucks

Usage Demo

• 1 hour 32 minutes run time
• 2570 unique hosts found

Future Plans:

• add a function to check that the needed tools exists within $PATH and throw errors if not.
• use rapid7 fdns data
• pull the most recent files from https://wordlists.assetnote.io/ for subdomain generation.

Resources:

This tool is based off awesome blogs by 0xPatrik

• Subdomain Enumeration: 2019 Workflow
• Subdomain Enumeration: Doing it a Bit Smarter