PR for issue 654 (environment identifiers dict)

[righettod]

Hi,

This PR refer to the issue #654

I have used the following command against several local (Luxembourg) domains:

curl -sk "https://crt.sh/?q=[BASE_DOMAIN]&output=json" | jq -r ".[].name_value" | cut -d'.' -f1 | sort -u

Domains used, as sources, were defined in each commits. You will find English and French identifiers depending on the companies owing the domains.

Thank you very much in advance 😃

[righettod]

I have run the following script against all LU domains from the Majestic files:

Source:

• https://blog.majestic.com/development/alexa-top-1-million-sites-retired-heres-majestic-million/

Script:

#!/bin/bash
# wget http://downloads.majestic.com/majestic_million.csv
# cat majestic_million.csv | cut -d',' -f3 | grep "\.lu" > lu-domains.txt
rm sub*.txt 2>/dev/null
while IFS= read -r line
do
	printf "\rDomain: %-40s" "$line"
	curl -sk "https://crt.sh/?q=$line&output=json" | jq -r ".[].name_value" | cut -d'.' -f1 >> sub.txt
done < lu-domains.txt
cat sub.txt | sort -u > sub2.txt 

Generated files:

• lu-domains.txt
• sub.txt
• sub2.txt

New entries related to environments were added to the commit f86abe0

[g0tmi1k]

Oh wow! Thanks!
I wonder if putting in the scripts used to generate could be helpful?
The last commit I did, I made a ./.bin//. What about putting any/all scripts there?

[righettod]

I added the results, after the extraction from FR + BE sub domains, to the commit 9e778c6 :

Command used:

cat sub-be.txt sub-fr.txt | sort -u > subFRBE.txt
grep -Ei "^(de|dv|ts|te|in|st|ho|pr|pp)" subFRBE.txt > selection.txt

Generated files:

• sub2-be.txt
• sub2-fr.txt
• selection.txt

[righettod]

Oh wow! Thanks! I wonder if putting in the scripts used to generate could be helpful? The last commit I did, I made a ./.bin//. What about putting any/all scripts there?

OK I will refactor the script to be more generic.

I keep you informed about this, please wait before to merge this PR 😃

[righettod]

@g0tmi1k I added a more generic version of the script to the PR in the bin folder, I have tried to follow as much as possible the format of your script in order to match your standard 😃

[g0tmi1k]

Thank you SO MUCH for doing this!

This is great work (as always!)

[righettod]

@g0tmi1k Thank to you and the SecLists's team for this amazing project!