Darknet mapper

Darknet mapper is a project I did in order to research about the darknet.
After exploring a bit inside the darknet I saw that is very hard to search for
links. Moreover, even if you find links a lot of them dosen't work.
So I decided to build a crawler that will search links for me and store a little bit of information on each link.

How it works

The way it works is by recursively extract all links from a web page and store them in a database
with the following information:

1. url
   
2. title
   
3. description
   
4. emails
   
5. telegram links
   
6. bitcoin wallets
   
7. gathered links
   
8. date
   
9. tags (category of the website by keywords)
   

Terminology

Clearnet - publicly accessible Internet.
Deep Web - parts of the World Wide Web whose contents are not indexed by standard web search-engines.
Darknet - overlay network within the Internet that can only be accessed with specific software, configurations, or
authorization, and often uses a unique customised communication protocol.

Installation

If you dont have mongodb installed make sure to install it:
https://docs.mongodb.com/manual/installation/

pip install -r requirements.txt

Usage

python3 crawler.py

Poc

After 12 hours of total run time I was able to get 1500 links as you can see in the json file (darknet_unique.json).

Disclaimer

This project is for educational purposes only, we are not responsible for any kind of abuse.