Skip to content

Latest commit

 

History

History
52 lines (36 loc) · 1.66 KB

README.md

File metadata and controls

52 lines (36 loc) · 1.66 KB

Darknet mapper

Darknet mapper is a project I did in order to research about the darknet.
After exploring a bit inside the darknet I saw that is very hard to search for
links. Moreover, even if you find links a lot of them dosen't work.
So I decided to build a crawler that will search links for me and store a little bit of information on each link.

How it works

The way it works is by recursively extract all links from a web page and store them in a database
with the following information:

  1. url
  2. title
  3. description
  4. emails
  5. telegram links
  6. bitcoin wallets
  7. gathered links
  8. date
  9. tags (category of the website by keywords)

Terminology

Clearnet - publicly accessible Internet.
Deep Web - parts of the World Wide Web whose contents are not indexed by standard web search-engines.
Darknet - overlay network within the Internet that can only be accessed with specific software, configurations, or
authorization, and often uses a unique customised communication protocol.

darknet_ice

Installation

If you dont have mongodb installed make sure to install it:
https://docs.mongodb.com/manual/installation/

pip install -r requirements.txt

Usage

python3 crawler.py

Poc

After 12 hours of total run time I was able to get 1500 links as you can see in the json file (darknet_unique.json).

Disclaimer

This project is for educational purposes only, we are not responsible for any kind of abuse.