Crawl_in_it is a python tool designed to discover URLs. It will help penteration tester and bugbounty hunters to save time, so that they don't have to go manually through all the pages. This tool will generate logs which can helps penetration tester and bugbounty hunters retrieve the results of actions they performed in the past and also will allow them to see the status codes and length of the pages, the tool has crawled through.
git clone https://github.com/itszeeshan/crawlinit.git
Crawl in it currently only supports python3.
- The recommended version for python3 is 3.10.x
python -m pip install -r requirements.txt
sudo pip install -r requirements.txt
pip install numpy
pip install argparse
pip install torch
pip install traitlets
pip install bs4
pip install requests
pip install typing_extensions
| Short | Long | Description |
|---|---|---|
| -d | --domain | Domain to crawl into |
| -o | --output | Save results to file |
| -h | --help | Shows help message and exit |
- to generate help message
python Crawler.py -h
- Specifying domain for crawling
python Crawler.py -d hackerone.com
- Specifying domain for crawling and saving results into a file
python Crawler.py -d hackerone.com -o hackerone.txt