Repository navigation
Releases: natzar/pythonSimpleCrawlerBot
Releases · natzar/pythonSimpleCrawlerBot
Release list
v.1.0.1
- Added limits to domains batches as user will encounter this script failing when number of domains collected is high
- Added request timeout to fetch_domains_details function
- Added a flag to allow or not subdomains or only store top level domains
Full Changelog: v1.0.0...v1.0.1
v.0.1.0-alpha
First version of this open crawler bot.