Skip to content

Releases: natzar/pythonSimpleCrawlerBot

v.1.0.1

Choose a tag to compare

@natzar natzar released this 22 Oct 18:55
  • Added limits to domains batches as user will encounter this script failing when number of domains collected is high
  • Added request timeout to fetch_domains_details function
  • Added a flag to allow or not subdomains or only store top level domains

Full Changelog: v1.0.0...v1.0.1

v.0.1.0-alpha

Choose a tag to compare

@natzar natzar released this 22 Oct 10:00

First version of this open crawler bot.