Skip to content
Tool to find stored robots.txt files from the past
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
README.md Update README.md Jun 7, 2019
wayrobots.py

README.md

WayRobots

A tool to find snapshots of robots.txt file stored at http://web.archive.org

Usage

Before you use make sure to have the required python libraries

$ pip install requests
$ pip install colorama

Basic use

$ python wayrobots.py -i target.com -y 2015-2019

Help page

$ python wayrobots.py -h
You can’t perform that action at this time.