tcrawl

tcrawl is a Command Line Interface for Turbo Crawler. It allows you to start, stop, and manage your web crawlers.

Quick Start

$ npm i -g tcrawl

$ tcrawl

Turbo Crawl 0.3.1

Here are some commands to get you started:

1$ tcrawl start &  | You probably want to do this first.
                   |
2$ tcrawl generate | Generates lists of websites for Turbo Crawl to visit.
                   |
3$ tcrawl crawl    | Begins crawling one or more websites.
                   |
4$ tcrawl help     | Show a list of all commands.