Welcome to crawling, documentation coming soon, for now see crawling -h.
Install ruby and gem then:
gem install crawling
Coming soon.
Bug reports and pull requests are welcome on GitHub at https://github.com/ohjames/crawling.
The gem is available as open source under the terms of the MIT License.