From de71860ef198076092a7a37e99bffc99e2b204aa Mon Sep 17 00:00:00 2001 From: Stewart McKee Date: Mon, 27 Nov 2017 13:39:10 +0000 Subject: [PATCH] Update README.textile --- README.textile | 4 ---- 1 file changed, 4 deletions(-) diff --git a/README.textile b/README.textile index 5656d85..f765d16 100644 --- a/README.textile +++ b/README.textile @@ -6,10 +6,6 @@ h1. Cobweb v1.1.0 !https://gemnasium.com/stewartmckee/cobweb.png! !https://coveralls.io/repos/stewartmckee/cobweb/badge.png?branch=master(Coverage Status)!:https://coveralls.io/r/stewartmckee/cobweb -" - !https://app.codesponsor.io/embed/Qr2oGXuncPZ4SKa8ABiA3QZn/stewartmckee/cobweb.svg(Sponsor)! -":https://app.codesponsor.io/link/Qr2oGXuncPZ4SKa8ABiA3QZn/stewartmckee/cobweb - h2. Intro CobWeb has three methods of running. Firstly it is a http client that allows get and head requests returning a hash of data relating to the requested resource. The second main function is to utilize this combined with the power of Resque to cluster the crawls allowing you crawl quickly. Lastly you can run the crawler with a block that uses each of the pages found in the crawl.