JavaScript
Switch branches/tags
Nothing to show
Latest commit 46f3505 May 5, 2013 @bencevans npm install
Permalink
Failed to load latest commit information.
.gitignore initial commit May 5, 2013
README.md npm install May 5, 2013
example.js initial commit May 5, 2013
index.js initial commit May 5, 2013
package.json package.json May 5, 2013

README.md

SpiderCheck

basic web crawler

npm install spidercheck

var spider = new SpiderCheck()

Events

  • 'error' - (error)
  • 'crawl' - (statusCode, url, res)

spider.addURL(url)

add a URL to the crawl queue

spider._crawl()

start crawling