Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Command line web page fetcher for dodgy site analysis

branch: master

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 README
Octocat-spinner-32 scraper
README
scraper - command line fetching tool for dodgy site analysis - rewritten in ruby

usage: scraper [options] URL
    --help, -h: show this help
    --verbose, -v: be verbose (defaults to stderr)
    --debug, -d: debug connections
    --log-file, -l: specify logfile, implies --verbose
    --user-agent, -u: specify the user-agent string
    --proxy, -p: proxy server to use in form http://host:port - defaults to on and http://localhost:8118 (privoxy/Tor)
    --no-proxy, -n: do not use a proxy
    --no-redirects, -r: do not follow redirects
    --no-body, -x: do not print the HTML response

    URL: the web page to scrape

Something went wrong with that request. Please try again.