Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
A careless caching client optimized for scraping.
Ruby
Branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
examples
lib
.gitignore
CHANGELOG.md
README.md
Rakefile
UNLICENSE
cacho.gemspec

README.md

Cacho

A careless caching client optimized for scraping.

Description

Cacho is an HTTP client for scraping. It will do most of the things you want when scraping:

  • Follow redirects.
  • Set the User-Agent to a browser-like string.
  • Accept and process gzip encoding.
  • Detect when it's been rate limited and wait.
  • Retry on silly network errors.
  • Use persistent HTTP connections.

Most importantly, Cacho will store responses on disk so that multiple runs of your script will not hit the endpoints you are scraping. This prevents being rate limited and also makes your script faster every time.

Usage

require "cacho"

client = Cacho.new

res = client.request(:get, "https://news.ycombinator.com")

Installation

$ gem install cacho

License

See the UNLICENSE.

Something went wrong with that request. Please try again.