Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Generic data adapter for CouchDB
tree: e9c03b44b2

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
workers
README.md
couchpotato.js

README.md

Couch Potato

Couch Potato is a data adapter for CouchDB, a generic server that lets you easily read from various data sources and put that data into CouchDB.

Usage

Start a couchpotato process with the url to a database, couchpotato will create this database if it doesnt exist, then listen to the database for scheduled jobs.

$ node couchpotato.js http://127.0.0.1:5984/couchpotato

Once the process is started, to schedule a job you write a document to the database, each job takes a custom job description which you can find documented below, this example will read the google.com homepage every out and save the results to your local database myscreenscapes

POST http://127.0.0.1:5984/couchpotato
{ "worker": "webpage"
, "ttl": 60 * 60 * 100
, "opts": {"uri":http://google.com"}
, "destination": "myscreenscrapes"
}

There are a few common global attributes you can add to jobs, they are all optional

  • ttl - (defaults to 30 minutes) This is the time in milliseconds between subsequen runs of the job
  • destination - This is the database that the results get written to, you can specify a full url (including auth details), if you only specify a string it will get saved to the same host that the couchpotato database is.
  • opaque - This will passed through and written in the same

Screenscrape a webpage

{ "worker": "webpage"
, "ttl": 60 * 60 * 100
, "opts": {"uri":http://google.com"}
, "destination": "myscreenscrapes"
}

This is currently the only worker written.

TODO:

  • RSS reader
  • Twitter API reader (problem with oauth?)
  • Github Issues
  • POP / Email Reader
  • Foursquare / Gowalla etc
Something went wrong with that request. Please try again.