Browse files

Updated README

  • Loading branch information...
1 parent fd44f7f commit 5fab6c31e74c8cbe586cb559c8da9d19e43769c6 @chriso chriso committed Nov 17, 2010
Showing with 5 additions and 6 deletions.
  1. +5 −6
@@ -6,6 +6,8 @@ For usage details, run
$ --help
+To get started, see the [documentation](, [examples](, or [API](
## What is []( is a framework for scraping and processing data. A job typically consists of a) taking some input, b) using or transforming it, and c) outputting something.
@@ -18,7 +20,7 @@ can simplify the process of:
- Parsing log files
- Transforming data from one format to another, e.g. from CSV to a database
- Recursively load all files in a directory and execute a command on each
## Why
- Create modular and extensible jobs for scraping and processing data
@@ -40,14 +42,11 @@ can simplify the process of:
Initial documentation is [available here](
-Better documentation will be available once I have time to write it.. See []( for updates.
-## Examples
-See [./examples](
+Better documentation will be available once I have time to write it.
## Roadmap
+- Fix up the []( site
- Automatically handle HTTP codes, e.g. redirect on 3xx or call fail() on 4xx/5xx
- Nested requests inherit referrer / cookies if to the same domain
- Add more DOM selector / traversal methods

0 comments on commit 5fab6c3

Please sign in to comment.