Do simple scraping from the command line using XPath or CSS Selectors!
CoffeeScript JavaScript
Switch branches/tags
Nothing to show
Latest commit c69a587 Oct 2, 2015 @pthrasher bump version
Permalink
Failed to load latest commit information.
docs updated annotated source Nov 14, 2012
lib fix #2 Oct 2, 2015
spec patch bump, working version. Nov 14, 2012
.gitignore initial Nov 14, 2012
.npmignore initial Nov 14, 2012
.travis.yml
LICENSE-MIT initial Nov 14, 2012
README.md bump version Oct 2, 2015
grunt.js patch bump, working version. Nov 14, 2012
package.json bump version Oct 2, 2015

README.md

cli-scrape

Build Status

screen scrape from command line with xpath or css selectors

Getting Started

Install the module with: npm install -g cli-scrape

Then try out the following:

$ scrape http://whatthecommit.com/ '//p[0]/text()'
$ scrape http://whatthecommit.com/ 'p:first-child'

Or some of these:

# get a list of all of my public repos from github
$ scrape https://github.com/pthrasher 'li.public.source h3 a'

# Check if a website is down via down for everyone or just me:
$ scrape http://www.downforeveryoneorjustme.com/google.com '/html/body//div[@id="container"]/text()' | head -n 1
It's just you.
$ scrape http://www.downforeveryoneorjustme.com/lkjsdflkjsdf.com '/html/body//div[@id="container"]/text()' | head -n 1
It's not just you!

# Or make it a shell function:
function justme() {
    scrape http://www.downforeveryoneorjustme.com/$1 '/html/body//div[@id="container"]/text()' | head -n 1
}

# Then run it like so:
$ justme google.com
It's just you
$ justme lkjsdflkjsdf.com
It's not just you!

Contributing

In lieu of a formal styleguide, take care to maintain the existing coding style. Add unit tests for any new or changed functionality. Lint and test your code using grunt.

Release History

  • 0.1.10 - Bug fix release
    • Supports node 4.x
    • Updated to latest jsdom
    • Fix for #2
  • 0.1.9 - Initial release
    • Supports XPath queries via google's wicked fast xpath library
    • Supports css selector queries using qwery

License

Copyright (c) 2012 pthrasher Licensed under the MIT license.