Skip to content
This repository

A cute HTML scraper / data extraction tool in under 70 lines of code

branch: master

Fetching latest commit…


Cannot retrieve the latest commit at this time

Octocat-spinner-32 examples
Octocat-spinner-32 lib
Octocat-spinner-32 .gitignore
Octocat-spinner-32 .travis.yml
Octocat-spinner-32 Gemfile
Octocat-spinner-32 LICENSE
Octocat-spinner-32 Rakefile
Octocat-spinner-32 nibbler.gemspec


Nibbler is a small little tool (~100 LOC) that helps you map data structures to objects that you define.

It can be used for HTML screen scraping:

require 'nibbler'
require 'open-uri'

class BlogScraper < Nibbler
  element :title

  elements 'div.hentry' => :articles do
    element 'h2' => :title
    element 'a/@href' => :url

blog = BlogScraper.parse open('')

#=> "My blog title"

#=> "First article title"

#=> ""

For mapping XML API payloads:

class Movie < Nibbler
  element './title/@regular' => :name
  element './box_art/@large' => :poster_large
  element 'release_year' => :year, :with => lambda { |node| node.text.to_i }
  element './/link[@title="web page"]/@href' => :url

response = Net::HTTP.get_response URI('')
movie = Movie.parse response.body  #=> "Toy Story 3"
movie.year  #=> 2010

Or even for JSON:

require 'json'
require 'nibbler/json'

class Movie < NibblerJSON
  element :title
  element :year
  elements :genres
  # JSONPath selectors:
  element '.links.alternate' => :url
  element '.ratings.critics_score' => :critics_score

movie = Movie.parse json_string

There are sample scripts in the "examples/" directory:

ruby -Ilib -rubygems examples/delicious.rb
ruby -Ilib -rubygems examples/tweetburner.rb > output.csv

See the wiki for more on how to use Nibbler.


None. Well, Nokogiri is a requirement if you pass in an HTML string for parsing, like in the example above. Otherwise you can initialize the scraper with an Hpricot document or anything else that implements at(selector) and search(selector) methods.

NibblerJSON needs a JSON parser if string content is passed, so "json" library should be installed on Ruby 1.8.

Something went wrong with that request. Please try again.