MIT-LICENSE @@ -1,8 +1,37 @@ --------------- -MetaInspector --------------- -DESCRIPTION: Ruby gem for web scraping purposes. It scrapes a given URL, and returns you a hash with data from it like for example the title, meta description, meta keywords, an array with all the links, all the images in it, etc. -AUTHOR: Jaime Iniesta (jaimeiniesta@gmail.com, http://www.jaimeiniesta.com) -LICENSE: GPLv3 - -(more to come soon...) += MetaInspector + + MetaInspector is a gem for web scraping purposes. You give it an URL, and it returns you a hash with metadata from it. + += Installation + + Run the following if you haven't already: + + gem sources -a http://gems.github.com + + Then install the gem: + + sudo gem install jaimeiniesta-metainspector + += Usage + + Scrape a web and store the returned data like this: + + data = MetaInspector.scrape('http://pagerankalert.com') + + Then you can access the data returned: + + data['ok'] # returns a boolean, true if scraping went OK, false if there was an error + data['title'] # title of the page, as string + data['description] # meta description, as string + data['keywords'] # meta keywords, as string + data['links'] # array of strings, with every link found on the page + += ToDo + +* Easier access to data, like data.title, data.description, instead of data['title']. Return a MetaInspector object, not a hash. +* Tests & Mocks +* Return array of images in page +* Return contents of meta robots tag +* Consider using nokogiri instead of hpricot + +Copyright (c) 2009 Jaime Iniesta, released under the MIT license README @@ -4,7 +4,7 @@ Gem::Specification.new do |s| s.date = "2008-06-27" s.summary = "Ruby gem for web scraping" s.email = "jaimeiniesta@gmail.com" - s.homepage = "http://code.jaimeiniesta.com/metainspector" + s.homepage = "http://github.com/jaimeiniesta/metainspector/tree/master" s.description = "MetaInspector is a ruby gem for web scraping purposes, that returns a hash with metadata from a given URL" s.has_rdoc = false s.authors = ["Jaime Iniesta"] metainspector.gemspec 97e5f2a4b6b6f86acbbc153a46be2edee5610767 Jaime Iniesta jaimeiniesta@gmail.com http://github.com/jaimeiniesta/metainspector/commit/f74ad2be50e7eafffaeae28de65e5659584c6ad1 f74ad2be50e7eafffaeae28de65e5659584c6ad1 2009-05-13T08:32:28-07:00 2009-05-13T08:32:28-07:00 Updated readme and license a5dadce490a4d5ccd5c58f94e8c37fd4ee98e63d Jaime Iniesta jaimeiniesta@gmail.com