Ruby and PostgreSQL scraper for Kickstarter
Ruby
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
classifier
Gemfile
Gemfile.lock
README
basic_kickstarter_test.rb
classes.rb
config_example.yaml
kickparser.rb
model.rb

README

Scraper for Kickstarter. 

Usage: 

ruby kickparser.rb

You'll have to edit kickparser to change what it's doing (Google scrape/directory) or scraping kickstarter itself. 

Hitting Google too hard can get you Googlebanned. 

Currently not yet under an open source license. 


Works with this gemset and Ruby 1.8.7. Might work with newer too. I dunno. Haven't debugged yet. :

actionmailer (2.3.5, 2.3.4)
actionpack (2.3.5, 2.3.4)
activerecord (2.3.5, 2.3.4)
activeresource (2.3.5, 2.3.4)
activesupport (2.3.5, 2.3.4)
addressable (2.2.3, 2.1.1)
classifier (1.3.3)
columnize (0.3.1)
data_objects (0.10.3, 0.10.2)
dm-core (1.0.2)
dm-validations (1.0.2)
em-http-request (0.2.7)
eventmachine (0.12.10)
extlib (0.9.15)
fast-stemmer (1.0.0)
ffi (0.6.3)
hoe (2.6.0)
hpricot (0.8.2)
htmlentities (4.2.0)
json (1.2.3)
json_pure (1.2.4)
linecache (0.43)
madeleine (0.7.3)
mechanize (2.0.1)
mysql (2.8.1)
net-http-digest_auth (1.1.1)
net-http-persistent (1.8)
nokogiri (1.4.6, 1.4.3.1)
ntp (1.0.0)
postgres (0.7.9.2008.01.28)
rack (1.0.1)
rails (2.3.5, 2.3.4)
rake (0.8.7)
ruby-debug (0.10.3)
ruby-debug-base (0.10.3)
rubyforge (2.0.4)
sequel (3.16.0)
stemmer (1.0.1)
webrobots (0.0.9)