Permalink
Browse files

Adding wrest and nokogiri example

  • Loading branch information...
1 parent 94bd457 commit 90b51c8356542876d3add2005917e41994c88a30 @zmcartor committed Nov 2, 2012
Showing with 39 additions and 0 deletions.
  1. +39 −0 http_fun/wrest_fun.rb
View
@@ -0,0 +1,39 @@
+#Web scraping with wrest is pretty fun. System interface is easy
+# http://wrest.rubyforge.org/
+#
+# calls can be made just with shooting strings
+# "http://google.com".to_uri.get.get.body
+#
+# and then maybe throw that guy into nokogiri
+#
+# blah = "http://google.com".to_uri.get.body
+# parsed = Nokogiri::HTML(blah)
+# parsed.class should be
+# => Nokogiri::HTML::Document
+#
+# Let's go!
+#
+require 'wrest'
+require 'nokogiri'
+
+if ARGV.length.zero?
+ p "Please pass a valid web address ex: taivara.com"
+end
+
+addr = "http://"+ARGV[0]
+
+begin
+ req_body = addr.to_uri.get.body
+rescue NoMethodError
+ p "Couldn't fetch the address #{addr} , maybe you mistyped it ?"
+
+rescue SocketError
+ p "Sorry, couldn't find that address (DNS error)"
+end
+
+#parse with Nokogiri
+
+parsed = Nokogiri::HTML req_body
+
+#print out all the links on the page, just for example:
+parsed.css('a').each {|x| p x.attributes['href'].value}

0 comments on commit 90b51c8

Please sign in to comment.