Permalink
Browse files

initial

  • Loading branch information...
1 parent 4665d83 commit c04c1666177933376188bc1f9fdb770ff8706eeb @redronin committed Nov 22, 2012
View
@@ -1,6 +1,6 @@
# GoogleScraper
-TODO: Write a gem description
+Scrapes first 10 pages of Google.ca search results and saves url,text into output.csv file.
## Installation
@@ -18,7 +18,8 @@ Or install it yourself as:
## Usage
-TODO: Write usage instructions here
+ $ google_scraper "search term site:somesite.com"
+
## Contributing
View
@@ -0,0 +1,19 @@
+= google_scraper - DESCRIBE YOUR GEM
+
+Author:: YOUR NAME (YOUR EMAIL)
+Copyright:: Copyright (c) 2012 YOUR NAME
+
+
+DESCRIBE YOUR GEM HERE
+
+== Links
+
+* {Source on Github}[LINK TO GITHUB]
+* RDoc[LINK TO RDOC.INFO]
+
+== Install
+
+== Examples
+
+== Contributing
+
View
@@ -0,0 +1,44 @@
+#!/usr/bin/env ruby
+
+require 'optparse'
+require 'methadone'
+require 'google_scraper'
+require 'nokogiri'
+require 'open-uri'
+
+class App
+ include Methadone::Main
+ include Methadone::CLILogging
+
+ main do |query| # Add args you want: |like,so|
+
+ GoogleScraper.run(query)
+ end
+
+ # supplemental methods here
+
+ # Declare command-line interface here
+
+ # description "one line description of your app"
+ #
+ # Accept flags via:
+ # on("--flag VAL","Some flag")
+ # options[flag] will contain VAL
+ #
+ # Specify switches via:
+ # on("--[no-]switch","Some switch")
+ #
+ # Or, just call OptionParser methods on opts
+ #
+ # Require an argument
+ arg :query, :required
+ #
+ # # Make an argument optional
+ # arg :optional_arg, :optional
+
+ version GoogleScraper::VERSION
+
+ use_log_level_option
+
+ go!
+end
@@ -0,0 +1,13 @@
+Feature: My bootstrapped app kinda works
+ In order to get going on coding my awesome app
+ I want to have aruba and cucumber setup
+ So I don't have to do it myself
+
+ Scenario: App just runs
+ When I get help for "google_scraper"
+ Then the exit status should be 0
+ And the banner should be present
+ And the banner should document that this app takes options
+ And the following options should be documented:
+ |--version|
+ And the banner should document that this app takes no arguments
@@ -0,0 +1 @@
+# Put your step definitions here
@@ -0,0 +1,16 @@
+require 'aruba/cucumber'
+require 'methadone/cucumber'
+
+ENV['PATH'] = "#{File.expand_path(File.dirname(__FILE__) + '/../../bin')}#{File::PATH_SEPARATOR}#{ENV['PATH']}"
+LIB_DIR = File.join(File.expand_path(File.dirname(__FILE__)),'..','..','lib')
+
+Before do
+ # Using "announce" causes massive warnings on 1.9.2
+ @puts = true
+ @original_rubylib = ENV['RUBYLIB']
+ ENV['RUBYLIB'] = LIB_DIR + File::PATH_SEPARATOR + ENV['RUBYLIB'].to_s
+end
+
+After do
+ ENV['RUBYLIB'] = @original_rubylib
+end
@@ -7,7 +7,7 @@ Gem::Specification.new do |gem|
gem.name = "google_scraper"
gem.version = GoogleScraper::VERSION
gem.authors = ["John Tajima"]
- gem.email = ["john@shopify.com"]
+ gem.email = ["johntajima@gmail.com"]
gem.description = %q{Scrapes first 10 pages of google with given query}
gem.summary = %q{Scrapes first 10 pages of google with given query}
gem.homepage = ""
View
@@ -0,0 +1,56 @@
+url,text
+http://www.shopify.com/blog/6700752-9-elements-you-need-to-conversion-test,What is Conversion Testing? 9 Website Elements to A/B Test.
+http://support.shopify.com/customer/portal/articles/75397-how-can-i-test-orders-without-being-charged-by-shopify-,Shopify Support: How can I test orders without being char...
+http://wiki.shopify.com/Troubleshooting_PayPal_Website_Payments_Pro,PayPal - Shopify Wiki
+http://ecommerce.shopify.com/c/shopify-discussion/t/testing-migration-from-magento-54355,testing migration from magento - Ecommerce Forums - Shopify
+http://ecommerce.shopify.com/c/ecommerce-marketing/t/a-b-testing-55230,A/B Testing - Ecommerce Forums - Shopify
+http://ecommerce.shopify.com/c/ecommerce-marketing/t/a-b-testing-44736,A/B Testing - Ecommerce Forums - Shopify
+http://ecommerce.shopify.com/c/shopify-discussion/t/testing-105919,Testing - Shopify Discussion - Shopify Ecommerce Discussion Forums
+http://ecommerce.shopify.com/ecommerce/bogus,bogus gateway - Ecommerce Forums - Shopify
+http://static.shopify.com/s/files/1/0079/8612/t/1/assets/PS_Stainless_Steel_Cleaner_8oz.pdf,Stainless Steel - Shopify
+http://static.shopify.com/s/files/1/0079/8612/t/1/assets/PS_Stainless_Steel_Cleaner.pdf,Stainless Steel - Shopify
+http://static.shopify.com/s/files/1/0079/8612/t/1/assets/PS_Glass_Cleaner.pdf,nuveraproducts.com - Shopify
+http://cdn.shopify.com/s/files/1/0088/4082/t/3/assets/Ludaire_Engineered_Flooring_Installation.pdf%3F1644,Engineered Flooring Installation Instructions
+http://ecommerce.shopify.com/c/shopify-discussion/t/smart-collections-acting-dumb-30558,Smart collections acting dumb? - Ecommerce Forums - Shopify
+http://cdn.shopify.com/s/files/1/0017/7152/files/MCN-GB-review-sm.pdf%3F1257882558,Next .. -
+http://cdn.shopify.com/s/files/1/0027/5622/files/Larry_McConkey_comments.pdf%3F1287650375,"June 15, 2005 Excerpt from Steadicam Forum posting I did not want ..."
+http://www.shopify.com/technology/4906352-salmagundi-for-thursday-december-15-2011,"Salmagundi for Thursday, December 15, 2011 — Ecommerce Blog ..."
+http://cdn.shopify.com/s/files/1/0153/9461/files/WHO_IPA_hand_sanitiser_formulations.pdf%3F2206,WHO-recommended Handrub Formulations
+http://static.shopify.com/s/files/1/0077/9972/t/1/assets/Ausblu_Breez_A4.pdf,How do Breez Air Purif iers work? - Shopify
+http://ecommerce.shopify.com/c/ecommerce-design/t/back-button-woes-35677,Back Button Woes - Ecommerce Forums - Shopify
+http://ecommerce.shopify.com/c/ecommerce-design/t/turn-contents-of-an-article-into-a-liquid-array-35152,turn contents of an article into a liquid array..? - Shopify Design ...
+http://static0.shopify.com/s/files/1/0011/2372/files/Toxicity_Testing_FDA-registered_lab_9_06_2007_R1.pdf,Neuon - Shopify
+http://cdn.shopify.com/s/files/1/0061/6332/files/IMDG_Code_35_Changes.pdf%3F1291827878,E
+http://ecommerce.shopify.com/c/ecommerce-design/t/theme-change-9199,Theme Change - Ecommerce Forums - Shopify
+http://cdn.shopify.com/s/files/1/0038/9582/files/ATSAM2195UserGuide.pdf%3F1,ATSAM2195 User Guide.pdf
+http://ecommerce.shopify.com/c/shopify-discussion/t/webhooks-17070,Webhooks - Ecommerce Forums - Shopify
+http://static3.shopify.com/s/files/1/0028/5492/files/Lead_Test.pdf,Report No. : GR:TX:7410062889 DATE : 27/07/2007 - Shopify
+http://ecommerce.shopify.com/c/ecommerce-gallery/t/what-do-you-hate-about-our-store-34897,What do you HATE about our store?! - Ecommerce Forums - Shopify
+http://cdn.shopify.com/s/files/1/0062/7112/files/SocialDesignVolumeOneDigitalEdition.pdf%3F100303,Social Design in Museums
+http://www.shopify.com/technology%3Fpage%3D8,8 - Shopify
+http://wiki.shopify.com/Asset_url,Asset url - Shopify Wiki
+http://support.shopify.com/customer/portal/articles/75397-how-can-i-test-orders-without-being-charged-by-shopify-,Shopify Support: How can I test orders without being char...
+http://cdn.shopify.com/s/files/1/0062/7112/files/SocialDesignVolumeOneDigitalEdition.pdf%3F100303,Social Design in Museums
+http://www.shopify.com/technology%3Fpage%3D8,8 - Shopify
+http://wiki.shopify.com/Asset_url,Asset url - Shopify Wiki
+http://support.shopify.com/customer/portal/articles/75397-how-can-i-test-orders-without-being-charged-by-shopify-,Shopify Support: How can I test orders without being char...
+http://cdn.shopify.com/s/files/1/0062/7112/files/SocialDesignVolumeOneDigitalEdition.pdf%3F100303,Social Design in Museums
+http://www.shopify.com/technology%3Fpage%3D8,8 - Shopify
+http://wiki.shopify.com/Asset_url,Asset url - Shopify Wiki
+http://support.shopify.com/customer/portal/articles/75397-how-can-i-test-orders-without-being-charged-by-shopify-,Shopify Support: How can I test orders without being char...
+http://cdn.shopify.com/s/files/1/0062/7112/files/SocialDesignVolumeOneDigitalEdition.pdf%3F100303,Social Design in Museums
+http://www.shopify.com/technology%3Fpage%3D8,8 - Shopify
+http://wiki.shopify.com/Asset_url,Asset url - Shopify Wiki
+http://support.shopify.com/customer/portal/articles/75397-how-can-i-test-orders-without-being-charged-by-shopify-,Shopify Support: How can I test orders without being char...
+http://cdn.shopify.com/s/files/1/0062/7112/files/SocialDesignVolumeOneDigitalEdition.pdf%3F100303,Social Design in Museums
+http://www.shopify.com/technology%3Fpage%3D8,8 - Shopify
+http://wiki.shopify.com/Asset_url,Asset url - Shopify Wiki
+http://support.shopify.com/customer/portal/articles/75397-how-can-i-test-orders-without-being-charged-by-shopify-,Shopify Support: How can I test orders without being char...
+http://cdn.shopify.com/s/files/1/0062/7112/files/SocialDesignVolumeOneDigitalEdition.pdf%3F100303,Social Design in Museums
+http://www.shopify.com/technology%3Fpage%3D8,8 - Shopify
+http://wiki.shopify.com/Asset_url,Asset url - Shopify Wiki
+http://support.shopify.com/customer/portal/articles/75397-how-can-i-test-orders-without-being-charged-by-shopify-,Shopify Support: How can I test orders without being char...
+http://cdn.shopify.com/s/files/1/0062/7112/files/SocialDesignVolumeOneDigitalEdition.pdf%3F100303,Social Design in Museums
+http://www.shopify.com/technology%3Fpage%3D8,8 - Shopify
+http://wiki.shopify.com/Asset_url,Asset url - Shopify Wiki
+http://support.shopify.com/customer/portal/articles/75397-how-can-i-test-orders-without-being-charged-by-shopify-,Shopify Support: How can I test orders without being char...
@@ -0,0 +1,7 @@
+require 'test/unit'
+
+class TestSomething < Test::Unit::TestCase
+ def test_truth
+ assert true
+ end
+end

0 comments on commit c04c166

Please sign in to comment.