Find file
Fetching contributors…
Cannot retrieve contributors at this time
143 lines (95 sloc) 4.37 KB

Google Custom Search

This project is a Ruby lib for Google's Custom Search ENgine API ( There seem to be quite a few cse libs out there that don't work so I rolled this up quickly.

Questions/comments, etc:


Add to your Gemfile:

gem "google_custom_search_api"


bundle install


You need to configure GOOGLE_SEARCH_CX and GOOGLE_API_KEY to config/initializers/google_cse_api.rb:

  GOOGLE_API_KEY = "..."

You can get your GOOGLE_API_KEY from - There are many choices - Simple API Access is probably what you want. There are more elaborate authorization schemes available for Google services but those aren't currently implemented.

You can get your GOOGLE_SEARCH_CX from Either create a custom engine or follow manage your existing search engines and go to your cse's Control panel. GOOGLE_SEARCH_CX == Search engine unique ID

Searching the web, not just your site, with CSE

Google CSE was set up so search specific sites. To search the entire web simply go to, find your CSE, go to it's control panel.

  • in Basics under Search Preferences choose Search the entire web but emphasize included sites.
  • in Sites add


To perform a search:

  results ="poker")

Results now contains a raw version and a class'ed version of the data show in Sample results below.

This means you can do:

  results["items"].each do |item|
    puts item["title"], item["link"]


  results.items.each do |item|
    puts item.title,

You can get all ten pages at once by doing:

  results = GoogleCustomSearchApi.search_and_return_all_results(query, opts) 
  results.size == 10
  results.collect {|r| r.items.size }.sum == 100 #if there were 100 results

search_and_return_all_results also yields results as it goes:

  GoogleCustomSearchApi.search_and_return_all_results(query, opts) do |results|
    results.items.size == 10

See Custom Search documentation for an explanation of all fields available.


By default CSE returns a maximum of 10 results at a time, you can't get more results without paging. BTW if you want fewer results just pass in the :num => 1-10 option when searching.

To do paging we pass in the :start option. Example:

  results ="poker", :start => 1)

The maximum number of pages CSE allows is 10 - or 100 results in total. To walk through the pages you can use :start => 1, :start => 11, etc. Or you can use the results to find the next value, like so:

  start = 1
    results ="poker",:start => start)
    if results.queries.keys.include?("nextPage")
      start = results.queries.nextPage.first.startIndex
      start = nil
  end while start.nil? == false

If you just want all results you can use the method search_and_return_all_results(query, opts = {}) works just like the normal search but iterates through all available results and puts them in an array.

Encoding issues

TODO - this section needs work

CSE will return non utf-8 results which can be problematic. I might add in a config value that you can explicitly set encoding. Until then a work around is doing stuff like:


More on this here:

Contributing - Running tests

Pull requests welcome.

To run tests

  git clone
  cd google_custom_search_api
  bundle install
  bundle exec rspec spec



  • pretty light on the tests

Sample results

See spec/fixtures/*.json for examples of data returned

Copyright (c) 2012 Ben Wiseley, released under the MIT license