Permalink
Browse files

This marks the first 0.0.1 release of the async client for sqs all

future development will land on feature branches and be picked into the
main line as needed.
  • Loading branch information...
0 parents commit f253b16164425576e036c91aad9188fe168d3026 @leongersing leongersing committed with Leon Gersing May 27, 2011
17 Gemfile
@@ -0,0 +1,17 @@
+source "http://rubygems.org"
+
+gem 'eventmachine'
+gem 'em-http-request', "1.0.0.beta.3"
+gem 'nokogiri'
+gem 'configuration'
+gem 'json'
+gem 'tzinfo'
+gem 'ruby-hmac'
+
+group :test do
+ gem 'rspec', ">= 2.6.0"
+ gem 'rspec-core'
+ gem 'rspec-expectations'
+ gem 'mocha'
+ gem 'timecop'
+end
@@ -0,0 +1,46 @@
+GEM
+ remote: http://rubygems.org/
+ specs:
+ addressable (2.2.6)
+ configuration (1.2.0)
+ diff-lcs (1.1.2)
+ em-http-request (1.0.0.beta.3)
+ addressable (>= 2.2.3)
+ em-socksify
+ eventmachine
+ http_parser.rb (>= 0.5.1)
+ em-socksify (0.1.0)
+ eventmachine
+ eventmachine (0.12.10)
+ http_parser.rb (0.5.1)
+ json (1.5.1)
+ mocha (0.9.12)
+ nokogiri (1.4.4)
+ rspec (2.6.0)
+ rspec-core (~> 2.6.0)
+ rspec-expectations (~> 2.6.0)
+ rspec-mocks (~> 2.6.0)
+ rspec-core (2.6.3)
+ rspec-expectations (2.6.0)
+ diff-lcs (~> 1.1.2)
+ rspec-mocks (2.6.0)
+ ruby-hmac (0.4.0)
+ timecop (0.3.5)
+ tzinfo (0.3.27)
+
+PLATFORMS
+ ruby
+
+DEPENDENCIES
+ configuration
+ em-http-request (= 1.0.0.beta.3)
+ eventmachine
+ json
+ mocha
+ nokogiri
+ rspec (>= 2.6.0)
+ rspec-core
+ rspec-expectations
+ ruby-hmac
+ timecop
+ tzinfo
@@ -0,0 +1,66 @@
+# SQS Async
+## A (mostly) non-blocking Amazon SQS client
+
+The goal of this library is simple. For those of us needing a way to communicate with SQS
+using an evented solution, like EventMachine, there exists no library that will make the requests
+without blocking the main thread and diminishing the value that EM provides.
+
+This simple library is a work in progress that allows for this behavior. At the moment very few
+options are supported but we will be add more as need arises and/or happily accept pull requests
+from those of you out there who may have found this gem out of a similar need.
+
+# Usage
+
+In order to use the library, simply mix the SQS module into the class that will be making your calls to SQS. At somepoint,
+You'll want to ensure that you've set your keys and secret in that class. Will we go more global with configuration? Probably,
+but like my uncle Larry says: "If it don't hurt, don't change it."
+
+ class MySQSClient
+ include SQS
+
+ def initialize
+ @aws_key = "YOUR KEY"
+ @aws_secret = "SHHHHHHHHHHHHHHHHHHHHHH"
+ end
+
+ end
+
+From there, you can make your sqs calls as part of your initialized object. Just be sure to do it inside of an
+EventMachine run loop.
+
+ client = MySQSClient.new
+ client.list_queues
+
+# Conventions
+
+The Library uses a simple "callback" aka "hollaback" system to communicate completion events. The last argument of all service calls
+is a hash of callbacks. Currently supported are :success and :failure. You can add others, we simply will ignore them.
+
+ client.list_queues( :success => lambda {|queues| puts queues } )
+
+# Fork it.
+
+We love to write and maintain code but there are only so many hours in a day! :) We encourage others to pick up where we left off and issue pull
+requests back to us. We only ask a few things....
+
++ fork the project
++ create a feature branch
++ create your patch (be sure to include specs!)
++ make sure that you patch can be applied cleanly
++ send us a pull request
++ bask in the glory that is Open Source!
+
+## Bonus points!
+
+You get extra points for the following...
+
++ Not blocking. Or minimize it as much as possible.
++ Not reaching out to the interwebs with your specs
++ Any new surface area additions to the API accept the hash with the supported callbacks.
++ Default callbacks are on keys :success and :failure
+
+That's pretty much it. For all bugs/errata/whathaveyou we're using github for issues or contact us @ contact@edgecase.com
+
+Thanks!
+
+-- Leon and John from EdgeCase
@@ -0,0 +1,7 @@
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec
+
+
@@ -0,0 +1,162 @@
+require 'uri'
+require 'cgi'
+require 'hmac'
+require 'hmac-sha2'
+require 'base64'
+require 'net/http'
+require 'time'
+require 'nokogiri'
+require 'json'
+require 'eventmachine'
+require 'em-http-request'
+require 'sqs_message'
+require 'sqs_queue'
+require 'sqs_attributes'
+require 'logger'
+
+module SQS
+ attr_accessor :aws_key, :aws_secret, :regions, :default_parameters, :post_options
+
+ def list_queues(options={})
+ prefix = options.delete(:prefix)
+ match = options.delete(:match)
+
+ options.merge!( "Action" => "ListQueues" )
+ options.merge!( "QueueNamePrefix" => encode(prefix) ) if prefix
+
+ call_amazon(options) do |req|
+ queues = SQSQueue.parse(req.response)
+ queues.select!{|q| q.queue_url.path.match(match) } if match
+ queues
+ end
+ end
+
+ def receive_message(options={})
+ raise "no target queue specified" unless options[:queue]
+ options.merge!("Action" => "ReceiveMessage", "MaxNumberOfMessages" => 10 )
+ call_amazon(options){ |req| SQSMessage.parse(req.response) }
+ end
+
+ def delete_message(options={})
+ raise "no Message specified" unless options[:message]
+ options.merge!("Action" => "DeleteMessage", "ReceiptHandle" => options.delete(:message).receipt_handle)
+ call_amazon(options)
+ end
+
+ def get_queue_attributes(options={})
+ raise "no target queue specified" unless options[:queue]
+ options.merge!( "Action" => "GetQueueAttributes", "AttributeName" => "All" )
+ call_amazon(options){ |req| SQSAttributes.parse(req.response) }
+ end
+
+ private
+
+ def call_amazon(options)
+ endpoint = (options[:queue] != nil) ? options.delete(:queue).queue_url : "http://" << ( options.delete(:host) || region_host(:us_east) )
+ callbacks = options.delete(:callbacks) || {:success=>nil, :failure =>nil }
+
+ params = sign_params( endpoint, options )
+ req = EM::HttpRequest.new("#{endpoint}?#{params}").get
+ req.callback do |req|
+ if(req.response.to_s.match(/<ErrorResponse>/i))
+ on_failure(req, callbacks)
+ else
+ result = req
+ result = yield req if block_given?
+ callbacks[:success].call(result) if callbacks[:success]
+ end
+ end
+ req.errback do |req|
+ on_failure(req, callbacks)
+ end
+ end
+
+ def on_failure(req, callbacks)
+ result = (req.error != nil) ? req.error : req.response
+ log result
+ callbacks[:failure].call(result) if callbacks[:failure]
+ end
+
+ def sign_params(uri, opts)
+ uri = URI.parse(uri) if uri.kind_of? String
+ opts = default_paramters.merge(opts)
+
+ sorted_params = opts.sort {|x,y| x[0] <=> y[0]}
+ encoded_params = sorted_params.collect do |p|
+ encode(p[0].to_s) << "=" << encode(p[1].to_s)
+ end
+ params_string = encoded_params.join("&")
+
+ req_desc = ["GET", uri.host.downcase, uri.request_uri, params_string].join("\n")
+ params_string << "&Signature=" << generate_signature(req_desc)
+ end
+
+ def generate_signature(request_description)
+ hmac = HMAC::SHA256.new(aws_secret)
+ hmac.update(request_description)
+ encode(Base64.encode64(hmac.digest).chomp)
+ end
+
+ def encoding_exclusions
+ /[^\w\d\-\_\.\~]/
+ end
+
+ def encode(val)
+ URI.encode(val, encoding_exclusions)
+ end
+
+ def region_host(key)
+ regions[key][:uri]
+ end
+
+ def regions
+ @regions ||= Regions
+ @regions
+ end
+
+ def default_paramters
+ @default_paramters ||= Parameters.merge("AWSAccessKeyId" => aws_key)
+ @default_paramters.merge("Expires" => (Time.now+(60*30)).utc.iso8601)
+ end
+
+ def post_options
+ @post_options ||= PostOptions
+ @post_options
+ end
+
+ def logger
+ return @logger if @logger
+
+ @logger = Logger.new @log_path || "./sqs_async.log"
+ @logger.level = @log_level || Logger::WARN
+ @logger
+ end
+
+ def log(msg)
+ log_msg = ["SERVICE ERROR"]
+ log_msg << caller[0..7].join("\n\t")
+ log_msg << "-".ljust(80, "-")
+ log_msg << msg
+ log_msg << "-".ljust(80, "-")
+ logger.error(log_msg.join("\n"))
+ end
+
+ Regions = {
+ :us_east => { :name => "US-East (Northern Virginia) Region", :uri => "sqs.us-east-1.amazonaws.com"},
+ :us_west => { :name => "US-West (Northern California) Region", :uri => "sqs.us-west-1.amazonaws.com"},
+ :eu => { :name => "EU (Ireland) Region", :uri => "sqs.eu-west-1.amazonaws.com"},
+ :asia_singapore => { :name => "Asia Pacific (Singapore) Region", :uri => "sqs.ap-southeast-1.amazonaws.com"},
+ :asia_tokyo => { :name => "Asia Pacific (Tokyo) Region", :uri => "sqs.ap-northeast-1.amazonaws.com"}
+ }
+
+
+ Parameters = {
+ "Version" => "2009-02-01",
+ "SignatureVersion"=>"2",
+ "SignatureMethod"=>"HmacSHA256",
+ }
+
+ PostOptions = {
+ "Content-Type" => "application/x-www-form-urlencoded"
+ }
+end
@@ -0,0 +1,38 @@
+require 'nokogiri'
+require 'json'
+require 'base64'
+require 'ostruct'
+
+class SQSAttributes
+ attr_accessor :approximate_number_of_messages,
+ :approximate_number_of_messages_not_visible,
+ :visibility_timeout,
+ :create_timestamp,
+ :last_modified_timestamp,
+ :policy,
+ :maximum_message_size,
+ :message_retention_period,
+ :queue_arn
+
+ def self.parse(xml)
+ doc = Nokogiri::XML(xml)
+ queue_data = SQSAttributes.new
+ doc.search("Attribute").each do |attribute|
+ meth = underscore(attribute.at("Name").text.strip)
+ if queue_data.respond_to? meth.to_sym
+ queue_data.send "#{meth}=", attribute.at("Value").text.strip
+ end
+ end
+ queue_data
+ end
+
+ # Taken from ActiveSupport. License information can be found at rubyonrails.org
+ def self.underscore(camel_cased_word)
+ word = camel_cased_word.to_s.dup
+ word.gsub!(/([A-Z]+)([A-Z][a-z])/,'\1_\2')
+ word.gsub!(/([a-z\d])([A-Z])/,'\1_\2')
+ word.tr!("-", "_")
+ word.downcase!
+ word
+ end
+end
@@ -0,0 +1,22 @@
+require 'nokogiri'
+require 'json'
+require 'base64'
+require 'ostruct'
+
+class SQSMessage
+ attr_accessor :body, :md5_of_body, :message_id, :receipt_handle
+
+ def self.parse(xml)
+ doc = Nokogiri::XML(xml)
+ messages = []
+ doc.search("Message").each do |message_element|
+ s = SQSMessage.new
+ s.body = message_element.at("Body").text.strip
+ s.md5_of_body = message_element.at("MD5OfBody").text.strip
+ s.message_id = message_element.at("MessageId").text.strip
+ s.receipt_handle = message_element.at("ReceiptHandle").text.strip
+ messages << s
+ end
+ messages
+ end
+end
@@ -0,0 +1,18 @@
+require 'nokogiri'
+require 'json'
+require 'base64'
+
+class SQSQueue
+ attr_accessor :queue_url
+
+ def self.parse(xml)
+ doc = Nokogiri::XML(xml)
+ queues = []
+ doc.search("QueueUrl").each do |element|
+ s = SQSQueue.new
+ s.queue_url = URI.parse(element.text.strip)
+ queues << s
+ end
+ queues
+ end
+end
@@ -0,0 +1,2 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<DeleteQueueResponse><ResponseMetadata><RequestId>6fde8d1e-52cd-4581-8cd9-c512f4c64223</RequestId></ResponseMetadata></DeleteQueueResponse>
@@ -0,0 +1,2 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<ErrorResponse><Error><Type>Sender</Type><Code>AuthFailure</Code><Message>The provided signature is not valid for this access token</Message><Detail/></Error><RequestId>ef3aba6a-dc84-4937-91bf-cef2ddd6775a</RequestId></ErrorResponse>
@@ -0,0 +1,2 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<ListQueuesResponse><ListQueuesResult><QueueUrl>http://sqs.us-east-1.amazonaws.com/123456789012/testQueue</QueueUrl></ListQueuesResult><ResponseMetadata><RequestId>725275ae-0b9b-4762-b238-436d7c65a1ac</RequestId></ResponseMetadata></ListQueuesResponse>
Oops, something went wrong.

0 comments on commit f253b16

Please sign in to comment.