Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Parallel requests with Celluloid::IO

Shannon Skipper edited this page · 5 revisions
Clone this wiki locally

Want to make multiple requests in parallel like you can with the Typhoeus gem? Want to do it from a single thread? And do you want to avoid callback hell?

You can use Celluloid::IO, an evented programming library for Ruby, to accomplish this by leveraging the HTTP Gem's dependency injection APIs.

Here is how to write a single-threaded parallel HTTP fetcher using the HTTP Gem:

Taken from examples/parallel_requests_with_celluloid.rb

require 'celluloid/io'
require 'http'

class HttpFetcher
  include Celluloid::IO

  def fetch(url)
    # Note: For SSL support specify:
    # ssl_socket_class: Celluloid::IO::SSLSocket
    HTTP.get(url, socket_class: Celluloid::IO::TCPSocket)
  end
end

fetcher = HttpFetcher.new

urls = %w(http://www.ruby-lang.org/ http://www.rubygems.org/ http://celluloid.io/)

# Kick off a bunch of future calls to HttpFetcher to grab the URLs in parallel
futures = urls.map { |u| [u, fetcher.future.fetch(u)] }

# Consume the results as they come in
futures.each do |url, future|
  # Wait for HttpFetcher#fetch to complete for this request
  response = future.value
  puts "*** Got #{url}: #{response.inspect}\n\n"
end
Something went wrong with that request. Please try again.