Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Concurrently download http requests in a synchronous environment.

branch: master

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 lib
Octocat-spinner-32 spec
Octocat-spinner-32 .gitignore
Octocat-spinner-32 .rspec
Octocat-spinner-32 Gemfile
Octocat-spinner-32 LICENSE
Octocat-spinner-32 README.md
Octocat-spinner-32 Rakefile initial commit
Octocat-spinner-32 concurrent_downloader.gemspec
README.md

ConcurrentDownloader

Concurrently download http requests in a synchronous environment. Basically this is an em-http-request wrapper that slots into a normal ruby script.

This would be useful inside a rails environment or a rake task.

Installation

Add this line to your application's Gemfile:

gem 'concurrent_downloader'

And then execute:

$ bundle

Or install it yourself as:

$ gem install concurrent_downloader

Usage

Check out spec/lib/concurrent_downloader_spec.rb for lots of usage examples. Here is one:

  queue = []
  100.times do |i|
    queue << {
      :method => "post",
      :path   => "/test"
    }
  end

  responses = []
  ConcurrentDownloader.process_queue!(queue, :host => "http://localhost", :concurrent_downloads => 10) do |queue_item, response|
    if response.status == 200
      responses << response
    end
  end

  # if all responses are 200
  responses.size.should == 100

The test suite passes in both 1.8.7 and 1.9.3

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Added some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

Contact

Oliver Clarke ollie@clarketus.net

Something went wrong with that request. Please try again.