Concurrently download http requests in a synchronous environment.
Ruby
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
lib
spec
.gitignore
.rspec
Gemfile
LICENSE
README.md
Rakefile
concurrent_downloader.gemspec

README.md

ConcurrentDownloader

Concurrently download http requests in a synchronous environment. Basically this is an em-http-request wrapper that slots into a normal ruby script.

This would be useful inside a rails environment or a rake task.

Installation

Add this line to your application's Gemfile:

gem 'concurrent_downloader'

And then execute:

$ bundle

Or install it yourself as:

$ gem install concurrent_downloader

Usage

Check out spec/lib/concurrent_downloader_spec.rb for lots of usage examples. Here is one:

  queue = []
  100.times do |i|
    queue << {
      :method => "post",
      :path   => "/test"
    }
  end

  responses = []
  ConcurrentDownloader.process_queue!(queue, :host => "http://localhost", :concurrent_downloads => 10) do |queue_item, response|
    if response.status == 200
      responses << response
    end
  end

  # if all responses are 200
  responses.size.should == 100

The test suite passes in both 1.8.7 and 1.9.3

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Added some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

Contact

Oliver Clarke ollie@clarketus.net