Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
retry on request limits for persistent connections
nginx has a keepalive_requests directive that limits the number of requests a client may make over a single keepalive connection. By default, this limit is 100 requests. With the test script below, I get IOError on the 101st request when hitting bogomips.org (my server). $prefix/lib/ruby/1.9.1/net/protocol.rb:141:in `read_nonblock': closed stream (IOError) from $prefix/lib/ruby/1.9.1/net/protocol.rb:141:in `rbuf_fill' from $prefix/lib/ruby/1.9.1/net/protocol.rb:122:in `readuntil' from $prefix/lib/ruby/1.9.1/net/protocol.rb:132:in `readline' from $prefix/lib/ruby/1.9.1/net/http.rb:2562:in `read_status_line' from $prefix/lib/ruby/1.9.1/net/http.rb:2551:in `read_new' from $prefix/lib/ruby/gems/1.9.1/gems/net-http-pipeline-1.0/lib/net/http/pipeline.rb:298:in `pipeline_receive' from $prefix/lib/ruby/gems/1.9.1/gems/net-http-pipeline-1.0/lib/net/http/pipeline.rb:179:in `pipeline' from $prefix/lib/ruby/gems/1.9.1/gems/net-http-persistent-2.3.2/lib/net/http/persistent.rb:543:in `pipeline' from t.rb:21:in `block in <main>' from t.rb:19:in `times' from t.rb:19:in `<main>' Ideally, Ruby could avoid raising IOError by not reading a closed stream, but I don't have time to dig into net/http internals so rescuing on IOError and retrying is easier. Feel free to hit bogomips.org with the following test script to confirm/test/debug this issue: # tested with ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux] require 'net/http/pipeline' require 'net/http/persistent' # tested with 2.3.2 require 'uri' nr = (ARGV.shift || 101).to_i np = (ARGV.shift || 1).to_i puts "requests per pipeline call: #{nr}" puts "pipeline calls: #{np}" # bogomips.org runs nginx with the default # keepalive_requests=100 directive # Apache servers have a similar MaxKeepaliveRequests directive # https://httpd.apache.org/docs/2.2/mod/core.html#maxkeepaliverequests uri = URI('http://bogomips.org/') http = Net::HTTP::Persistent.new('x') requests = nr.times.map { Net::HTTP::Get.new(uri.path) } # if nr is > keepalive_requests value on the server, # IOError gets raised np.times do |j| i = -1 http.pipeline(uri, requests.dup) do |resp| p [ j, i += 1, resp ] end end
- Loading branch information