Browse files

retry on request limits for persistent connections

nginx has a keepalive_requests directive that limits the
number of requests a client may make over a single keepalive
connection.  By default, this limit is 100 requests.

With the test script below, I get IOError on the 101st request
when hitting (my server).

$prefix/lib/ruby/1.9.1/net/protocol.rb:141:in `read_nonblock': closed stream (IOError)
	from $prefix/lib/ruby/1.9.1/net/protocol.rb:141:in `rbuf_fill'
	from $prefix/lib/ruby/1.9.1/net/protocol.rb:122:in `readuntil'
	from $prefix/lib/ruby/1.9.1/net/protocol.rb:132:in `readline'
	from $prefix/lib/ruby/1.9.1/net/http.rb:2562:in `read_status_line'
	from $prefix/lib/ruby/1.9.1/net/http.rb:2551:in `read_new'
	from $prefix/lib/ruby/gems/1.9.1/gems/net-http-pipeline-1.0/lib/net/http/pipeline.rb:298:in `pipeline_receive'
	from $prefix/lib/ruby/gems/1.9.1/gems/net-http-pipeline-1.0/lib/net/http/pipeline.rb:179:in `pipeline'
	from $prefix/lib/ruby/gems/1.9.1/gems/net-http-persistent-2.3.2/lib/net/http/persistent.rb:543:in `pipeline'
	from t.rb:21:in `block in <main>'
	from t.rb:19:in `times'
	from t.rb:19:in `<main>'

Ideally, Ruby could avoid raising IOError by not reading a
closed stream, but I don't have time to dig into net/http
internals so rescuing on IOError and retrying is easier.

Feel free to hit with the following test script
to confirm/test/debug this issue:

  # tested with ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux]
  require 'net/http/pipeline'
  require 'net/http/persistent' # tested with 2.3.2
  require 'uri'

  nr = (ARGV.shift || 101).to_i
  np = (ARGV.shift || 1).to_i
  puts "requests per pipeline call: #{nr}"
  puts "pipeline calls: #{np}"

  # runs nginx with the default
  # keepalive_requests=100 directive
  # Apache servers have a similar MaxKeepaliveRequests directive
  uri = URI('')
  http ='x')
  requests = { }

  # if nr is > keepalive_requests value on the server,
  # IOError gets raised
  np.times do |j|
    i = -1
    http.pipeline(uri, requests.dup) do |resp|
      p [ j, i += 1,  resp ]
  • Loading branch information...
1 parent 5a79031 commit 22005b9013920c8ae5c6fd39b92cfbf6beabe692 Eric Wong committed with Dec 19, 2011
Showing with 1 addition and 1 deletion.
  1. +1 −1 lib/net/http/pipeline.rb
2 lib/net/http/pipeline.rb
@@ -312,7 +312,7 @@ def pipeline_receive in_flight, responses
rescue Timeout::Error, EOFError, Errno::ECONNABORTED, Errno::ECONNRESET,
- Errno::EPIPE, Net::HTTPBadResponse => e
+ Errno::EPIPE, Net::HTTPBadResponse, IOError => e
raise, in_flight, responses)

0 comments on commit 22005b9

Please sign in to comment.