Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 1017 lines (881 sloc) 25.244 kB
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
1 # HTTPClient - HTTP client library.
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
2 # Copyright (C) 2000-2009 NAKAMURA, Hiroshi <nahi@ruby-lang.org>.
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
3 #
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
4 # This program is copyrighted free software by NAKAMURA, Hiroshi. You can
5 # redistribute it and/or modify it under the same terms of Ruby's license;
6 # either the dual license version in 2003, or any later version.
7
8 # httpclient/session.rb is based on http-access.rb in http-access/0.0.4.
9 # Some part of code in http-access.rb was recycled in httpclient.rb.
10 # Those part is copyrighted by Maehashi-san.
11
12
13 require 'socket'
14 require 'thread'
15 require 'stringio'
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
16 require 'zlib'
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
17
12581c4 * added original timeout.rb. see #191.
nahi authored
18 require 'httpclient/timeout'
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
19 require 'httpclient/ssl_config'
20 require 'httpclient/http'
21
22
23 class HTTPClient
24
25
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
26 # Represents a Site: protocol scheme, host String and port Number.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
27 class Site
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
28 # Protocol scheme.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
29 attr_accessor :scheme
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
30 # Host String.
c9b5f3e @nahi Rename 'local_bind' to 'socket_local'
authored
31 attr_accessor :host
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
32 # Port number.
c9b5f3e @nahi Rename 'local_bind' to 'socket_local'
authored
33 attr_accessor :port
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
34
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
35 # Creates a new Site based on the given URI.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
36 def initialize(uri = nil)
37 if uri
38 @scheme = uri.scheme
39 @host = uri.host
40 @port = uri.port.to_i
41 else
42 @scheme = 'tcp'
43 @host = '0.0.0.0'
44 @port = 0
45 end
46 end
47
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
48 # Returns address String.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
49 def addr
50 "#{@scheme}://#{@host}:#{@port.to_s}"
51 end
52
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
53 # Returns true is scheme, host and port are '=='
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
54 def ==(rhs)
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
55 (@scheme == rhs.scheme) and (@host == rhs.host) and (@port == rhs.port)
4fc7a2b * added tests for coverage. closes #189.
nahi authored
56 end
57
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
58 # Same as ==.
4fc7a2b * added tests for coverage. closes #189.
nahi authored
59 def eql?(rhs)
60 self == rhs
61 end
62
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
63 def hash # :nodoc:
4fc7a2b * added tests for coverage. closes #189.
nahi authored
64 [@scheme, @host, @port].hash
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
65 end
66
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
67 def to_s # :nodoc:
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
68 addr
69 end
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
70
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
71 # Returns true if scheme, host and port of the given URI matches with this.
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
72 def match(uri)
73 (@scheme == uri.scheme) and (@host == uri.host) and (@port == uri.port.to_i)
74 end
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
75
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
76 def inspect # :nodoc:
77 sprintf("#<%s:0x%x %s>", self.class.name, __id__, addr)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
78 end
d64b7bd @nahi Use @socket_local iif user specified it.
authored
79
80 EMPTY = Site.new.freeze
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
81 end
82
83
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
84 # Manages sessions for a HTTPClient instance.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
85 class SessionManager
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
86 # Name of this client. Used for 'User-Agent' header in HTTP request.
87 attr_accessor :agent_name
88 # Owner of this client. Used for 'From' header in HTTP request.
89 attr_accessor :from
90
91 # Requested protocol version
92 attr_accessor :protocol_version
93 # Chunk size for chunked request
94 attr_accessor :chunk_size
95 # Device for dumping log for debugging
96 attr_accessor :debug_dev
97 # Boolean value for Socket#sync
98 attr_accessor :socket_sync
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
99
100 attr_accessor :connect_timeout
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
101 # Maximum retry count. 0 for infinite.
102 attr_accessor :connect_retry
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
103 attr_accessor :send_timeout
104 attr_accessor :receive_timeout
08ec3ad @nahi Reuse cached session in MRU order, not in LRU
authored
105 attr_accessor :keep_alive_timeout
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
106 attr_accessor :read_block_size
3f2a996 * tested proxy + SSL against squid.
nahi authored
107 attr_accessor :protocol_retry_count
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
108
c9b5f3e @nahi Rename 'local_bind' to 'socket_local'
authored
109 # Local address to bind local side of the socket to
110 attr_accessor :socket_local
0297555 @icblenke Adding local_sockaddr
icblenke authored
111
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
112 attr_accessor :ssl_config
113
114 attr_reader :test_loopback_http_response
115
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
116 attr_accessor :transparent_gzip_decompression
117
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
118 def initialize(client)
119 @client = client
33b4eab * let SessionManager inherit proxy and debug_dev setting from client…
nahi authored
120 @proxy = client.proxy
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
121
122 @agent_name = nil
123 @from = nil
124
125 @protocol_version = nil
33b4eab * let SessionManager inherit proxy and debug_dev setting from client…
nahi authored
126 @debug_dev = client.debug_dev
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
127 @socket_sync = true
3df5871 @nahi Add test for streaming + non-chunked post
authored
128 @chunk_size = ::HTTP::Message::Body::DEFAULT_CHUNK_SIZE
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
129
130 @connect_timeout = 60
131 @connect_retry = 1
132 @send_timeout = 120
08ec3ad @nahi Reuse cached session in MRU order, not in LRU
authored
133 @receive_timeout = 60 # For each read_block_size bytes
134 @keep_alive_timeout = 15 # '15' is from Apache 2 default
7a3a73f * changed read buf size: 8k -> 16k. just follows net/http. see #191.
nahi authored
135 @read_block_size = 1024 * 16 # follows net/http change in 1.8.7
3f2a996 * tested proxy + SSL against squid.
nahi authored
136 @protocol_retry_count = 5
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
137
138 @ssl_config = nil
139 @test_loopback_http_response = []
140
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
141 @transparent_gzip_decompression = false
c9b5f3e @nahi Rename 'local_bind' to 'socket_local'
authored
142 @socket_local = Site.new
0297555 @icblenke Adding local_sockaddr
icblenke authored
143
0190684 @nahi Optimize cached session access performance
authored
144 @sess_pool = {}
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
145 @sess_pool_mutex = Mutex.new
0190684 @nahi Optimize cached session access performance
authored
146 @sess_pool_last_checked = Time.now
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
147 end
148
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
149 def proxy=(proxy)
150 if proxy.nil?
151 @proxy = nil
152 else
153 @proxy = Site.new(proxy)
154 end
155 end
156
157 def query(req, via_proxy)
3050bde @nahi Add shortcut methods.
authored
158 req.http_body.chunk_size = @chunk_size
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
159 sess = open(req.header.request_uri, via_proxy)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
160 begin
161 sess.query(req)
162 rescue
163 sess.close
164 raise
165 end
166 sess
167 end
168
169 def reset(uri)
170 site = Site.new(uri)
171 close(site)
172 end
173
174 def reset_all
175 close_all
176 end
177
08ec3ad @nahi Reuse cached session in MRU order, not in LRU
authored
178 # assert: sess.last_used must not be nil
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
179 def keep(sess)
180 add_cached_session(sess)
181 end
182
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
183 def invalidate(site)
184 @sess_pool_mutex.synchronize do
0190684 @nahi Optimize cached session access performance
authored
185 if pool = @sess_pool[site]
186 pool.each do |sess|
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
187 sess.invalidate
188 end
189 end
190 end
191 end
192
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
193 private
194
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
195 def open(uri, via_proxy = false)
0190684 @nahi Optimize cached session access performance
authored
196 site = Site.new(uri)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
197 sess = nil
0190684 @nahi Optimize cached session access performance
authored
198 if cached = get_cached_session(site)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
199 sess = cached
200 else
0190684 @nahi Optimize cached session access performance
authored
201 sess = Session.new(@client, site, @agent_name, @from)
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
202 sess.proxy = via_proxy ? @proxy : nil
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
203 sess.socket_sync = @socket_sync
204 sess.requested_version = @protocol_version if @protocol_version
205 sess.connect_timeout = @connect_timeout
206 sess.connect_retry = @connect_retry
207 sess.send_timeout = @send_timeout
208 sess.receive_timeout = @receive_timeout
209 sess.read_block_size = @read_block_size
3f2a996 * tested proxy + SSL against squid.
nahi authored
210 sess.protocol_retry_count = @protocol_retry_count
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
211 sess.ssl_config = @ssl_config
212 sess.debug_dev = @debug_dev
c9b5f3e @nahi Rename 'local_bind' to 'socket_local'
authored
213 sess.socket_local = @socket_local
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
214 sess.test_loopback_http_response = @test_loopback_http_response
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
215 sess.transparent_gzip_decompression = @transparent_gzip_decompression
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
216 end
217 sess
218 end
219
220 def close_all
e3c1331 * a few performance improvements. see #191.
nahi authored
221 @sess_pool_mutex.synchronize do
0190684 @nahi Optimize cached session access performance
authored
222 @sess_pool.each do |site, pool|
223 pool.each do |sess|
224 sess.close
225 end
e3c1331 * a few performance improvements. see #191.
nahi authored
226 end
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
227 end
228 @sess_pool.clear
229 end
230
9ada537 @nahi Make Keep-Alive test use threads properly.
authored
231 # This method might not work as you expected...
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
232 def close(dest)
0190684 @nahi Optimize cached session access performance
authored
233 if cached = get_cached_session(Site.new(dest))
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
234 cached.close
235 true
236 else
237 false
238 end
239 end
240
0190684 @nahi Optimize cached session access performance
authored
241 def get_cached_session(site)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
242 @sess_pool_mutex.synchronize do
0190684 @nahi Optimize cached session access performance
authored
243 now = Time.now
244 if now > @sess_pool_last_checked + @keep_alive_timeout
245 scrub_cached_session(now)
246 @sess_pool_last_checked = now
247 end
248 if pool = @sess_pool[site]
249 pool.each_with_index do |sess, idx|
250 if valid_session?(sess, now)
251 return pool.slice!(idx)
252 end
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
253 end
254 end
255 end
0190684 @nahi Optimize cached session access performance
authored
256 nil
257 end
258
259 def scrub_cached_session(now)
260 @sess_pool.each do |site, pool|
261 pool.replace(pool.select { |sess|
262 if valid_session?(sess, now)
263 true
264 else
265 sess.close # close & remove from the pool
266 false
267 end
268 })
269 end
270 end
271
272 def valid_session?(sess, now)
273 !sess.invalidated? and (now <= sess.last_used + @keep_alive_timeout)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
274 end
275
276 def add_cached_session(sess)
277 @sess_pool_mutex.synchronize do
0190684 @nahi Optimize cached session access performance
authored
278 (@sess_pool[sess.dest] ||= []).unshift(sess)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
279 end
280 end
281 end
282
283
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
284 # Wraps up OpenSSL::SSL::SSLSocket and offers debugging features.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
285 class SSLSocketWrap
286 def initialize(socket, context, debug_dev = nil)
287 unless SSLEnabled
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
288 raise ConfigurationError.new('Ruby/OpenSSL module is required')
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
289 end
290 @context = context
291 @socket = socket
4fc7a2b * added tests for coverage. closes #189.
nahi authored
292 @ssl_socket = create_openssl_socket(@socket)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
293 @debug_dev = debug_dev
294 end
295
a65bbbd @nahi Added SNI support. see #49.
authored
296 def ssl_connect(hostname = nil)
297 if hostname && @ssl_socket.respond_to?(:hostname=)
298 @ssl_socket.hostname = hostname
299 end
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
300 @ssl_socket.connect
301 end
302
303 def post_connection_check(host)
304 verify_mode = @context.verify_mode || OpenSSL::SSL::VERIFY_NONE
305 if verify_mode == OpenSSL::SSL::VERIFY_NONE
306 return
307 elsif @ssl_socket.peer_cert.nil? and
308 check_mask(verify_mode, OpenSSL::SSL::VERIFY_FAIL_IF_NO_PEER_CERT)
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
309 raise OpenSSL::SSL::SSLError.new('no peer cert')
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
310 end
311 hostname = host.host
312 if @ssl_socket.respond_to?(:post_connection_check) and RUBY_VERSION > "1.8.4"
313 @ssl_socket.post_connection_check(hostname)
314 else
315 @context.post_connection_check(@ssl_socket.peer_cert, hostname)
316 end
317 end
318
ed9c807 @nahi Dump more about SSL connection if $DEBUG
authored
319 def ssl_version
320 @ssl_socket.ssl_version if @ssl_socket.respond_to?(:ssl_version)
321 end
322
323 def ssl_cipher
324 @ssl_socket.cipher
325 end
326
327 def ssl_state
328 @ssl_socket.state
329 end
330
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
331 def peer_cert
332 @ssl_socket.peer_cert
333 end
334
335 def close
336 @ssl_socket.close
337 @socket.close
338 end
339
340 def closed?
341 @socket.closed?
342 end
343
344 def eof?
345 @ssl_socket.eof?
346 end
347
348 def gets(*args)
349 str = @ssl_socket.gets(*args)
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
350 debug(str)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
351 str
352 end
353
354 def read(*args)
355 str = @ssl_socket.read(*args)
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
356 debug(str)
357 str
358 end
359
360 def readpartial(*args)
361 str = @ssl_socket.readpartial(*args)
362 debug(str)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
363 str
364 end
365
366 def <<(str)
367 rv = @ssl_socket.write(str)
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
368 debug(str)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
369 rv
370 end
371
372 def flush
373 @ssl_socket.flush
374 end
375
376 def sync
377 @ssl_socket.sync
378 end
379
380 def sync=(sync)
381 @ssl_socket.sync = sync
382 end
383
384 private
385
386 def check_mask(value, mask)
387 value & mask == mask
388 end
389
4fc7a2b * added tests for coverage. closes #189.
nahi authored
390 def create_openssl_socket(socket)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
391 ssl_socket = nil
392 if OpenSSL::SSL.const_defined?("SSLContext")
393 ctx = OpenSSL::SSL::SSLContext.new
394 @context.set_context(ctx)
395 ssl_socket = OpenSSL::SSL::SSLSocket.new(socket, ctx)
396 else
397 ssl_socket = OpenSSL::SSL::SSLSocket.new(socket)
398 @context.set_context(ssl_socket)
399 end
400 ssl_socket
401 end
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
402
403 def debug(str)
404 @debug_dev << str if @debug_dev && str
405 end
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
406 end
407
408
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
409 # Wraps up a Socket for method interception.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
410 module SocketWrap
411 def initialize(socket, *args)
412 super(*args)
413 @socket = socket
414 end
415
416 def close
417 @socket.close
418 end
419
420 def closed?
421 @socket.closed?
422 end
423
424 def eof?
425 @socket.eof?
426 end
427
428 def gets(*args)
429 @socket.gets(*args)
430 end
431
432 def read(*args)
433 @socket.read(*args)
434 end
435
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
436 def readpartial(*args)
437 # StringIO doesn't support :readpartial
438 if @socket.respond_to?(:readpartial)
439 @socket.readpartial(*args)
440 else
441 @socket.read(*args)
442 end
443 end
444
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
445 def <<(str)
446 @socket << str
447 end
448
449 def flush
450 @socket.flush
451 end
452
453 def sync
454 @socket.sync
455 end
456
457 def sync=(sync)
458 @socket.sync = sync
459 end
460 end
461
462
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
463 # Module for intercepting Socket methods and dumps in/out to given debugging
464 # device. debug_dev must respond to <<.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
465 module DebugSocket
466 extend SocketWrap
467
468 def debug_dev=(debug_dev)
469 @debug_dev = debug_dev
470 end
471
472 def close
473 super
474 debug("! CONNECTION CLOSED\n")
475 end
476
477 def gets(*args)
478 str = super
479 debug(str)
480 str
481 end
482
483 def read(*args)
484 str = super
485 debug(str)
486 str
487 end
488
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
489 def readpartial(*args)
490 str = super
491 debug(str)
492 str
493 end
494
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
495 def <<(str)
496 super
497 debug(str)
498 end
499
500 private
501
502 def debug(str)
cfc2abc @chetan when debugging, don't dump binary data; display message instead
chetan authored
503 if str && @debug_dev
d5f0010 @nahi Additional change for the previous commit; do hexdump if a content lo…
authored
504 if str.index("\0")
505 require 'hexdump'
506 str.force_encoding('BINARY') if str.respond_to?(:force_encoding)
507 @debug_dev << HexDump.encode(str).join("\n")
cfc2abc @chetan when debugging, don't dump binary data; display message instead
chetan authored
508 else
509 @debug_dev << str
510 end
511 end
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
512 end
513 end
514
515
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
516 # Dummy Socket for emulating loopback test.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
517 class LoopBackSocket
518 include SocketWrap
519
520 def initialize(host, port, response)
350acc3 * applied a patch from user. IO#readpartial does not clear the secon…
nahi authored
521 super(response.is_a?(StringIO) ? response : StringIO.new(response))
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
522 @host = host
523 @port = port
524 end
525
526 def <<(str)
527 # ignored
528 end
529 end
530
531
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
532 # Manages a HTTP session with a Site.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
533 class Session
7936c39 * do not override original timeout. just use it in HTTPClient::Sess…
nahi authored
534 include HTTPClient::Timeout
9a079a1 @nahi Use absolute URI iif via proxy AND not HTTPS. closes #41.
authored
535 include Util
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
536
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
537 # Destination site
538 attr_reader :dest
539 # Proxy site
540 attr_accessor :proxy
541 # Boolean value for Socket#sync
542 attr_accessor :socket_sync
543 # Requested protocol version
544 attr_accessor :requested_version
545 # Device for dumping log for debugging
546 attr_accessor :debug_dev
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
547
548 attr_accessor :connect_timeout
549 attr_accessor :connect_retry
550 attr_accessor :send_timeout
551 attr_accessor :receive_timeout
552 attr_accessor :read_block_size
3f2a996 * tested proxy + SSL against squid.
nahi authored
553 attr_accessor :protocol_retry_count
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
554
c9b5f3e @nahi Rename 'local_bind' to 'socket_local'
authored
555 attr_accessor :socket_local
0297555 @icblenke Adding local_sockaddr
icblenke authored
556
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
557 attr_accessor :ssl_config
558 attr_reader :ssl_peer_cert
559 attr_accessor :test_loopback_http_response
560
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
561 attr_accessor :transparent_gzip_decompression
08ec3ad @nahi Reuse cached session in MRU order, not in LRU
authored
562 attr_reader :last_used
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
563
2253f95 * test coverage checked. see #189.
nahi authored
564 def initialize(client, dest, agent_name, from)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
565 @client = client
566 @dest = dest
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
567 @invalidated = false
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
568 @proxy = nil
569 @socket_sync = true
570 @requested_version = nil
571
572 @debug_dev = nil
573
574 @connect_timeout = nil
575 @connect_retry = 1
576 @send_timeout = nil
577 @receive_timeout = nil
578 @read_block_size = nil
3f2a996 * tested proxy + SSL against squid.
nahi authored
579 @protocol_retry_count = 5
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
580
581 @ssl_config = nil
582 @ssl_peer_cert = nil
583
584 @test_loopback_http_response = nil
c9b5f3e @nahi Rename 'local_bind' to 'socket_local'
authored
585 @socket_local = Site::EMPTY
0297555 @icblenke Adding local_sockaddr
icblenke authored
586
2253f95 * test coverage checked. see #189.
nahi authored
587 @agent_name = agent_name
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
588 @from = from
589 @state = :INIT
590
591 @requests = []
592
593 @status = nil
594 @reason = nil
595 @headers = []
596
597 @socket = nil
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
598 @readbuf = nil
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
599
600 @transparent_gzip_decompression = false
08ec3ad @nahi Reuse cached session in MRU order, not in LRU
authored
601 @last_used = nil
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
602 end
603
604 # Send a request to the server
605 def query(req)
4fc7a2b * added tests for coverage. closes #189.
nahi authored
606 connect if @state == :INIT
9a079a1 @nahi Use absolute URI iif via proxy AND not HTTPS. closes #41.
authored
607 # Use absolute URI (not absolute path) iif via proxy AND not HTTPS.
608 req.header.request_absolute_uri = !@proxy.nil? and !https?(@dest)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
609 begin
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
610 timeout(@send_timeout, SendTimeoutError) do
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
611 set_header(req)
612 req.dump(@socket)
613 # flush the IO stream as IO::sync mode is false
614 @socket.flush unless @socket_sync
615 end
2cd3385 @nahi rescue IOError for rather old JRuby versions.
authored
616 rescue Errno::ECONNABORTED, Errno::ECONNRESET, Errno::EPIPE, IOError
3ec4757 @nahi JRuby exception support: see #25.
authored
617 # JRuby can raise IOError instead of ECONNRESET for now
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
618 close
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
619 raise KeepAliveDisconnected.new(self)
2253f95 * test coverage checked. see #189.
nahi authored
620 rescue HTTPClient::TimeoutError
621 close
622 raise
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
623 rescue
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
624 close
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
625 if SSLEnabled and $!.is_a?(OpenSSL::SSL::SSLError)
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
626 raise KeepAliveDisconnected.new(self)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
627 else
628 raise
629 end
630 end
631
632 @state = :META if @state == :WAIT
633 @next_connection = nil
634 @requests.push(req)
08ec3ad @nahi Reuse cached session in MRU order, not in LRU
authored
635 @last_used = Time.now
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
636 end
637
638 def close
639 if !@socket.nil? and !@socket.closed?
640 # @socket.flush may block when it the socket is already closed by
641 # foreign host and the client runs under MT-condition.
642 @socket.close
643 end
644 @state = :INIT
645 end
646
647 def closed?
648 @state == :INIT
649 end
650
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
651 def invalidate
652 @invalidated = true
653 end
654
655 def invalidated?
656 @invalidated
657 end
658
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
659 def get_header
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
660 begin
661 if @state != :META
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
662 raise RuntimeError.new("get_status must be called at the beginning of a session")
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
663 end
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
664 read_header
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
665 rescue
666 close
667 raise
668 end
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
669 [@version, @status, @reason, @headers]
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
670 end
671
672 def eof?
673 if !@content_length.nil?
674 @content_length == 0
675 else
676 @socket.closed? or @socket.eof?
677 end
678 end
679
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
680 def get_body(&block)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
681 begin
4fc7a2b * added tests for coverage. closes #189.
nahi authored
682 read_header if @state == :META
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
683 return nil if @state != :DATA
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
684 if @gzipped and @transparent_gzip_decompression
685 # zlib itself has a functionality to decompress gzip stream.
686 # - zlib 1.2.5 Manual
687 # http://www.zlib.net/manual.html#Advanced
688 # > windowBits can also be greater than 15 for optional gzip decoding. Add 32 to
689 # > windowBits to enable zlib and gzip decoding with automatic header detection,
690 # > or add 16 to decode only the gzip format
691 inflate_stream = Zlib::Inflate.new(Zlib::MAX_WBITS + 32)
692 original_block = block
693 block = Proc.new { |buf|
694 original_block.call(inflate_stream.inflate(buf))
695 }
696 end
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
697 if @chunked
698 read_body_chunked(&block)
699 elsif @content_length
700 read_body_length(&block)
701 else
702 read_body_rest(&block)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
703 end
704 rescue
705 close
706 raise
707 end
708 if eof?
709 if @next_connection
710 @state = :WAIT
711 else
712 close
713 end
714 end
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
715 nil
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
716 end
717
718 private
719
720 def set_header(req)
b4523c3 * performance improvements especially on HTTP::Message. see #191.
nahi authored
721 if @requested_version
722 if /^(?:HTTP\/|)(\d+.\d+)$/ =~ @requested_version
1949938 @nahi Do not depend on Float#to_s behavior. closes #36.
authored
723 req.http_version = $1
b4523c3 * performance improvements especially on HTTP::Message. see #191.
nahi authored
724 end
725 end
2253f95 * test coverage checked. see #189.
nahi authored
726 if @agent_name
727 req.header.set('User-Agent', "#{@agent_name} #{LIB_NAME}")
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
728 end
729 if @from
730 req.header.set('From', @from)
731 end
c327d9b @nahi Add 'Accept: */*' header to request by default
authored
732 if req.header.get('Accept').empty?
733 req.header.set('Accept', '*/*')
734 end
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
735 if @transparent_gzip_decompression
736 req.header.set('Accept-Encoding', 'gzip,deflate')
737 end
c041811 @nahi Allow to set Date header manually
authored
738 if req.header.get('Date').empty?
739 req.header.set_date_header
740 end
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
741 end
742
743 # Connect to the server
744 def connect
745 site = @proxy || @dest
746 retry_number = 0
747 begin
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
748 timeout(@connect_timeout, ConnectTimeoutError) do
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
749 @socket = create_socket(site)
9a079a1 @nahi Use absolute URI iif via proxy AND not HTTPS. closes #41.
authored
750 if https?(@dest)
ac0fa20 * added a test for CONNECT proxy auth.
nahi authored
751 if @socket.is_a?(LoopBackSocket)
752 connect_ssl_proxy(@socket, URI.parse(@dest.to_s)) if @proxy
753 else
754 @socket = create_ssl_socket(@socket)
755 connect_ssl_proxy(@socket, URI.parse(@dest.to_s)) if @proxy
ed9c807 @nahi Dump more about SSL connection if $DEBUG
authored
756 begin
757 @socket.ssl_connect(@dest.host)
758 ensure
759 if $DEBUG
760 warn("Protocol version: #{@socket.ssl_version}")
761 warn("Cipher: #{@socket.ssl_cipher.inspect}")
762 warn("State: #{@socket.ssl_state}")
763 end
764 end
ac0fa20 * added a test for CONNECT proxy auth.
nahi authored
765 @socket.post_connection_check(@dest)
766 @ssl_peer_cert = @socket.peer_cert
767 end
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
768 end
90174d9 * spell check.
nahi authored
769 # Use Ruby internal buffering instead of passing data immediately
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
770 # to the underlying layer
90174d9 * spell check.
nahi authored
771 # => we need to to call explicitly flush on the socket
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
772 @socket.sync = @socket_sync
773 end
774 rescue RetryableResponse
775 retry_number += 1
776 if retry_number < @protocol_retry_count
777 retry
778 end
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
779 raise BadResponseError.new("connect to the server failed with status #{@status} #{@reason}")
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
780 rescue TimeoutError
781 if @connect_retry == 0
782 retry
783 else
784 retry_number += 1
785 retry if retry_number < @connect_retry
786 end
787 close
788 raise
789 end
790 @state = :WAIT
791 end
792
793 def create_socket(site)
794 socket = nil
795 begin
796 @debug_dev << "! CONNECT TO #{site.host}:#{site.port}\n" if @debug_dev
797 if str = @test_loopback_http_response.shift
798 socket = LoopBackSocket.new(site.host, site.port, str)
d64b7bd @nahi Use @socket_local iif user specified it.
authored
799 elsif @socket_local == Site::EMPTY
800 socket = TCPSocket.new(site.host, site.port)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
801 else
c9b5f3e @nahi Rename 'local_bind' to 'socket_local'
authored
802 socket = TCPSocket.new(site.host, site.port, @socket_local.host, @socket_local.port)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
803 end
804 if @debug_dev
805 @debug_dev << "! CONNECTION ESTABLISHED\n"
806 socket.extend(DebugSocket)
807 socket.debug_dev = @debug_dev
808 end
809 rescue SystemCallError => e
810 e.message << " (#{site})"
811 raise
812 rescue SocketError => e
813 e.message << " (#{site})"
814 raise
815 end
816 socket
817 end
818
819 # wrap socket with OpenSSL.
820 def create_ssl_socket(raw_socket)
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
821 SSLSocketWrap.new(raw_socket, @ssl_config, @debug_dev)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
822 end
823
824 def connect_ssl_proxy(socket, uri)
288f58b * added RDocs for util, http, auth, ssl_config and timeout. closes …
nahi authored
825 req = HTTP::Message.new_connect_request(uri)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
826 @client.request_filter.each do |filter|
827 filter.filter_request(req)
828 end
829 set_header(req)
830 req.dump(@socket)
831 @socket.flush unless @socket_sync
832 res = HTTP::Message.new_response('')
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
833 parse_header
1949938 @nahi Do not depend on Float#to_s behavior. closes #36.
authored
834 res.http_version, res.status, res.reason = @version, @status, @reason
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
835 @headers.each do |key, value|
2739f8d @nahi Allow symbol Header name for HTTP request
authored
836 res.header.set(key.to_s, value)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
837 end
838 commands = @client.request_filter.collect { |filter|
839 filter.filter_response(req, res)
840 }
841 if commands.find { |command| command == :retry }
842 raise RetryableResponse.new
843 end
844 unless @status == 200
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
845 raise BadResponseError.new("connect to ssl proxy failed with status #{@status} #{@reason}", res)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
846 end
847 end
848
849 # Read status block.
850 def read_header
2a71767 * when we hit some site through http-proxy we get a response without…
nahi authored
851 @content_length = nil
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
852 @chunked = false
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
853 @gzipped = false
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
854 @chunk_length = 0
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
855 parse_header
d24ab1d @nahi do not try to read when a response code is 1XX, 204 or 304. closes #10.
authored
856 # Header of the request has been parsed.
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
857 @state = :DATA
858 req = @requests.shift
d24ab1d @nahi do not try to read when a response code is 1XX, 204 or 304. closes #10.
authored
859 if req.header.request_method == 'HEAD' or no_message_body?(@status)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
860 @content_length = 0
861 if @next_connection
862 @state = :WAIT
863 else
864 close
865 end
866 end
43730ac @nahi Reenable keep-alive for chunked response.
authored
867 @next_connection = false if !@content_length and !@chunked
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
868 end
869
870 StatusParseRegexp = %r(\AHTTP/(\d+\.\d+)\s+(\d\d\d)\s*([^\r\n]+)?\r?\n\z)
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
871 def parse_header
2253f95 * test coverage checked. see #189.
nahi authored
872 timeout(@receive_timeout, ReceiveTimeoutError) do
3ec4757 @nahi JRuby exception support: see #25.
authored
873 initial_line = nil
2253f95 * test coverage checked. see #189.
nahi authored
874 begin
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
875 initial_line = @socket.gets("\n")
2253f95 * test coverage checked. see #189.
nahi authored
876 if initial_line.nil?
3ec4757 @nahi JRuby exception support: see #25.
authored
877 close
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
878 raise KeepAliveDisconnected.new(self)
2253f95 * test coverage checked. see #189.
nahi authored
879 end
3ec4757 @nahi JRuby exception support: see #25.
authored
880 rescue Errno::ECONNABORTED, Errno::ECONNRESET, Errno::EPIPE, IOError
881 # JRuby can raise IOError instead of ECONNRESET for now
882 close
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
883 raise KeepAliveDisconnected.new(self)
3ec4757 @nahi JRuby exception support: see #25.
authored
884 end
885 begin
2253f95 * test coverage checked. see #189.
nahi authored
886 if StatusParseRegexp !~ initial_line
887 @version = '0.9'
888 @status = nil
889 @reason = nil
890 @next_connection = false
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
891 @content_length = nil
2253f95 * test coverage checked. see #189.
nahi authored
892 @readbuf = initial_line
893 break
894 end
895 @version, @status, @reason = $1, $2.to_i, $3
1949938 @nahi Do not depend on Float#to_s behavior. closes #36.
authored
896 @next_connection = HTTP::Message.keep_alive_enabled?(@version)
2253f95 * test coverage checked. see #189.
nahi authored
897 @headers = []
898 while true
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
899 line = @socket.gets("\n")
2253f95 * test coverage checked. see #189.
nahi authored
900 unless line
901 raise BadResponseError.new('unexpected EOF')
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
902 end
2253f95 * test coverage checked. see #189.
nahi authored
903 line.chomp!
904 break if line.empty?
7968719 @nahi multiline response message header support.
authored
905 if line[0] == ?\ or line[0] == ?\t
906 last = @headers.last[1]
907 last << ' ' unless last.empty?
908 last << line.strip
909 else
910 key, value = line.strip.split(/\s*:\s*/, 2)
911 parse_keepalive_header(key, value)
912 @headers << [key, value]
913 end
2253f95 * test coverage checked. see #189.
nahi authored
914 end
915 end while (@version == '1.1' && @status == 100)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
916 end
917 end
918
d24ab1d @nahi do not try to read when a response code is 1XX, 204 or 304. closes #10.
authored
919 def no_message_body?(status)
920 !status.nil? && # HTTP/0.9
921 ((status >= 100 && status < 200) || status == 204 || status == 304)
922 end
923
e3c1331 * a few performance improvements. see #191.
nahi authored
924 def parse_keepalive_header(key, value)
925 key = key.downcase
926 if key == 'content-length'
927 @content_length = value.to_i
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
928 elsif key == 'content-encoding' and ( value.downcase == 'gzip' or
929 value.downcase == 'x-gzip' or value.downcase == 'deflate' )
930 @gzipped = true
e3c1331 * a few performance improvements. see #191.
nahi authored
931 elsif key == 'transfer-encoding' and value.downcase == 'chunked'
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
932 @chunked = true
933 @chunk_length = 0
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
934 @content_length = nil
e3c1331 * a few performance improvements. see #191.
nahi authored
935 elsif key == 'connection' or key == 'proxy-connection'
936 if value.downcase == 'keep-alive'
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
937 @next_connection = true
e3c1331 * a few performance improvements. see #191.
nahi authored
938 else
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
939 @next_connection = false
940 end
941 end
942 end
943
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
944 def read_body_length(&block)
945 return nil if @content_length == 0
946 while true
fe69708 @nahi Do not recycle buffer String object for yielding
authored
947 buf = ''
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
948 maxbytes = @read_block_size
949 maxbytes = @content_length if maxbytes > @content_length
950 timeout(@receive_timeout, ReceiveTimeoutError) do
350acc3 * applied a patch from user. IO#readpartial does not clear the secon…
nahi authored
951 begin
952 @socket.readpartial(maxbytes, buf)
953 rescue EOFError
85105ce @nahi Close session when we get EOF while reading message body
authored
954 close
350acc3 * applied a patch from user. IO#readpartial does not clear the secon…
nahi authored
955 buf = nil
956 end
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
957 end
261b280 @nahi define String#bytesize and use it for 1.9 compatibility. closes #23.
authored
958 if buf && buf.bytesize > 0
959 @content_length -= buf.bytesize
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
960 yield buf
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
961 else
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
962 @content_length = 0
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
963 end
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
964 return if @content_length == 0
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
965 end
966 end
967
968 RS = "\r\n"
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
969 def read_body_chunked(&block)
970 buf = ''
971 while true
972 len = @socket.gets(RS)
85105ce @nahi Close session when we get EOF while reading message body
authored
973 if len.nil? # EOF
974 close
975 return
976 end
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
977 @chunk_length = len.hex
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
978 if @chunk_length == 0
979 @content_length = 0
980 @socket.gets(RS)
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
981 return
982 end
983 timeout(@receive_timeout, ReceiveTimeoutError) do
984 @socket.read(@chunk_length + 2, buf)
985 end
986 unless buf.empty?
987 yield buf.slice(0, @chunk_length)
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
988 end
989 end
990 end
991
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
992 def read_body_rest
261b280 @nahi define String#bytesize and use it for 1.9 compatibility. closes #23.
authored
993 if @readbuf and @readbuf.bytesize > 0
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
994 yield @readbuf
995 @readbuf = nil
996 end
997 while true
fe69708 @nahi Do not recycle buffer String object for yielding
authored
998 buf = ''
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
999 timeout(@receive_timeout, ReceiveTimeoutError) do
350acc3 * applied a patch from user. IO#readpartial does not clear the secon…
nahi authored
1000 begin
1001 @socket.readpartial(@read_block_size, buf)
1002 rescue EOFError
1003 buf = nil
1004 end
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
1005 end
261b280 @nahi define String#bytesize and use it for 1.9 compatibility. closes #23.
authored
1006 if buf && buf.bytesize > 0
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
1007 yield buf
1008 else
1009 return
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
1010 end
1011 end
1012 end
1013 end
1014
1015
1016 end
Something went wrong with that request. Please try again.