Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 1134 lines (1053 sloc) 37.696 kb
b6b5e4f * http-access2 -> httpclient
nahi authored
1 # HTTPClient - HTTP client library.
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
2 # Copyright (C) 2000-2009 NAKAMURA, Hiroshi <nahi@ruby-lang.org>.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
3 #
b6b5e4f * http-access2 -> httpclient
nahi authored
4 # This program is copyrighted free software by NAKAMURA, Hiroshi. You can
5 # redistribute it and/or modify it under the same terms of Ruby's license;
6 # either the dual license version in 2003, or any later version.
7
8
9 require 'uri'
4fc7a2b * added tests for coverage. closes #189.
nahi authored
10 require 'stringio'
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
11 require 'digest/sha1'
b6b5e4f * http-access2 -> httpclient
nahi authored
12
13 # Extra library
a126235 Hiroshi Nakamura Split out VERSION definition to version.rb
authored
14 require 'httpclient/version'
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
15 require 'httpclient/util'
16 require 'httpclient/ssl_config'
17 require 'httpclient/connection'
18 require 'httpclient/session'
b6b5e4f * http-access2 -> httpclient
nahi authored
19 require 'httpclient/http'
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
20 require 'httpclient/auth'
b6b5e4f * http-access2 -> httpclient
nahi authored
21 require 'httpclient/cookie'
22
611d2c0 Thomas de Grivel Added doc for custom multipart
thodg authored
23 # :main:HTTPClient
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
24 # The HTTPClient class provides several methods for accessing Web resources
25 # via HTTP.
26 #
27 # HTTPClient instance is designed to be MT-safe. You can call a HTTPClient
90174d9 * spell check.
nahi authored
28 # instance from several threads without synchronization after setting up an
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
29 # instance.
30 #
31 # clnt = HTTPClient.new
32 # clnt.set_cookie_store('/home/nahi/cookie.dat')
33 # urls.each do |url|
34 # Thread.new(url) do |u|
35 # p clnt.head(u).status
36 # end
37 # end
38 #
39 # == How to use
40 #
41 # At first, how to create your client. See initialize for more detail.
42 #
43 # 1. Create simple client.
b6b5e4f * http-access2 -> httpclient
nahi authored
44 #
45 # clnt = HTTPClient.new
46 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
47 # 2. Accessing resources through HTTP proxy. You can use environment
48 # variable 'http_proxy' or 'HTTP_PROXY' instead.
49 #
50 # clnt = HTTPClient.new('http://myproxy:8080')
51 #
52 # === How to retrieve web resources
53 #
894d9ad Hiroshi Nakamura Update method usage examples.
authored
54 # See get and get_content.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
55 #
894d9ad Hiroshi Nakamura Update method usage examples.
authored
56 # 1. Get content of specified URL. It returns HTTP::Message object and
57 # calling 'body' method of it returns a content String.
58 #
59 # puts clnt.get('http://dev.ctor.org/').body
60 #
61 # 2. For getting content directly, use get_content. It follows redirect
62 # response and returns a String of whole result.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
63 #
64 # puts clnt.get_content('http://dev.ctor.org/')
65 #
894d9ad Hiroshi Nakamura Update method usage examples.
authored
66 # 3. You can pass :follow_redirect option to follow redirect response in get.
67 #
94e696e Jordon Bedwell Fix a small spelling mistake in clnt.get.
envygeeks authored
68 # puts clnt.get('http://dev.ctor.org/', :follow_redirect => true)
894d9ad Hiroshi Nakamura Update method usage examples.
authored
69 #
70 # 4. Get content as chunks of String. It yields chunks of String.
b6b5e4f * http-access2 -> httpclient
nahi authored
71 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
72 # clnt.get_content('http://dev.ctor.org/') do |chunk|
73 # puts chunk
74 # end
b6b5e4f * http-access2 -> httpclient
nahi authored
75 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
76 # === Invoking other HTTP methods
77 #
78 # See head, get, post, put, delete, options, propfind, proppatch and trace.
79 # It returns a HTTP::Message instance as a response.
80 #
81 # 1. Do HEAD request.
b6b5e4f * http-access2 -> httpclient
nahi authored
82 #
83 # res = clnt.head(uri)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
84 # p res.header['Last-Modified'][0]
85 #
86 # 2. Do GET request with query.
87 #
88 # query = { 'keyword' => 'ruby', 'lang' => 'en' }
89 # res = clnt.get(uri, query)
90 # p res.status
91 # p res.contenttype
92 # p res.header['X-Custom']
894d9ad Hiroshi Nakamura Update method usage examples.
authored
93 # puts res.body
94 #
95 # You can also use keyword argument style.
96 #
97 # res = clnt.get(uri, :query => { :keyword => 'ruby', :lang => 'en' })
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
98 #
99 # === How to POST
100 #
101 # See post.
102 #
103 # 1. Do POST a form data.
104 #
105 # body = { 'keyword' => 'ruby', 'lang' => 'en' }
106 # res = clnt.post(uri, body)
107 #
894d9ad Hiroshi Nakamura Update method usage examples.
authored
108 # Keyword argument style.
109 #
110 # res = clnt.post(uri, :body => ...)
111 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
112 # 2. Do multipart file upload with POST. No need to set extra header by
b6b7a20 * preparing 2.1.4 release.
nahi authored
113 # yourself from httpclient/2.1.4.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
114 #
115 # File.open('/tmp/post_data') do |file|
116 # body = { 'upload' => file, 'user' => 'nahi' }
117 # res = clnt.post(uri, body)
118 # end
119 #
611d2c0 Thomas de Grivel Added doc for custom multipart
thodg authored
120 # 3. Do multipart wth custom body.
121 #
122 # File.open('/tmp/post_data') do |file|
123 # body = [{ 'Content-Type' => 'application/atom+xml; charset=UTF-8',
124 # :content => '<entry>...</entry>' },
125 # { 'Content-Type' => 'video/mp4',
126 # 'Content-Transfer-Encoding' => 'binary',
127 # :content => file }]
128 # res = clnt.post(uri, body)
129 # end
130 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
131 # === Accessing via SSL
132 #
133 # Ruby needs to be compiled with OpenSSL.
134 #
135 # 1. Get content of specified URL via SSL.
136 # Just pass an URL which starts with 'https://'.
137 #
138 # https_url = 'https://www.rsa.com'
894d9ad Hiroshi Nakamura Update method usage examples.
authored
139 # clnt.get(https_url)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
140 #
141 # 2. Getting peer certificate from response.
142 #
143 # res = clnt.get(https_url)
144 # p res.peer_cert #=> returns OpenSSL::X509::Certificate
145 #
146 # 3. Configuring OpenSSL options. See HTTPClient::SSLConfig for more details.
147 #
148 # user_cert_file = 'cert.pem'
149 # user_key_file = 'privkey.pem'
150 # clnt.ssl_config.set_client_cert_file(user_cert_file, user_key_file)
894d9ad Hiroshi Nakamura Update method usage examples.
authored
151 # clnt.get(https_url)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
152 #
153 # === Handling Cookies
154 #
155 # 1. Using volatile Cookies. Nothing to do. HTTPClient handles Cookies.
156 #
157 # clnt = HTTPClient.new
894d9ad Hiroshi Nakamura Update method usage examples.
authored
158 # res = clnt.get(url1) # receives Cookies.
159 # res = clnt.get(url2) # sends Cookies if needed.
160 # p res.cookies
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
161 #
162 # 2. Saving non volatile Cookies to a specified file. Need to set a file at
163 # first and invoke save method at last.
164 #
165 # clnt = HTTPClient.new
166 # clnt.set_cookie_store('/home/nahi/cookie.dat')
894d9ad Hiroshi Nakamura Update method usage examples.
authored
167 # clnt.get(url)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
168 # ...
169 # clnt.save_cookie_store
170 #
171 # 3. Disabling Cookies.
172 #
173 # clnt = HTTPClient.new
174 # clnt.cookie_manager = nil
175 #
90174d9 * spell check.
nahi authored
176 # === Configuring authentication credentials
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
177 #
178 # 1. Authentication with Web server. Supports BasicAuth, DigestAuth, and
179 # Negotiate/NTLM (requires ruby/ntlm module).
180 #
181 # clnt = HTTPClient.new
182 # domain = 'http://dev.ctor.org/http-access2/'
183 # user = 'user'
184 # password = 'user'
185 # clnt.set_auth(domain, user, password)
894d9ad Hiroshi Nakamura Update method usage examples.
authored
186 # p clnt.get('http://dev.ctor.org/http-access2/login').status
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
187 #
90174d9 * spell check.
nahi authored
188 # 2. Authentication with Proxy server. Supports BasicAuth and NTLM
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
189 # (requires win32/sspi)
190 #
191 # clnt = HTTPClient.new(proxy)
192 # user = 'proxy'
193 # password = 'proxy'
194 # clnt.set_proxy_auth(user, password)
894d9ad Hiroshi Nakamura Update method usage examples.
authored
195 # p clnt.get(url)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
196 #
197 # === Invoking HTTP methods with custom header
198 #
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
199 # Pass a Hash or an Array for header argument.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
200 #
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
201 # header = { 'Accept' => '*/*' }
894d9ad Hiroshi Nakamura Update method usage examples.
authored
202 # clnt.get(uri, query, header)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
203 #
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
204 # header = [['Accept', 'image/jpeg'], ['Accept', 'image/png']]
205 # clnt.get_content(uri, query, header)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
206 #
207 # === Invoking HTTP methods asynchronously
208 #
209 # See head_async, get_async, post_async, put_async, delete_async,
210 # options_async, propfind_async, proppatch_async, and trace_async.
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
211 # It immediately returns a HTTPClient::Connection instance as a returning value.
212 #
213 # connection = clnt.post_async(url, body)
214 # print 'posting.'
215 # while true
216 # break if connection.finished?
217 # print '.'
218 # sleep 1
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
219 # end
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
220 # puts '.'
221 # res = connection.pop
222 # p res.status
894d9ad Hiroshi Nakamura Update method usage examples.
authored
223 # p res.body.read # res.body is an IO for the res of async method.
b6b5e4f * http-access2 -> httpclient
nahi authored
224 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
225 # === Shortcut methods
226 #
227 # You can invoke get_content, get, etc. without creating HTTPClient instance.
228 #
229 # ruby -rhttpclient -e 'puts HTTPClient.get_content(ARGV.shift)' http://dev.ctor.org/
230 # ruby -rhttpclient -e 'p HTTPClient.head(ARGV.shift).header["last-modified"]' http://dev.ctor.org/
b6b5e4f * http-access2 -> httpclient
nahi authored
231 #
232 class HTTPClient
233 RUBY_VERSION_STRING = "ruby #{RUBY_VERSION} (#{RUBY_RELEASE_DATE}) [#{RUBY_PLATFORM}]"
c4c9e6b Hiroshi Nakamura Set VERSION string in User-Agent header
authored
234 LIB_NAME = "(#{VERSION}, #{RUBY_VERSION_STRING})"
b6b5e4f * http-access2 -> httpclient
nahi authored
235
2253f95 * test coverage checked. see #189.
nahi authored
236 include Util
237
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
238 # Raised for indicating running environment configuration error for example
239 # accessing via SSL under the ruby which is not compiled with OpenSSL.
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
240 class ConfigurationError < StandardError
241 end
242
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
243 # Raised for indicating HTTP response error.
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
244 class BadResponseError < RuntimeError
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
245 # HTTP::Message:: a response
5ddb7c5 * added 'res' attribute to HTTPClient::BadResponse. you can get respon...
nahi authored
246 attr_reader :res
247
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
248 def initialize(msg, res = nil) # :nodoc:
5ddb7c5 * added 'res' attribute to HTTPClient::BadResponse. you can get respon...
nahi authored
249 super(msg)
250 @res = res
251 end
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
252 end
253
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
254 # Raised for indicating a timeout error.
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
255 class TimeoutError < RuntimeError
256 end
257
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
258 # Raised for indicating a connection timeout error.
259 # You can configure connection timeout via HTTPClient#connect_timeout=.
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
260 class ConnectTimeoutError < TimeoutError
261 end
262
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
263 # Raised for indicating a request sending timeout error.
264 # You can configure request sending timeout via HTTPClient#send_timeout=.
265 class SendTimeoutError < TimeoutError
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
266 end
267
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
268 # Raised for indicating a response receiving timeout error.
269 # You can configure response receiving timeout via
270 # HTTPClient#receive_timeout=.
271 class ReceiveTimeoutError < TimeoutError
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
272 end
273
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
274 # Deprecated. just for backward compatibility
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
275 class Session
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
276 BadResponse = ::HTTPClient::BadResponseError
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
277 end
b6b5e4f * http-access2 -> httpclient
nahi authored
278
2253f95 * test coverage checked. see #189.
nahi authored
279 class << self
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
280 %w(get_content post_content head get post put delete options propfind proppatch trace).each do |name|
281 eval <<-EOD
88f1ed2 * ensure to reset connection after invoking HTTPClient singleton method...
nahi authored
282 def #{name}(*arg, &block)
283 clnt = new
284 begin
285 clnt.#{name}(*arg, &block)
286 ensure
287 clnt.reset_all
288 end
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
289 end
290 EOD
291 end
292
293 private
294
2253f95 * test coverage checked. see #189.
nahi authored
295 def attr_proxy(symbol, assignable = false)
296 name = symbol.to_s
297 define_method(name) {
298 @session_manager.__send__(name)
299 }
300 if assignable
301 aname = name + '='
302 define_method(aname) { |rhs|
303 reset_all
304 @session_manager.__send__(aname, rhs)
305 }
306 end
307 end
308 end
309
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
310 # HTTPClient::SSLConfig:: SSL configurator.
b6b5e4f * http-access2 -> httpclient
nahi authored
311 attr_reader :ssl_config
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
312 # WebAgent::CookieManager:: Cookies configurator.
b6b5e4f * http-access2 -> httpclient
nahi authored
313 attr_accessor :cookie_manager
90174d9 * spell check.
nahi authored
314 # An array of response HTTP message body String which is used for loop-back
315 # test. See test/* to see how to use it. If you want to do loop-back test
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
316 # of HTTP header, use test_loopback_http_response instead.
b6b5e4f * http-access2 -> httpclient
nahi authored
317 attr_reader :test_loopback_response
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
318 # An array of request filter which can trap HTTP request/response.
319 # See HTTPClient::WWWAuth to see how to use it.
b6b5e4f * http-access2 -> httpclient
nahi authored
320 attr_reader :request_filter
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
321 # HTTPClient::ProxyAuth:: Proxy authentication handler.
b6b5e4f * http-access2 -> httpclient
nahi authored
322 attr_reader :proxy_auth
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
323 # HTTPClient::WWWAuth:: WWW authentication handler.
b6b5e4f * http-access2 -> httpclient
nahi authored
324 attr_reader :www_auth
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
325 # How many times get_content and post_content follows HTTP redirect.
326 # 10 by default.
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
327 attr_accessor :follow_redirect_count
328
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
329 # Set HTTP version as a String:: 'HTTP/1.0' or 'HTTP/1.1'
2253f95 * test coverage checked. see #189.
nahi authored
330 attr_proxy(:protocol_version, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
331 # Connect timeout in sec.
2253f95 * test coverage checked. see #189.
nahi authored
332 attr_proxy(:connect_timeout, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
333 # Request sending timeout in sec.
2253f95 * test coverage checked. see #189.
nahi authored
334 attr_proxy(:send_timeout, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
335 # Response receiving timeout in sec.
2253f95 * test coverage checked. see #189.
nahi authored
336 attr_proxy(:receive_timeout, true)
08ec3ad Hiroshi Nakamura Reuse cached session in MRU order, not in LRU
authored
337 # Reuse the same connection within this timeout in sec. from last used.
338 attr_proxy(:keep_alive_timeout, true)
fe69708 Hiroshi Nakamura Do not recycle buffer String object for yielding
authored
339 # Size of reading block for non-chunked response.
340 attr_proxy(:read_block_size, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
341 # Negotiation retry count for authentication. 5 by default.
3f2a996 * tested proxy + SSL against squid.
nahi authored
342 attr_proxy(:protocol_retry_count, true)
2253f95 * test coverage checked. see #189.
nahi authored
343 # if your ruby is older than 2005-09-06, do not set socket_sync = false to
344 # avoid an SSL socket blocking bug in openssl/buffering.rb.
345 attr_proxy(:socket_sync, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
346 # User-Agent header in HTTP request.
2253f95 * test coverage checked. see #189.
nahi authored
347 attr_proxy(:agent_name, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
348 # From header in HTTP request.
2253f95 * test coverage checked. see #189.
nahi authored
349 attr_proxy(:from, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
350 # An array of response HTTP String (not a HTTP message body) which is used
351 # for loopback test. See test/* to see how to use it.
2253f95 * test coverage checked. see #189.
nahi authored
352 attr_proxy(:test_loopback_http_response)
95964f1 Hiroshi Nakamura Added transparent_gzip_decompression property. closes #42.
authored
353 # Decompress a compressed (with gzip or deflate) content body transparently. false by default.
354 attr_proxy(:transparent_gzip_decompression, true)
370ac62 Hiroshi Nakamura Add doc for HTTPClient#socket_local.
authored
355 # Local socket address. Set HTTPClient#socket_local.host and HTTPClient#socket_local.port to specify local binding hostname and port of TCP socket.
c9b5f3e Hiroshi Nakamura Rename 'local_bind' to 'socket_local'
authored
356 attr_proxy(:socket_local, true)
b6b5e4f * http-access2 -> httpclient
nahi authored
357
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
358 # Default header for PROPFIND request.
9192e7d * applied propfind/proppatch patch from an user. Thanks!
nahi authored
359 PROPFIND_DEFAULT_EXTHEADER = { 'Depth' => '0' }
360
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
361 # Creates a HTTPClient instance which manages sessions, cookies, etc.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
362 #
363 # HTTPClient.new takes 3 optional arguments for proxy url string,
364 # User-Agent String and From header String. User-Agent and From are embedded
365 # in HTTP request Header if given. No User-Agent and From header added
366 # without setting it explicitly.
b6b5e4f * http-access2 -> httpclient
nahi authored
367 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
368 # proxy = 'http://myproxy:8080'
369 # agent_name = 'MyAgent/0.1'
370 # from = 'from@example.com'
371 # HTTPClient.new(proxy, agent_name, from)
b6b5e4f * http-access2 -> httpclient
nahi authored
372 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
373 # You can use a keyword argument style Hash. Keys are :proxy, :agent_name
374 # and :from.
b6b5e4f * http-access2 -> httpclient
nahi authored
375 #
ab26d05 Hiroshi Nakamura RDoc typo fixed. closes #32.
authored
376 # HTTPClient.new(:agent_name => 'MyAgent/0.1')
2253f95 * test coverage checked. see #189.
nahi authored
377 def initialize(*args)
378 proxy, agent_name, from = keyword_argument(args, :proxy, :agent_name, :from)
b6b5e4f * http-access2 -> httpclient
nahi authored
379 @proxy = nil # assigned later.
380 @no_proxy = nil
381 @www_auth = WWWAuth.new
382 @proxy_auth = ProxyAuth.new
383 @request_filter = [@proxy_auth, @www_auth]
384 @debug_dev = nil
385 @redirect_uri_callback = method(:default_redirect_uri_callback)
386 @test_loopback_response = []
c7688af * support SSL connection via HTTP Proxy which requires Proxy authorizat...
nahi authored
387 @session_manager = SessionManager.new(self)
2253f95 * test coverage checked. see #189.
nahi authored
388 @session_manager.agent_name = agent_name
389 @session_manager.from = from
b6b5e4f * http-access2 -> httpclient
nahi authored
390 @session_manager.ssl_config = @ssl_config = SSLConfig.new(self)
391 @cookie_manager = WebAgent::CookieManager.new
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
392 @follow_redirect_count = 10
27d4279 * recover http_proxy/HTTP_PROXY environment variable support.
nahi authored
393 load_environment
394 self.proxy = proxy if proxy
16ce3df Hiroshi Nakamura restore webmock compatibility.
authored
395 keep_webmock_compat
396 end
397
398 # webmock 1.6.2 depends on HTTP::Message#body.content to work.
399 # let's keep it work iif webmock is loaded for a while.
400 def keep_webmock_compat
401 if respond_to?(:do_get_block_with_webmock)
402 ::HTTP::Message.module_eval do
403 def body
404 def (o = self.content).content
405 self
406 end
407 o
408 end
409 end
410 end
b6b5e4f * http-access2 -> httpclient
nahi authored
411 end
412
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
413 # Returns debug device if exists. See debug_dev=.
b6b5e4f * http-access2 -> httpclient
nahi authored
414 def debug_dev
415 @debug_dev
416 end
417
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
418 # Sets debug device. Once debug device is set, all HTTP requests and
419 # responses are dumped to given device. dev must respond to << for dump.
420 #
421 # Calling this method resets all existing sessions.
b6b5e4f * http-access2 -> httpclient
nahi authored
422 def debug_dev=(dev)
423 @debug_dev = dev
424 reset_all
425 @session_manager.debug_dev = dev
426 end
427
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
428 # Returns URI object of HTTP proxy if exists.
b6b5e4f * http-access2 -> httpclient
nahi authored
429 def proxy
430 @proxy
431 end
432
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
433 # Sets HTTP proxy used for HTTP connection. Given proxy can be an URI,
434 # a String or nil. You can set user/password for proxy authentication like
435 # HTTPClient#proxy = 'http://user:passwd@myproxy:8080'
436 #
437 # You can use environment variable 'http_proxy' or 'HTTP_PROXY' for it.
438 # You need to use 'cgi_http_proxy' or 'CGI_HTTP_PROXY' instead if you run
439 # HTTPClient from CGI environment from security reason. (HTTPClient checks
440 # 'REQUEST_METHOD' environment variable whether it's CGI or not)
441 #
442 # Calling this method resets all existing sessions.
b6b5e4f * http-access2 -> httpclient
nahi authored
443 def proxy=(proxy)
796a731 Hiroshi Nakamura Regression: setting proxy with URI
authored
444 if proxy.nil? || proxy.to_s.empty?
b6b5e4f * http-access2 -> httpclient
nahi authored
445 @proxy = nil
446 @proxy_auth.reset_challenge
447 else
448 @proxy = urify(proxy)
449 if @proxy.scheme == nil or @proxy.scheme.downcase != 'http' or
450 @proxy.host == nil or @proxy.port == nil
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
451 raise ArgumentError.new("unsupported proxy #{proxy}")
b6b5e4f * http-access2 -> httpclient
nahi authored
452 end
453 @proxy_auth.reset_challenge
454 if @proxy.user || @proxy.password
455 @proxy_auth.set_auth(@proxy.user, @proxy.password)
456 end
457 end
458 reset_all
17cbbf9 * refactoring around proxy handling. reduces created Objects. see #19...
nahi authored
459 @session_manager.proxy = @proxy
b6b5e4f * http-access2 -> httpclient
nahi authored
460 @proxy
461 end
462
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
463 # Returns NO_PROXY setting String if given.
b6b5e4f * http-access2 -> httpclient
nahi authored
464 def no_proxy
465 @no_proxy
466 end
467
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
468 # Sets NO_PROXY setting String. no_proxy must be a comma separated String.
469 # Each entry must be 'host' or 'host:port' such as;
470 # HTTPClient#no_proxy = 'example.com,example.co.jp:443'
471 #
472 # 'localhost' is treated as a no_proxy site regardless of explicitly listed.
473 # HTTPClient checks given URI objects before accessing it.
474 # 'host' is tail string match. No IP-addr conversion.
475 #
476 # You can use environment variable 'no_proxy' or 'NO_PROXY' for it.
477 #
478 # Calling this method resets all existing sessions.
b6b5e4f * http-access2 -> httpclient
nahi authored
479 def no_proxy=(no_proxy)
480 @no_proxy = no_proxy
481 reset_all
482 end
483
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
484 # Sets credential for Web server authentication.
485 # domain:: a String or an URI to specify where HTTPClient should use this
486 # credential. If you set uri to nil, HTTPClient uses this credential
487 # wherever a server requires it.
488 # user:: username String.
489 # passwd:: password String.
490 #
491 # You can set multiple credentials for each uri.
492 #
493 # clnt.set_auth('http://www.example.com/foo/', 'foo_user', 'passwd')
494 # clnt.set_auth('http://www.example.com/bar/', 'bar_user', 'passwd')
495 #
496 # Calling this method resets all existing sessions.
497 def set_auth(domain, user, passwd)
498 uri = urify(domain)
b6b5e4f * http-access2 -> httpclient
nahi authored
499 @www_auth.set_auth(uri, user, passwd)
500 reset_all
501 end
502
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
503 # Deprecated. Use set_auth instead.
504 def set_basic_auth(domain, user, passwd)
505 uri = urify(domain)
b6b5e4f * http-access2 -> httpclient
nahi authored
506 @www_auth.basic_auth.set(uri, user, passwd)
507 reset_all
508 end
509
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
510 # Sets credential for Proxy authentication.
511 # user:: username String.
512 # passwd:: password String.
513 #
514 # Calling this method resets all existing sessions.
b6b5e4f * http-access2 -> httpclient
nahi authored
515 def set_proxy_auth(user, passwd)
516 @proxy_auth.set_auth(user, passwd)
517 reset_all
518 end
519
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
520 # Sets the filename where non-volatile Cookies be saved by calling
521 # save_cookie_store.
522 # This method tries to load and managing Cookies from the specified file.
523 #
524 # Calling this method resets all existing sessions.
b6b5e4f * http-access2 -> httpclient
nahi authored
525 def set_cookie_store(filename)
526 @cookie_manager.cookies_file = filename
527 @cookie_manager.load_cookies if filename
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
528 reset_all
b6b5e4f * http-access2 -> httpclient
nahi authored
529 end
530
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
531 # Try to save Cookies to the file specified in set_cookie_store. Unexpected
532 # error will be raised if you don't call set_cookie_store first.
533 # (interface mismatch between WebAgent::CookieManager implementation)
b6b5e4f * http-access2 -> httpclient
nahi authored
534 def save_cookie_store
535 @cookie_manager.save_cookies
536 end
537
3050bde Hiroshi Nakamura Add shortcut methods.
authored
538 # Returns stored cookies.
539 def cookies
540 if @cookie_manager
541 @cookie_manager.cookies
542 end
543 end
544
90174d9 * spell check.
nahi authored
545 # Sets callback proc when HTTP redirect status is returned for get_content
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
546 # and post_content. default_redirect_uri_callback is used by default.
547 #
548 # If you need strict implementation which does not allow relative URI
549 # redirection, set strict_redirect_uri_callback instead.
550 #
551 # clnt.redirect_uri_callback = clnt.method(:strict_redirect_uri_callback)
552 #
b6b5e4f * http-access2 -> httpclient
nahi authored
553 def redirect_uri_callback=(redirect_uri_callback)
554 @redirect_uri_callback = redirect_uri_callback
555 end
556
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
557 # Retrieves a web resource.
b6b5e4f * http-access2 -> httpclient
nahi authored
558 #
90174d9 * spell check.
nahi authored
559 # uri:: a String or an URI object which represents an URL of web resource.
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
560 # query:: a Hash or an Array of query part of URL.
561 # e.g. { "a" => "b" } => 'http://host/part?a=b'.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
562 # Give an array to pass multiple value like
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
563 # [["a", "b"], ["a", "c"]] => 'http://host/part?a=b&a=c'.
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
564 # header:: a Hash or an Array of extra headers. e.g.
565 # { 'Accept' => '*/*' } or
566 # [['Accept', 'image/jpeg'], ['Accept', 'image/png']].
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
567 # &block:: Give a block to get chunked message-body of response like
568 # get_content(uri) { |chunked_body| ... }.
569 # Size of each chunk may not be the same.
b6b5e4f * http-access2 -> httpclient
nahi authored
570 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
571 # get_content follows HTTP redirect status (see HTTP::Status.redirect?)
572 # internally and try to retrieve content from redirected URL. See
573 # redirect_uri_callback= how HTTP redirection is handled.
b6b5e4f * http-access2 -> httpclient
nahi authored
574 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
575 # If you need to get full HTTP response including HTTP status and headers,
576 # use get method. get returns HTTP::Message as a response and you need to
577 # follow HTTP redirect by yourself if you need.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
578 def get_content(uri, *args, &block)
579 query, header = keyword_argument(args, :query, :header)
580 follow_redirect(:get, uri, query, nil, header || {}, &block).content
b6b5e4f * http-access2 -> httpclient
nahi authored
581 end
582
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
583 # Posts a content.
584 #
90174d9 * spell check.
nahi authored
585 # uri:: a String or an URI object which represents an URL of web resource.
611d2c0 Thomas de Grivel Added doc for custom multipart
thodg authored
586 # body:: a Hash or an Array of body part. e.g.
587 # { "a" => "b" } => 'a=b'
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
588 # Give an array to pass multiple value like
611d2c0 Thomas de Grivel Added doc for custom multipart
thodg authored
589 # [["a", "b"], ["a", "c"]] => 'a=b&a=c'
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
590 # When you pass a File as a value, it will be posted as a
611d2c0 Thomas de Grivel Added doc for custom multipart
thodg authored
591 # multipart/form-data. e.g.
592 # { 'upload' => file }
593 # You can also send custom multipart by passing an array of hashes.
594 # Each part must have a :content attribute which can be a file, all
595 # other keys will become headers.
596 # [{ 'Content-Type' => 'text/plain', :content => "some text" },
597 # { 'Content-Type' => 'video/mp4', :content => File.new('video.mp4') }]
598 # => <Two parts with custom Content-Type header>
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
599 # header:: a Hash or an Array of extra headers. e.g.
600 # { 'Accept' => '*/*' }
601 # or
602 # [['Accept', 'image/jpeg'], ['Accept', 'image/png']].
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
603 # &block:: Give a block to get chunked message-body of response like
611d2c0 Thomas de Grivel Added doc for custom multipart
thodg authored
604 # post_content(uri) { |chunked_body| ... }.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
605 # Size of each chunk may not be the same.
606 #
607 # post_content follows HTTP redirect status (see HTTP::Status.redirect?)
608 # internally and try to post the content to redirected URL. See
609 # redirect_uri_callback= how HTTP redirection is handled.
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
610 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
611 # If you need to get full HTTP response including HTTP status and headers,
612 # use post method.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
613 def post_content(uri, *args, &block)
614 body, header = keyword_argument(args, :body, :header)
615 follow_redirect(:post, uri, nil, body, header || {}, &block).content
b6b5e4f * http-access2 -> httpclient
nahi authored
616 end
617
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
618 # A method for redirect uri callback. How to use:
619 # clnt.redirect_uri_callback = clnt.method(:strict_redirect_uri_callback)
620 # This callback does not allow relative redirect such as
621 # Location: ../foo/
622 # in HTTP header. (raises BadResponseError instead)
b6b5e4f * http-access2 -> httpclient
nahi authored
623 def strict_redirect_uri_callback(uri, res)
624 newuri = URI.parse(res.header['location'][0])
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
625 if https?(uri) && !https?(newuri)
626 raise BadResponseError.new("redirecting to non-https resource")
627 end
4fc7a2b * added tests for coverage. closes #189.
nahi authored
628 unless newuri.is_a?(URI::HTTP)
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
629 raise BadResponseError.new("unexpected location: #{newuri}", res)
4fc7a2b * added tests for coverage. closes #189.
nahi authored
630 end
17cbbf9 * refactoring around proxy handling. reduces created Objects. see #19...
nahi authored
631 puts "redirect to: #{newuri}" if $DEBUG
b6b5e4f * http-access2 -> httpclient
nahi authored
632 newuri
633 end
634
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
635 # A default method for redirect uri callback. This method is used by
636 # HTTPClient instance by default.
637 # This callback allows relative redirect such as
638 # Location: ../foo/
639 # in HTTP header.
b6b5e4f * http-access2 -> httpclient
nahi authored
640 def default_redirect_uri_callback(uri, res)
641 newuri = URI.parse(res.header['location'][0])
642 unless newuri.is_a?(URI::HTTP)
643 newuri = uri + newuri
285cf2f Hiroshi Nakamura Supress known warnings during test
authored
644 warn("could be a relative URI in location header which is not recommended")
645 warn("'The field value consists of a single absolute URI' in HTTP spec")
b6b5e4f * http-access2 -> httpclient
nahi authored
646 end
8a94b21 Hiroshi Nakamura Fixed NME for non absolute-URI redirect on https. closes #20.
authored
647 if https?(uri) && !https?(newuri)
648 raise BadResponseError.new("redirecting to non-https resource")
649 end
17cbbf9 * refactoring around proxy handling. reduces created Objects. see #19...
nahi authored
650 puts "redirect to: #{newuri}" if $DEBUG
b6b5e4f * http-access2 -> httpclient
nahi authored
651 newuri
652 end
653
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
654 # Sends HEAD request to the specified URL. See request for arguments.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
655 def head(uri, *args)
29bd2d6 Hiroshi Nakamura Handle argument Hash at new method: argument_to_hash
authored
656 request(:head, uri, argument_to_hash(args, :query, :header))
b6b5e4f * http-access2 -> httpclient
nahi authored
657 end
658
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
659 # Sends GET request to the specified URL. See request for arguments.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
660 def get(uri, *args, &block)
0623a18 Hiroshi Nakamura Add keyword argument :follow_redirect: bool to get/post
authored
661 request(:get, uri, argument_to_hash(args, :query, :header, :follow_redirect), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
662 end
663
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
664 # Sends POST request to the specified URL. See request for arguments.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
665 def post(uri, *args, &block)
0623a18 Hiroshi Nakamura Add keyword argument :follow_redirect: bool to get/post
authored
666 request(:post, uri, argument_to_hash(args, :body, :header, :follow_redirect), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
667 end
668
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
669 # Sends PUT request to the specified URL. See request for arguments.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
670 def put(uri, *args, &block)
29bd2d6 Hiroshi Nakamura Handle argument Hash at new method: argument_to_hash
authored
671 request(:put, uri, argument_to_hash(args, :body, :header), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
672 end
673
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
674 # Sends DELETE request to the specified URL. See request for arguments.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
675 def delete(uri, *args, &block)
29bd2d6 Hiroshi Nakamura Handle argument Hash at new method: argument_to_hash
authored
676 request(:delete, uri, argument_to_hash(args, :header), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
677 end
678
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
679 # Sends OPTIONS request to the specified URL. See request for arguments.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
680 def options(uri, *args, &block)
29bd2d6 Hiroshi Nakamura Handle argument Hash at new method: argument_to_hash
authored
681 request(:options, uri, argument_to_hash(args, :header), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
682 end
683
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
684 # Sends PROPFIND request to the specified URL. See request for arguments.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
685 def propfind(uri, *args, &block)
29bd2d6 Hiroshi Nakamura Handle argument Hash at new method: argument_to_hash
authored
686 request(:propfind, uri, argument_to_hash(args, :header), &block)
9192e7d * applied propfind/proppatch patch from an user. Thanks!
nahi authored
687 end
688
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
689 # Sends PROPPATCH request to the specified URL. See request for arguments.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
690 def proppatch(uri, *args, &block)
29bd2d6 Hiroshi Nakamura Handle argument Hash at new method: argument_to_hash
authored
691 request(:proppatch, uri, argument_to_hash(args, :body, :header), &block)
9192e7d * applied propfind/proppatch patch from an user. Thanks!
nahi authored
692 end
693
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
694 # Sends TRACE request to the specified URL. See request for arguments.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
695 def trace(uri, *args, &block)
29bd2d6 Hiroshi Nakamura Handle argument Hash at new method: argument_to_hash
authored
696 request('TRACE', uri, argument_to_hash(args, :query, :body, :header), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
697 end
698
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
699 # Sends a request to the specified URL.
700 #
701 # method:: HTTP method to be sent. method.to_s.upcase is used.
90174d9 * spell check.
nahi authored
702 # uri:: a String or an URI object which represents an URL of web resource.
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
703 # query:: a Hash or an Array of query part of URL.
704 # e.g. { "a" => "b" } => 'http://host/part?a=b'
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
705 # Give an array to pass multiple value like
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
706 # [["a", "b"], ["a", "c"]] => 'http://host/part?a=b&a=c'
87634ea Thomas de Grivel Clean up doc for request
thodg authored
707 # body:: a Hash or an Array of body part. e.g.
708 # { "a" => "b" }
709 # => 'a=b'
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
710 # Give an array to pass multiple value like
87634ea Thomas de Grivel Clean up doc for request
thodg authored
711 # [["a", "b"], ["a", "c"]]
712 # => 'a=b&a=c'.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
713 # When the given method is 'POST' and the given body contains a file
87634ea Thomas de Grivel Clean up doc for request
thodg authored
714 # as a value, it will be posted as a multipart/form-data. e.g.
715 # { 'upload' => file }
00c7985 Thomas de Grivel Added custom multipart request, with doc.
thodg authored
716 # You can also send custom multipart by passing an array of hashes.
717 # Each part must have a :content attribute which can be a file, all
718 # other keys will become headers.
719 # [{ 'Content-Type' => 'text/plain', :content => "some text" },
720 # { 'Content-Type' => 'video/mp4', :content => File.new('video.mp4') }]
721 # => <Two parts with custom Content-Type header>
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
722 # See HTTP::Message.file? for actual condition of 'a file'.
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
723 # header:: a Hash or an Array of extra headers. e.g.
724 # { 'Accept' => '*/*' } or
725 # [['Accept', 'image/jpeg'], ['Accept', 'image/png']].
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
726 # &block:: Give a block to get chunked message-body of response like
727 # get(uri) { |chunked_body| ... }.
728 # Size of each chunk may not be the same.
729 #
730 # You can also pass a String as a body. HTTPClient just sends a String as
731 # a HTTP request message body.
732 #
733 # When you pass an IO as a body, HTTPClient sends it as a HTTP request with
5d49093 TANABE Ken-ichi Do not send request with Transfer-Encoding: chuncked when IO respond to ...
nabeken authored
734 # chunked encoding (Transfer-Encoding: chunked in HTTP header) if IO does not
735 # respond to :read. Bear in mind that some server application does not support
736 # chunked request. At least cgi.rb does not support it.
07569d5 Hiroshi Nakamura Allow keyword style argument for HTTPClient#get, post, etc.
authored
737 def request(method, uri, *args, &block)
0623a18 Hiroshi Nakamura Add keyword argument :follow_redirect: bool to get/post
authored
738 query, body, header, follow_redirect = keyword_argument(args, :query, :body, :header, :follow_redirect)
29bd2d6 Hiroshi Nakamura Handle argument Hash at new method: argument_to_hash
authored
739 if [:post, :put].include?(method)
740 body ||= ''
741 end
742 if method == :propfind
743 header ||= PROPFIND_DEFAULT_EXTHEADER
744 else
745 header ||= {}
746 end
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
747 uri = urify(uri)
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
748 if block
749 filtered_block = proc { |res, str|
750 block.call(str)
751 }
b6b5e4f * http-access2 -> httpclient
nahi authored
752 end
0623a18 Hiroshi Nakamura Add keyword argument :follow_redirect: bool to get/post
authored
753 if follow_redirect
754 follow_redirect(method, uri, query, body, header, &block)
755 else
756 do_request(method, uri, query, body, header, &filtered_block)
757 end
b6b5e4f * http-access2 -> httpclient
nahi authored
758 end
759
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
760 # Sends HEAD request in async style. See request_async for arguments.
761 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 Hiroshi Nakamura Same keyword arg support for _async methods.
authored
762 def head_async(uri, *args)
763 query, header = keyword_argument(args, :query, :header)
764 request_async(:head, uri, query, nil, header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
765 end
766
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
767 # Sends GET request in async style. See request_async for arguments.
768 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 Hiroshi Nakamura Same keyword arg support for _async methods.
authored
769 def get_async(uri, *args)
770 query, header = keyword_argument(args, :query, :header)
771 request_async(:get, uri, query, nil, header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
772 end
773
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
774 # Sends POST request in async style. See request_async for arguments.
775 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 Hiroshi Nakamura Same keyword arg support for _async methods.
authored
776 def post_async(uri, *args)
777 body, header = keyword_argument(args, :body, :header)
778 request_async(:post, uri, nil, body || '', header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
779 end
780
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
781 # Sends PUT request in async style. See request_async for arguments.
782 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 Hiroshi Nakamura Same keyword arg support for _async methods.
authored
783 def put_async(uri, *args)
784 body, header = keyword_argument(args, :body, :header)
785 request_async(:put, uri, nil, body || '', header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
786 end
787
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
788 # Sends DELETE request in async style. See request_async for arguments.
789 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 Hiroshi Nakamura Same keyword arg support for _async methods.
authored
790 def delete_async(uri, *args)
791 header = keyword_argument(args, :header)
792 request_async(:delete, uri, nil, nil, header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
793 end
794
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
795 # Sends OPTIONS request in async style. See request_async for arguments.
796 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 Hiroshi Nakamura Same keyword arg support for _async methods.
authored
797 def options_async(uri, *args)
798 header = keyword_argument(args, :header)
799 request_async(:options, uri, nil, nil, header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
800 end
801
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
802 # Sends PROPFIND request in async style. See request_async for arguments.
803 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 Hiroshi Nakamura Same keyword arg support for _async methods.
authored
804 def propfind_async(uri, *args)
805 header = keyword_argument(args, :header)
806 request_async(:propfind, uri, nil, nil, header || PROPFIND_DEFAULT_EXTHEADER)
9192e7d * applied propfind/proppatch patch from an user. Thanks!
nahi authored
807 end
808
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
809 # Sends PROPPATCH request in async style. See request_async for arguments.
810 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 Hiroshi Nakamura Same keyword arg support for _async methods.
authored
811 def proppatch_async(uri, *args)
812 body, header = keyword_argument(args, :body, :header)
813 request_async(:proppatch, uri, nil, body, header || {})
9192e7d * applied propfind/proppatch patch from an user. Thanks!
nahi authored
814 end
815
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
816 # Sends TRACE request in async style. See request_async for arguments.
817 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 Hiroshi Nakamura Same keyword arg support for _async methods.
authored
818 def trace_async(uri, *args)
819 query, body, header = keyword_argument(args, :query, :body, :header)
820 request_async(:trace, uri, query, body, header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
821 end
822
90174d9 * spell check.
nahi authored
823 # Sends a request in async style. request method creates new Thread for
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
824 # HTTP connection and returns a HTTPClient::Connection instance immediately.
825 #
826 # Arguments definition is the same as request.
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
827 def request_async(method, uri, query = nil, body = nil, header = {})
b6b5e4f * http-access2 -> httpclient
nahi authored
828 uri = urify(uri)
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
829 do_request_async(method, uri, query, body, header)
b6b5e4f * http-access2 -> httpclient
nahi authored
830 end
831
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
832 # Resets internal session for the given URL. Keep-alive connection for the
833 # site (host-port pair) is disconnected if exists.
b6b5e4f * http-access2 -> httpclient
nahi authored
834 def reset(uri)
835 uri = urify(uri)
836 @session_manager.reset(uri)
837 end
838
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
839 # Resets all of internal sessions. Keep-alive connections are disconnected.
b6b5e4f * http-access2 -> httpclient
nahi authored
840 def reset_all
841 @session_manager.reset_all
842 end
843
844 private
845
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
846 class RetryableResponse < StandardError # :nodoc:
847 end
848
849 class KeepAliveDisconnected < StandardError # :nodoc:
a620b5f Hiroshi Nakamura Invalidate pooled sessions for the same destination when we got a KeepAl...
authored
850 attr_reader :sess
851 def initialize(sess = nil)
852 @sess = sess
853 end
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
854 end
855
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
856 def do_request(method, uri, query, body, header, &block)
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
857 conn = Connection.new
858 res = nil
cf55b01 * Cookie header is not set in authentication negotiation. do not reuse ...
nahi authored
859 if HTTP::Message.file?(body)
860 pos = body.pos rescue nil
861 end
3f2a996 * tested proxy + SSL against squid.
nahi authored
862 retry_count = @session_manager.protocol_retry_count
4d3c829 * refactoring: push down create_request invocation into do_request/do_r...
nahi authored
863 proxy = no_proxy?(uri) ? nil : @proxy
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
864 while retry_count > 0
cf55b01 * Cookie header is not set in authentication negotiation. do not reuse ...
nahi authored
865 body.pos = pos if pos
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
866 req = create_request(method, uri, query, body, header)
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
867 begin
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
868 protect_keep_alive_disconnected do
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
869 do_get_block(req, proxy, conn, &block)
870 end
871 res = conn.pop
872 break
873 rescue RetryableResponse
874 res = conn.pop
875 retry_count -= 1
876 end
877 end
878 res
879 end
880
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
881 def do_request_async(method, uri, query, body, header)
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
882 conn = Connection.new
883 t = Thread.new(conn) { |tconn|
1986c59 Hiroshi Nakamura Applied the patch in #44: let Connection#pop raise the Exception when th...
authored
884 begin
885 if HTTP::Message.file?(body)
886 pos = body.pos rescue nil
887 end
888 retry_count = @session_manager.protocol_retry_count
889 proxy = no_proxy?(uri) ? nil : @proxy
890 while retry_count > 0
891 body.pos = pos if pos
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
892 req = create_request(method, uri, query, body, header)
1986c59 Hiroshi Nakamura Applied the patch in #44: let Connection#pop raise the Exception when th...
authored
893 begin
894 protect_keep_alive_disconnected do
895 do_get_stream(req, proxy, tconn)
896 end
897 break
898 rescue RetryableResponse
899 retry_count -= 1
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
900 end
901 end
8aa3a34 Hiroshi Nakamura Make sure to rescue an Exception. closes #44.
authored
902 rescue Exception
1986c59 Hiroshi Nakamura Applied the patch in #44: let Connection#pop raise the Exception when th...
authored
903 conn.push $!
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
904 end
905 }
906 conn.async_thread = t
907 conn
908 end
909
27d4279 * recover http_proxy/HTTP_PROXY environment variable support.
nahi authored
910 def load_environment
911 # http_proxy
912 if getenv('REQUEST_METHOD')
913 # HTTP_PROXY conflicts with the environment variable usage in CGI where
914 # HTTP_* is used for HTTP header information. Unlike open-uri, we
90174d9 * spell check.
nahi authored
915 # simply ignore http_proxy in CGI env and use cgi_http_proxy instead.
27d4279 * recover http_proxy/HTTP_PROXY environment variable support.
nahi authored
916 self.proxy = getenv('cgi_http_proxy')
917 else
918 self.proxy = getenv('http_proxy')
919 end
920 # no_proxy
921 self.no_proxy = getenv('no_proxy')
922 end
923
924 def getenv(name)
925 ENV[name.downcase] || ENV[name.upcase]
926 end
927
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
928 def follow_redirect(method, uri, query, body, header, &block)
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
929 uri = urify(uri)
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
930 if block
e3c1331 * a few performance improvements. see #191.
nahi authored
931 filtered_block = proc { |r, str|
932 block.call(str) if HTTP::Status.successful?(r.status)
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
933 }
934 end
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
935 if HTTP::Message.file?(body)
936 pos = body.pos rescue nil
937 end
938 retry_number = 0
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
939 while retry_number < @follow_redirect_count
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
940 body.pos = pos if pos
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
941 res = do_request(method, uri, query, body, header, &filtered_block)
b6b5e4f * http-access2 -> httpclient
nahi authored
942 if HTTP::Status.successful?(res.status)
943 return res
944 elsif HTTP::Status.redirect?(res.status)
4d3c829 * refactoring: push down create_request invocation into do_request/do_r...
nahi authored
945 uri = urify(@redirect_uri_callback.call(uri, res))
b6b5e4f * http-access2 -> httpclient
nahi authored
946 retry_number += 1
947 else
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
948 raise BadResponseError.new("unexpected response: #{res.header.inspect}", res)
b6b5e4f * http-access2 -> httpclient
nahi authored
949 end
950 end
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
951 raise BadResponseError.new("retry count exceeded", res)
b6b5e4f * http-access2 -> httpclient
nahi authored
952 end
953
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
954 def protect_keep_alive_disconnected
b6b5e4f * http-access2 -> httpclient
nahi authored
955 begin
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
956 yield
a620b5f Hiroshi Nakamura Invalidate pooled sessions for the same destination when we got a KeepAl...
authored
957 rescue KeepAliveDisconnected => e
958 if e.sess
959 @session_manager.invalidate(e.sess.dest)
960 end
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
961 yield
b6b5e4f * http-access2 -> httpclient
nahi authored
962 end
963 end
964
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
965 def create_request(method, uri, query, body, header)
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
966 method = method.to_s.upcase
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
967 if header.is_a?(Hash)
968 header = header.to_a
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
969 else
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
970 header = header.dup
b6b5e4f * http-access2 -> httpclient
nahi authored
971 end
972 boundary = nil
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
973 if body
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
974 _, content_type = header.find { |key, value|
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
975 key.downcase == 'content-type'
976 }
977 if content_type
978 if /\Amultipart/ =~ content_type
979 if content_type =~ /boundary=(.+)\z/
980 boundary = $1
981 else
982 boundary = create_boundary
983 content_type = "#{content_type}; boundary=#{boundary}"
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
984 header = override_header(header, 'Content-Type', content_type)
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
985 end
986 end
987 elsif method == 'POST'
988 if file_in_form_data?(body)
989 boundary = create_boundary
990 content_type = "multipart/form-data; boundary=#{boundary}"
991 else
992 content_type = 'application/x-www-form-urlencoded'
993 end
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
994 header << ['Content-Type', content_type]
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
995 end
b6b5e4f * http-access2 -> httpclient
nahi authored
996 end
17cbbf9 * refactoring around proxy handling. reduces created Objects. see #19...
nahi authored
997 req = HTTP::Message.new_request(method, uri, query, body, boundary)
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
998 header.each do |key, value|
83dc6ec * avoid duplicated request header line while HTTP access negotiation (r...
nahi authored
999 req.header.add(key, value)
b6b5e4f * http-access2 -> httpclient
nahi authored
1000 end
83dc6ec * avoid duplicated request header line while HTTP access negotiation (r...
nahi authored
1001 if @cookie_manager && cookie = @cookie_manager.find(uri)
1002 req.header.add('Cookie', cookie)
fad6e92 * post an IO body on the second request as well as the first request. ...
nahi authored
1003 end
b6b5e4f * http-access2 -> httpclient
nahi authored
1004 req
1005 end
1006
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
1007 def create_boundary
1008 Digest::SHA1.hexdigest(Time.now.to_s)
1009 end
1010
1011 def file_in_form_data?(body)
2253f95 * test coverage checked. see #189.
nahi authored
1012 HTTP::Message.multiparam_query?(body) &&
1013 body.any? { |k, v| HTTP::Message.file?(v) }
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
1014 end
1015
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
1016 def override_header(header, key, value)
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
1017 result = []
25afc79 Hiroshi Nakamura Change variable name: extheader -> header.
authored
1018 header.each do |k, v|
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the re...
nahi authored
1019 if k.downcase == key.downcase
1020 result << [key, value]
1021 else
1022 result << [k, v]
1023 end
1024 end
1025 result
1026 end
1027
b6b5e4f * http-access2 -> httpclient
nahi authored
1028 NO_PROXY_HOSTS = ['localhost']
1029
1030 def no_proxy?(uri)
1031 if !@proxy or NO_PROXY_HOSTS.include?(uri.host)
1032 return true
1033 end
1034 unless @no_proxy
1035 return false
1036 end
1037 @no_proxy.scan(/([^:,]+)(?::(\d+))?/) do |host, port|
1038 if /(\A|\.)#{Regexp.quote(host)}\z/i =~ uri.host &&
1039 (!port || uri.port == port.to_i)
1040 return true
1041 end
1042 end
1043 false
1044 end
1045
1046 # !! CAUTION !!
1047 # Method 'do_get*' runs under MT conditon. Be careful to change.
1048 def do_get_block(req, proxy, conn, &block)
1049 @request_filter.each do |filter|
1050 filter.filter_request(req)
1051 end
1052 if str = @test_loopback_response.shift
3050bde Hiroshi Nakamura Add shortcut methods.
authored
1053 dump_dummy_request_response(req.http_body.dump, str) if @debug_dev
b6b5e4f * http-access2 -> httpclient
nahi authored
1054 conn.push(HTTP::Message.new_response(str))
1055 return
1056 end
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
1057 content = block ? nil : ''
b6b5e4f * http-access2 -> httpclient
nahi authored
1058 res = HTTP::Message.new_response(content)
1059 @debug_dev << "= Request\n\n" if @debug_dev
1060 sess = @session_manager.query(req, proxy)
1061 res.peer_cert = sess.ssl_peer_cert
1062 @debug_dev << "\n\n= Response\n\n" if @debug_dev
1063 do_get_header(req, res, sess)
1064 conn.push(res)
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
1065 sess.get_body do |part|
e5efea5 Hiroshi Nakamura Set response String encoding
authored
1066 set_encoding(part, res.body_encoding)
19a2f15 * reduce memory usage when calling HTTPClient#get with a block. applie...
nahi authored
1067 if block
c9d999f * get_content with block yielded unexpected message body. closes #173.
nahi authored
1068 block.call(res, part)
19a2f15 * reduce memory usage when calling HTTPClient#get with a block. applie...
nahi authored
1069 else
1070 content << part
1071 end
b6b5e4f * http-access2 -> httpclient
nahi authored
1072 end
9ada537 Hiroshi Nakamura Make Keep-Alive test use threads properly.
authored
1073 # there could be a race condition but it's OK to cache unreusable
1074 # connection because we do retry for that case.
b6b5e4f * http-access2 -> httpclient
nahi authored
1075 @session_manager.keep(sess) unless sess.closed?
1076 commands = @request_filter.collect { |filter|
1077 filter.filter_response(req, res)
1078 }
1079 if commands.find { |command| command == :retry }
1080 raise RetryableResponse.new
1081 end
1082 end
1083
1084 def do_get_stream(req, proxy, conn)
1085 @request_filter.each do |filter|
1086 filter.filter_request(req)
1087 end
1088 if str = @test_loopback_response.shift
3050bde Hiroshi Nakamura Add shortcut methods.
authored
1089 dump_dummy_request_response(req.http_body.dump, str) if @debug_dev
4fc7a2b * added tests for coverage. closes #189.
nahi authored
1090 conn.push(HTTP::Message.new_response(StringIO.new(str)))
b6b5e4f * http-access2 -> httpclient
nahi authored
1091 return
1092 end
1093 piper, pipew = IO.pipe
1094 res = HTTP::Message.new_response(piper)
1095 @debug_dev << "= Request\n\n" if @debug_dev
1096 sess = @session_manager.query(req, proxy)
1097 res.peer_cert = sess.ssl_peer_cert
1098 @debug_dev << "\n\n= Response\n\n" if @debug_dev
1099 do_get_header(req, res, sess)
1100 conn.push(res)
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
1101 sess.get_body do |part|
e5efea5 Hiroshi Nakamura Set response String encoding
authored
1102 set_encoding(part, res.body_encoding)
0b7e801 Hiroshi Nakamura avoid data corruption bug in async method.
authored
1103 pipew.write(part)
b6b5e4f * http-access2 -> httpclient
nahi authored
1104 end
1105 pipew.close
1106 @session_manager.keep(sess) unless sess.closed?
7c8ebd1 Hiroshi Nakamura Avoid unused variable warnings from 1.9. closes #24.
authored
1107 _ = @request_filter.collect { |filter|
b6b5e4f * http-access2 -> httpclient
nahi authored
1108 filter.filter_response(req, res)
1109 }
1110 # ignore commands (not retryable in async mode)
1111 end
1112
1113 def do_get_header(req, res, sess)
1949938 Hiroshi Nakamura Do not depend on Float#to_s behavior. closes #36.
authored
1114 res.http_version, res.status, res.reason, headers = sess.get_header
e5efea5 Hiroshi Nakamura Set response String encoding
authored
1115 res.header.set_headers(headers)
17cbbf9 * refactoring around proxy handling. reduces created Objects. see #19...
nahi authored
1116 if @cookie_manager
b6b5e4f * http-access2 -> httpclient
nahi authored
1117 res.header['set-cookie'].each do |cookie|
1118 @cookie_manager.parse(cookie, req.header.request_uri)
1119 end
1120 end
1121 end
1122
1123 def dump_dummy_request_response(req, res)
1124 @debug_dev << "= Dummy Request\n\n"
1125 @debug_dev << req
1126 @debug_dev << "\n\n= Dummy Response\n\n"
1127 @debug_dev << res
1128 end
e5efea5 Hiroshi Nakamura Set response String encoding
authored
1129
1130 def set_encoding(str, encoding)
1131 str.force_encoding(encoding) if encoding
1132 end
b6b5e4f * http-access2 -> httpclient
nahi authored
1133 end
Something went wrong with that request. Please try again.