Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 1134 lines (1053 sloc) 37.696 kB
b6b5e4f * http-access2 -> httpclient
nahi authored
1 # HTTPClient - HTTP client library.
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
2 # Copyright (C) 2000-2009 NAKAMURA, Hiroshi <nahi@ruby-lang.org>.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
3 #
b6b5e4f * http-access2 -> httpclient
nahi authored
4 # This program is copyrighted free software by NAKAMURA, Hiroshi. You can
5 # redistribute it and/or modify it under the same terms of Ruby's license;
6 # either the dual license version in 2003, or any later version.
7
8
9 require 'uri'
4fc7a2b * added tests for coverage. closes #189.
nahi authored
10 require 'stringio'
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
11 require 'digest/sha1'
b6b5e4f * http-access2 -> httpclient
nahi authored
12
13 # Extra library
a126235 @nahi Split out VERSION definition to version.rb
authored
14 require 'httpclient/version'
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
15 require 'httpclient/util'
16 require 'httpclient/ssl_config'
17 require 'httpclient/connection'
18 require 'httpclient/session'
b6b5e4f * http-access2 -> httpclient
nahi authored
19 require 'httpclient/http'
6626924 * split httpclient.rb into several files at httpclient/*.rb. see #162.
nahi authored
20 require 'httpclient/auth'
b6b5e4f * http-access2 -> httpclient
nahi authored
21 require 'httpclient/cookie'
22
611d2c0 @thodg Added doc for custom multipart
thodg authored
23 # :main:HTTPClient
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
24 # The HTTPClient class provides several methods for accessing Web resources
25 # via HTTP.
26 #
27 # HTTPClient instance is designed to be MT-safe. You can call a HTTPClient
90174d9 * spell check.
nahi authored
28 # instance from several threads without synchronization after setting up an
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
29 # instance.
30 #
31 # clnt = HTTPClient.new
32 # clnt.set_cookie_store('/home/nahi/cookie.dat')
33 # urls.each do |url|
34 # Thread.new(url) do |u|
35 # p clnt.head(u).status
36 # end
37 # end
38 #
39 # == How to use
40 #
41 # At first, how to create your client. See initialize for more detail.
42 #
43 # 1. Create simple client.
b6b5e4f * http-access2 -> httpclient
nahi authored
44 #
45 # clnt = HTTPClient.new
46 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
47 # 2. Accessing resources through HTTP proxy. You can use environment
48 # variable 'http_proxy' or 'HTTP_PROXY' instead.
49 #
50 # clnt = HTTPClient.new('http://myproxy:8080')
51 #
52 # === How to retrieve web resources
53 #
894d9ad @nahi Update method usage examples.
authored
54 # See get and get_content.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
55 #
894d9ad @nahi Update method usage examples.
authored
56 # 1. Get content of specified URL. It returns HTTP::Message object and
57 # calling 'body' method of it returns a content String.
58 #
59 # puts clnt.get('http://dev.ctor.org/').body
60 #
61 # 2. For getting content directly, use get_content. It follows redirect
62 # response and returns a String of whole result.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
63 #
64 # puts clnt.get_content('http://dev.ctor.org/')
65 #
894d9ad @nahi Update method usage examples.
authored
66 # 3. You can pass :follow_redirect option to follow redirect response in get.
67 #
94e696e @envygeeks Fix a small spelling mistake in clnt.get.
envygeeks authored
68 # puts clnt.get('http://dev.ctor.org/', :follow_redirect => true)
894d9ad @nahi Update method usage examples.
authored
69 #
70 # 4. Get content as chunks of String. It yields chunks of String.
b6b5e4f * http-access2 -> httpclient
nahi authored
71 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
72 # clnt.get_content('http://dev.ctor.org/') do |chunk|
73 # puts chunk
74 # end
b6b5e4f * http-access2 -> httpclient
nahi authored
75 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
76 # === Invoking other HTTP methods
77 #
78 # See head, get, post, put, delete, options, propfind, proppatch and trace.
79 # It returns a HTTP::Message instance as a response.
80 #
81 # 1. Do HEAD request.
b6b5e4f * http-access2 -> httpclient
nahi authored
82 #
83 # res = clnt.head(uri)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
84 # p res.header['Last-Modified'][0]
85 #
86 # 2. Do GET request with query.
87 #
88 # query = { 'keyword' => 'ruby', 'lang' => 'en' }
89 # res = clnt.get(uri, query)
90 # p res.status
91 # p res.contenttype
92 # p res.header['X-Custom']
894d9ad @nahi Update method usage examples.
authored
93 # puts res.body
94 #
95 # You can also use keyword argument style.
96 #
97 # res = clnt.get(uri, :query => { :keyword => 'ruby', :lang => 'en' })
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
98 #
99 # === How to POST
100 #
101 # See post.
102 #
103 # 1. Do POST a form data.
104 #
105 # body = { 'keyword' => 'ruby', 'lang' => 'en' }
106 # res = clnt.post(uri, body)
107 #
894d9ad @nahi Update method usage examples.
authored
108 # Keyword argument style.
109 #
110 # res = clnt.post(uri, :body => ...)
111 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
112 # 2. Do multipart file upload with POST. No need to set extra header by
b6b7a20 * preparing 2.1.4 release.
nahi authored
113 # yourself from httpclient/2.1.4.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
114 #
115 # File.open('/tmp/post_data') do |file|
116 # body = { 'upload' => file, 'user' => 'nahi' }
117 # res = clnt.post(uri, body)
118 # end
119 #
611d2c0 @thodg Added doc for custom multipart
thodg authored
120 # 3. Do multipart wth custom body.
121 #
122 # File.open('/tmp/post_data') do |file|
123 # body = [{ 'Content-Type' => 'application/atom+xml; charset=UTF-8',
124 # :content => '<entry>...</entry>' },
125 # { 'Content-Type' => 'video/mp4',
126 # 'Content-Transfer-Encoding' => 'binary',
127 # :content => file }]
128 # res = clnt.post(uri, body)
129 # end
130 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
131 # === Accessing via SSL
132 #
133 # Ruby needs to be compiled with OpenSSL.
134 #
135 # 1. Get content of specified URL via SSL.
136 # Just pass an URL which starts with 'https://'.
137 #
138 # https_url = 'https://www.rsa.com'
894d9ad @nahi Update method usage examples.
authored
139 # clnt.get(https_url)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
140 #
141 # 2. Getting peer certificate from response.
142 #
143 # res = clnt.get(https_url)
144 # p res.peer_cert #=> returns OpenSSL::X509::Certificate
145 #
146 # 3. Configuring OpenSSL options. See HTTPClient::SSLConfig for more details.
147 #
148 # user_cert_file = 'cert.pem'
149 # user_key_file = 'privkey.pem'
150 # clnt.ssl_config.set_client_cert_file(user_cert_file, user_key_file)
894d9ad @nahi Update method usage examples.
authored
151 # clnt.get(https_url)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
152 #
153 # === Handling Cookies
154 #
155 # 1. Using volatile Cookies. Nothing to do. HTTPClient handles Cookies.
156 #
157 # clnt = HTTPClient.new
894d9ad @nahi Update method usage examples.
authored
158 # res = clnt.get(url1) # receives Cookies.
159 # res = clnt.get(url2) # sends Cookies if needed.
160 # p res.cookies
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
161 #
162 # 2. Saving non volatile Cookies to a specified file. Need to set a file at
163 # first and invoke save method at last.
164 #
165 # clnt = HTTPClient.new
166 # clnt.set_cookie_store('/home/nahi/cookie.dat')
894d9ad @nahi Update method usage examples.
authored
167 # clnt.get(url)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
168 # ...
169 # clnt.save_cookie_store
170 #
171 # 3. Disabling Cookies.
172 #
173 # clnt = HTTPClient.new
174 # clnt.cookie_manager = nil
175 #
90174d9 * spell check.
nahi authored
176 # === Configuring authentication credentials
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
177 #
178 # 1. Authentication with Web server. Supports BasicAuth, DigestAuth, and
179 # Negotiate/NTLM (requires ruby/ntlm module).
180 #
181 # clnt = HTTPClient.new
182 # domain = 'http://dev.ctor.org/http-access2/'
183 # user = 'user'
184 # password = 'user'
185 # clnt.set_auth(domain, user, password)
894d9ad @nahi Update method usage examples.
authored
186 # p clnt.get('http://dev.ctor.org/http-access2/login').status
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
187 #
90174d9 * spell check.
nahi authored
188 # 2. Authentication with Proxy server. Supports BasicAuth and NTLM
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
189 # (requires win32/sspi)
190 #
191 # clnt = HTTPClient.new(proxy)
192 # user = 'proxy'
193 # password = 'proxy'
194 # clnt.set_proxy_auth(user, password)
894d9ad @nahi Update method usage examples.
authored
195 # p clnt.get(url)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
196 #
197 # === Invoking HTTP methods with custom header
198 #
25afc79 @nahi Change variable name: extheader -> header.
authored
199 # Pass a Hash or an Array for header argument.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
200 #
25afc79 @nahi Change variable name: extheader -> header.
authored
201 # header = { 'Accept' => '*/*' }
894d9ad @nahi Update method usage examples.
authored
202 # clnt.get(uri, query, header)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
203 #
25afc79 @nahi Change variable name: extheader -> header.
authored
204 # header = [['Accept', 'image/jpeg'], ['Accept', 'image/png']]
205 # clnt.get_content(uri, query, header)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
206 #
207 # === Invoking HTTP methods asynchronously
208 #
209 # See head_async, get_async, post_async, put_async, delete_async,
210 # options_async, propfind_async, proppatch_async, and trace_async.
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
211 # It immediately returns a HTTPClient::Connection instance as a returning value.
212 #
213 # connection = clnt.post_async(url, body)
214 # print 'posting.'
215 # while true
216 # break if connection.finished?
217 # print '.'
218 # sleep 1
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
219 # end
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
220 # puts '.'
221 # res = connection.pop
222 # p res.status
894d9ad @nahi Update method usage examples.
authored
223 # p res.body.read # res.body is an IO for the res of async method.
b6b5e4f * http-access2 -> httpclient
nahi authored
224 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
225 # === Shortcut methods
226 #
227 # You can invoke get_content, get, etc. without creating HTTPClient instance.
228 #
229 # ruby -rhttpclient -e 'puts HTTPClient.get_content(ARGV.shift)' http://dev.ctor.org/
230 # ruby -rhttpclient -e 'p HTTPClient.head(ARGV.shift).header["last-modified"]' http://dev.ctor.org/
b6b5e4f * http-access2 -> httpclient
nahi authored
231 #
232 class HTTPClient
233 RUBY_VERSION_STRING = "ruby #{RUBY_VERSION} (#{RUBY_RELEASE_DATE}) [#{RUBY_PLATFORM}]"
c4c9e6b @nahi Set VERSION string in User-Agent header
authored
234 LIB_NAME = "(#{VERSION}, #{RUBY_VERSION_STRING})"
b6b5e4f * http-access2 -> httpclient
nahi authored
235
2253f95 * test coverage checked. see #189.
nahi authored
236 include Util
237
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
238 # Raised for indicating running environment configuration error for example
239 # accessing via SSL under the ruby which is not compiled with OpenSSL.
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
240 class ConfigurationError < StandardError
241 end
242
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
243 # Raised for indicating HTTP response error.
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
244 class BadResponseError < RuntimeError
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
245 # HTTP::Message:: a response
5ddb7c5 * added 'res' attribute to HTTPClient::BadResponse. you can get res…
nahi authored
246 attr_reader :res
247
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
248 def initialize(msg, res = nil) # :nodoc:
5ddb7c5 * added 'res' attribute to HTTPClient::BadResponse. you can get res…
nahi authored
249 super(msg)
250 @res = res
251 end
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
252 end
253
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
254 # Raised for indicating a timeout error.
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
255 class TimeoutError < RuntimeError
256 end
257
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
258 # Raised for indicating a connection timeout error.
259 # You can configure connection timeout via HTTPClient#connect_timeout=.
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
260 class ConnectTimeoutError < TimeoutError
261 end
262
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
263 # Raised for indicating a request sending timeout error.
264 # You can configure request sending timeout via HTTPClient#send_timeout=.
265 class SendTimeoutError < TimeoutError
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
266 end
267
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
268 # Raised for indicating a response receiving timeout error.
269 # You can configure response receiving timeout via
270 # HTTPClient#receive_timeout=.
271 class ReceiveTimeoutError < TimeoutError
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
272 end
273
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
274 # Deprecated. just for backward compatibility
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
275 class Session
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
276 BadResponse = ::HTTPClient::BadResponseError
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
277 end
b6b5e4f * http-access2 -> httpclient
nahi authored
278
2253f95 * test coverage checked. see #189.
nahi authored
279 class << self
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
280 %w(get_content post_content head get post put delete options propfind proppatch trace).each do |name|
281 eval <<-EOD
88f1ed2 * ensure to reset connection after invoking HTTPClient singleton met…
nahi authored
282 def #{name}(*arg, &block)
283 clnt = new
284 begin
285 clnt.#{name}(*arg, &block)
286 ensure
287 clnt.reset_all
288 end
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
289 end
290 EOD
291 end
292
293 private
294
2253f95 * test coverage checked. see #189.
nahi authored
295 def attr_proxy(symbol, assignable = false)
296 name = symbol.to_s
297 define_method(name) {
298 @session_manager.__send__(name)
299 }
300 if assignable
301 aname = name + '='
302 define_method(aname) { |rhs|
303 reset_all
304 @session_manager.__send__(aname, rhs)
305 }
306 end
307 end
308 end
309
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
310 # HTTPClient::SSLConfig:: SSL configurator.
b6b5e4f * http-access2 -> httpclient
nahi authored
311 attr_reader :ssl_config
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
312 # WebAgent::CookieManager:: Cookies configurator.
b6b5e4f * http-access2 -> httpclient
nahi authored
313 attr_accessor :cookie_manager
90174d9 * spell check.
nahi authored
314 # An array of response HTTP message body String which is used for loop-back
315 # test. See test/* to see how to use it. If you want to do loop-back test
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
316 # of HTTP header, use test_loopback_http_response instead.
b6b5e4f * http-access2 -> httpclient
nahi authored
317 attr_reader :test_loopback_response
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
318 # An array of request filter which can trap HTTP request/response.
319 # See HTTPClient::WWWAuth to see how to use it.
b6b5e4f * http-access2 -> httpclient
nahi authored
320 attr_reader :request_filter
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
321 # HTTPClient::ProxyAuth:: Proxy authentication handler.
b6b5e4f * http-access2 -> httpclient
nahi authored
322 attr_reader :proxy_auth
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
323 # HTTPClient::WWWAuth:: WWW authentication handler.
b6b5e4f * http-access2 -> httpclient
nahi authored
324 attr_reader :www_auth
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
325 # How many times get_content and post_content follows HTTP redirect.
326 # 10 by default.
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
327 attr_accessor :follow_redirect_count
328
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
329 # Set HTTP version as a String:: 'HTTP/1.0' or 'HTTP/1.1'
2253f95 * test coverage checked. see #189.
nahi authored
330 attr_proxy(:protocol_version, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
331 # Connect timeout in sec.
2253f95 * test coverage checked. see #189.
nahi authored
332 attr_proxy(:connect_timeout, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
333 # Request sending timeout in sec.
2253f95 * test coverage checked. see #189.
nahi authored
334 attr_proxy(:send_timeout, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
335 # Response receiving timeout in sec.
2253f95 * test coverage checked. see #189.
nahi authored
336 attr_proxy(:receive_timeout, true)
08ec3ad @nahi Reuse cached session in MRU order, not in LRU
authored
337 # Reuse the same connection within this timeout in sec. from last used.
338 attr_proxy(:keep_alive_timeout, true)
fe69708 @nahi Do not recycle buffer String object for yielding
authored
339 # Size of reading block for non-chunked response.
340 attr_proxy(:read_block_size, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
341 # Negotiation retry count for authentication. 5 by default.
3f2a996 * tested proxy + SSL against squid.
nahi authored
342 attr_proxy(:protocol_retry_count, true)
2253f95 * test coverage checked. see #189.
nahi authored
343 # if your ruby is older than 2005-09-06, do not set socket_sync = false to
344 # avoid an SSL socket blocking bug in openssl/buffering.rb.
345 attr_proxy(:socket_sync, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
346 # User-Agent header in HTTP request.
2253f95 * test coverage checked. see #189.
nahi authored
347 attr_proxy(:agent_name, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
348 # From header in HTTP request.
2253f95 * test coverage checked. see #189.
nahi authored
349 attr_proxy(:from, true)
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
350 # An array of response HTTP String (not a HTTP message body) which is used
351 # for loopback test. See test/* to see how to use it.
2253f95 * test coverage checked. see #189.
nahi authored
352 attr_proxy(:test_loopback_http_response)
95964f1 @nahi Added transparent_gzip_decompression property. closes #42.
authored
353 # Decompress a compressed (with gzip or deflate) content body transparently. false by default.
354 attr_proxy(:transparent_gzip_decompression, true)
370ac62 @nahi Add doc for HTTPClient#socket_local.
authored
355 # Local socket address. Set HTTPClient#socket_local.host and HTTPClient#socket_local.port to specify local binding hostname and port of TCP socket.
c9b5f3e @nahi Rename 'local_bind' to 'socket_local'
authored
356 attr_proxy(:socket_local, true)
b6b5e4f * http-access2 -> httpclient
nahi authored
357
25afc79 @nahi Change variable name: extheader -> header.
authored
358 # Default header for PROPFIND request.
9192e7d * applied propfind/proppatch patch from an user. Thanks!
nahi authored
359 PROPFIND_DEFAULT_EXTHEADER = { 'Depth' => '0' }
360
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
361 # Creates a HTTPClient instance which manages sessions, cookies, etc.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
362 #
363 # HTTPClient.new takes 3 optional arguments for proxy url string,
364 # User-Agent String and From header String. User-Agent and From are embedded
365 # in HTTP request Header if given. No User-Agent and From header added
366 # without setting it explicitly.
b6b5e4f * http-access2 -> httpclient
nahi authored
367 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
368 # proxy = 'http://myproxy:8080'
369 # agent_name = 'MyAgent/0.1'
370 # from = 'from@example.com'
371 # HTTPClient.new(proxy, agent_name, from)
b6b5e4f * http-access2 -> httpclient
nahi authored
372 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
373 # You can use a keyword argument style Hash. Keys are :proxy, :agent_name
374 # and :from.
b6b5e4f * http-access2 -> httpclient
nahi authored
375 #
ab26d05 @nahi RDoc typo fixed. closes #32.
authored
376 # HTTPClient.new(:agent_name => 'MyAgent/0.1')
2253f95 * test coverage checked. see #189.
nahi authored
377 def initialize(*args)
378 proxy, agent_name, from = keyword_argument(args, :proxy, :agent_name, :from)
b6b5e4f * http-access2 -> httpclient
nahi authored
379 @proxy = nil # assigned later.
380 @no_proxy = nil
381 @www_auth = WWWAuth.new
382 @proxy_auth = ProxyAuth.new
383 @request_filter = [@proxy_auth, @www_auth]
384 @debug_dev = nil
385 @redirect_uri_callback = method(:default_redirect_uri_callback)
386 @test_loopback_response = []
c7688af * support SSL connection via HTTP Proxy which requires Proxy authori…
nahi authored
387 @session_manager = SessionManager.new(self)
2253f95 * test coverage checked. see #189.
nahi authored
388 @session_manager.agent_name = agent_name
389 @session_manager.from = from
b6b5e4f * http-access2 -> httpclient
nahi authored
390 @session_manager.ssl_config = @ssl_config = SSLConfig.new(self)
391 @cookie_manager = WebAgent::CookieManager.new
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
392 @follow_redirect_count = 10
27d4279 * recover http_proxy/HTTP_PROXY environment variable support.
nahi authored
393 load_environment
394 self.proxy = proxy if proxy
16ce3df @nahi restore webmock compatibility.
authored
395 keep_webmock_compat
396 end
397
398 # webmock 1.6.2 depends on HTTP::Message#body.content to work.
399 # let's keep it work iif webmock is loaded for a while.
400 def keep_webmock_compat
401 if respond_to?(:do_get_block_with_webmock)
402 ::HTTP::Message.module_eval do
403 def body
404 def (o = self.content).content
405 self
406 end
407 o
408 end
409 end
410 end
b6b5e4f * http-access2 -> httpclient
nahi authored
411 end
412
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
413 # Returns debug device if exists. See debug_dev=.
b6b5e4f * http-access2 -> httpclient
nahi authored
414 def debug_dev
415 @debug_dev
416 end
417
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
418 # Sets debug device. Once debug device is set, all HTTP requests and
419 # responses are dumped to given device. dev must respond to << for dump.
420 #
421 # Calling this method resets all existing sessions.
b6b5e4f * http-access2 -> httpclient
nahi authored
422 def debug_dev=(dev)
423 @debug_dev = dev
424 reset_all
425 @session_manager.debug_dev = dev
426 end
427
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
428 # Returns URI object of HTTP proxy if exists.
b6b5e4f * http-access2 -> httpclient
nahi authored
429 def proxy
430 @proxy
431 end
432
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
433 # Sets HTTP proxy used for HTTP connection. Given proxy can be an URI,
434 # a String or nil. You can set user/password for proxy authentication like
435 # HTTPClient#proxy = 'http://user:passwd@myproxy:8080'
436 #
437 # You can use environment variable 'http_proxy' or 'HTTP_PROXY' for it.
438 # You need to use 'cgi_http_proxy' or 'CGI_HTTP_PROXY' instead if you run
439 # HTTPClient from CGI environment from security reason. (HTTPClient checks
440 # 'REQUEST_METHOD' environment variable whether it's CGI or not)
441 #
442 # Calling this method resets all existing sessions.
b6b5e4f * http-access2 -> httpclient
nahi authored
443 def proxy=(proxy)
796a731 @nahi Regression: setting proxy with URI
authored
444 if proxy.nil? || proxy.to_s.empty?
b6b5e4f * http-access2 -> httpclient
nahi authored
445 @proxy = nil
446 @proxy_auth.reset_challenge
447 else
448 @proxy = urify(proxy)
449 if @proxy.scheme == nil or @proxy.scheme.downcase != 'http' or
450 @proxy.host == nil or @proxy.port == nil
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
451 raise ArgumentError.new("unsupported proxy #{proxy}")
b6b5e4f * http-access2 -> httpclient
nahi authored
452 end
453 @proxy_auth.reset_challenge
454 if @proxy.user || @proxy.password
455 @proxy_auth.set_auth(@proxy.user, @proxy.password)
456 end
457 end
458 reset_all
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
459 @session_manager.proxy = @proxy
b6b5e4f * http-access2 -> httpclient
nahi authored
460 @proxy
461 end
462
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
463 # Returns NO_PROXY setting String if given.
b6b5e4f * http-access2 -> httpclient
nahi authored
464 def no_proxy
465 @no_proxy
466 end
467
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
468 # Sets NO_PROXY setting String. no_proxy must be a comma separated String.
469 # Each entry must be 'host' or 'host:port' such as;
470 # HTTPClient#no_proxy = 'example.com,example.co.jp:443'
471 #
472 # 'localhost' is treated as a no_proxy site regardless of explicitly listed.
473 # HTTPClient checks given URI objects before accessing it.
474 # 'host' is tail string match. No IP-addr conversion.
475 #
476 # You can use environment variable 'no_proxy' or 'NO_PROXY' for it.
477 #
478 # Calling this method resets all existing sessions.
b6b5e4f * http-access2 -> httpclient
nahi authored
479 def no_proxy=(no_proxy)
480 @no_proxy = no_proxy
481 reset_all
482 end
483
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
484 # Sets credential for Web server authentication.
485 # domain:: a String or an URI to specify where HTTPClient should use this
486 # credential. If you set uri to nil, HTTPClient uses this credential
487 # wherever a server requires it.
488 # user:: username String.
489 # passwd:: password String.
490 #
491 # You can set multiple credentials for each uri.
492 #
493 # clnt.set_auth('http://www.example.com/foo/', 'foo_user', 'passwd')
494 # clnt.set_auth('http://www.example.com/bar/', 'bar_user', 'passwd')
495 #
496 # Calling this method resets all existing sessions.
497 def set_auth(domain, user, passwd)
498 uri = urify(domain)
b6b5e4f * http-access2 -> httpclient
nahi authored
499 @www_auth.set_auth(uri, user, passwd)
500 reset_all
501 end
502
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
503 # Deprecated. Use set_auth instead.
504 def set_basic_auth(domain, user, passwd)
505 uri = urify(domain)
b6b5e4f * http-access2 -> httpclient
nahi authored
506 @www_auth.basic_auth.set(uri, user, passwd)
507 reset_all
508 end
509
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
510 # Sets credential for Proxy authentication.
511 # user:: username String.
512 # passwd:: password String.
513 #
514 # Calling this method resets all existing sessions.
b6b5e4f * http-access2 -> httpclient
nahi authored
515 def set_proxy_auth(user, passwd)
516 @proxy_auth.set_auth(user, passwd)
517 reset_all
518 end
519
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
520 # Sets the filename where non-volatile Cookies be saved by calling
521 # save_cookie_store.
522 # This method tries to load and managing Cookies from the specified file.
523 #
524 # Calling this method resets all existing sessions.
b6b5e4f * http-access2 -> httpclient
nahi authored
525 def set_cookie_store(filename)
526 @cookie_manager.cookies_file = filename
527 @cookie_manager.load_cookies if filename
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
528 reset_all
b6b5e4f * http-access2 -> httpclient
nahi authored
529 end
530
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
531 # Try to save Cookies to the file specified in set_cookie_store. Unexpected
532 # error will be raised if you don't call set_cookie_store first.
533 # (interface mismatch between WebAgent::CookieManager implementation)
b6b5e4f * http-access2 -> httpclient
nahi authored
534 def save_cookie_store
535 @cookie_manager.save_cookies
536 end
537
3050bde @nahi Add shortcut methods.
authored
538 # Returns stored cookies.
539 def cookies
540 if @cookie_manager
541 @cookie_manager.cookies
542 end
543 end
544
90174d9 * spell check.
nahi authored
545 # Sets callback proc when HTTP redirect status is returned for get_content
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
546 # and post_content. default_redirect_uri_callback is used by default.
547 #
548 # If you need strict implementation which does not allow relative URI
549 # redirection, set strict_redirect_uri_callback instead.
550 #
551 # clnt.redirect_uri_callback = clnt.method(:strict_redirect_uri_callback)
552 #
b6b5e4f * http-access2 -> httpclient
nahi authored
553 def redirect_uri_callback=(redirect_uri_callback)
554 @redirect_uri_callback = redirect_uri_callback
555 end
556
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
557 # Retrieves a web resource.
b6b5e4f * http-access2 -> httpclient
nahi authored
558 #
90174d9 * spell check.
nahi authored
559 # uri:: a String or an URI object which represents an URL of web resource.
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
560 # query:: a Hash or an Array of query part of URL.
561 # e.g. { "a" => "b" } => 'http://host/part?a=b'.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
562 # Give an array to pass multiple value like
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
563 # [["a", "b"], ["a", "c"]] => 'http://host/part?a=b&a=c'.
25afc79 @nahi Change variable name: extheader -> header.
authored
564 # header:: a Hash or an Array of extra headers. e.g.
565 # { 'Accept' => '*/*' } or
566 # [['Accept', 'image/jpeg'], ['Accept', 'image/png']].
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
567 # &block:: Give a block to get chunked message-body of response like
568 # get_content(uri) { |chunked_body| ... }.
569 # Size of each chunk may not be the same.
b6b5e4f * http-access2 -> httpclient
nahi authored
570 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
571 # get_content follows HTTP redirect status (see HTTP::Status.redirect?)
572 # internally and try to retrieve content from redirected URL. See
573 # redirect_uri_callback= how HTTP redirection is handled.
b6b5e4f * http-access2 -> httpclient
nahi authored
574 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
575 # If you need to get full HTTP response including HTTP status and headers,
576 # use get method. get returns HTTP::Message as a response and you need to
577 # follow HTTP redirect by yourself if you need.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
578 def get_content(uri, *args, &block)
579 query, header = keyword_argument(args, :query, :header)
580 follow_redirect(:get, uri, query, nil, header || {}, &block).content
b6b5e4f * http-access2 -> httpclient
nahi authored
581 end
582
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
583 # Posts a content.
584 #
90174d9 * spell check.
nahi authored
585 # uri:: a String or an URI object which represents an URL of web resource.
611d2c0 @thodg Added doc for custom multipart
thodg authored
586 # body:: a Hash or an Array of body part. e.g.
587 # { "a" => "b" } => 'a=b'
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
588 # Give an array to pass multiple value like
611d2c0 @thodg Added doc for custom multipart
thodg authored
589 # [["a", "b"], ["a", "c"]] => 'a=b&a=c'
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
590 # When you pass a File as a value, it will be posted as a
611d2c0 @thodg Added doc for custom multipart
thodg authored
591 # multipart/form-data. e.g.
592 # { 'upload' => file }
593 # You can also send custom multipart by passing an array of hashes.
594 # Each part must have a :content attribute which can be a file, all
595 # other keys will become headers.
596 # [{ 'Content-Type' => 'text/plain', :content => "some text" },
597 # { 'Content-Type' => 'video/mp4', :content => File.new('video.mp4') }]
598 # => <Two parts with custom Content-Type header>
25afc79 @nahi Change variable name: extheader -> header.
authored
599 # header:: a Hash or an Array of extra headers. e.g.
600 # { 'Accept' => '*/*' }
601 # or
602 # [['Accept', 'image/jpeg'], ['Accept', 'image/png']].
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
603 # &block:: Give a block to get chunked message-body of response like
611d2c0 @thodg Added doc for custom multipart
thodg authored
604 # post_content(uri) { |chunked_body| ... }.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
605 # Size of each chunk may not be the same.
606 #
607 # post_content follows HTTP redirect status (see HTTP::Status.redirect?)
608 # internally and try to post the content to redirected URL. See
609 # redirect_uri_callback= how HTTP redirection is handled.
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
610 #
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
611 # If you need to get full HTTP response including HTTP status and headers,
612 # use post method.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
613 def post_content(uri, *args, &block)
614 body, header = keyword_argument(args, :body, :header)
615 follow_redirect(:post, uri, nil, body, header || {}, &block).content
b6b5e4f * http-access2 -> httpclient
nahi authored
616 end
617
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
618 # A method for redirect uri callback. How to use:
619 # clnt.redirect_uri_callback = clnt.method(:strict_redirect_uri_callback)
620 # This callback does not allow relative redirect such as
621 # Location: ../foo/
622 # in HTTP header. (raises BadResponseError instead)
b6b5e4f * http-access2 -> httpclient
nahi authored
623 def strict_redirect_uri_callback(uri, res)
624 newuri = URI.parse(res.header['location'][0])
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
625 if https?(uri) && !https?(newuri)
626 raise BadResponseError.new("redirecting to non-https resource")
627 end
4fc7a2b * added tests for coverage. closes #189.
nahi authored
628 unless newuri.is_a?(URI::HTTP)
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
629 raise BadResponseError.new("unexpected location: #{newuri}", res)
4fc7a2b * added tests for coverage. closes #189.
nahi authored
630 end
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
631 puts "redirect to: #{newuri}" if $DEBUG
b6b5e4f * http-access2 -> httpclient
nahi authored
632 newuri
633 end
634
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
635 # A default method for redirect uri callback. This method is used by
636 # HTTPClient instance by default.
637 # This callback allows relative redirect such as
638 # Location: ../foo/
639 # in HTTP header.
b6b5e4f * http-access2 -> httpclient
nahi authored
640 def default_redirect_uri_callback(uri, res)
641 newuri = URI.parse(res.header['location'][0])
642 unless newuri.is_a?(URI::HTTP)
643 newuri = uri + newuri
285cf2f @nahi Supress known warnings during test
authored
644 warn("could be a relative URI in location header which is not recommended")
645 warn("'The field value consists of a single absolute URI' in HTTP spec")
b6b5e4f * http-access2 -> httpclient
nahi authored
646 end
8a94b21 @nahi Fixed NME for non absolute-URI redirect on https. closes #20.
authored
647 if https?(uri) && !https?(newuri)
648 raise BadResponseError.new("redirecting to non-https resource")
649 end
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
650 puts "redirect to: #{newuri}" if $DEBUG
b6b5e4f * http-access2 -> httpclient
nahi authored
651 newuri
652 end
653
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
654 # Sends HEAD request to the specified URL. See request for arguments.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
655 def head(uri, *args)
29bd2d6 @nahi Handle argument Hash at new method: argument_to_hash
authored
656 request(:head, uri, argument_to_hash(args, :query, :header))
b6b5e4f * http-access2 -> httpclient
nahi authored
657 end
658
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
659 # Sends GET request to the specified URL. See request for arguments.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
660 def get(uri, *args, &block)
0623a18 @nahi Add keyword argument :follow_redirect: bool to get/post
authored
661 request(:get, uri, argument_to_hash(args, :query, :header, :follow_redirect), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
662 end
663
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
664 # Sends POST request to the specified URL. See request for arguments.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
665 def post(uri, *args, &block)
0623a18 @nahi Add keyword argument :follow_redirect: bool to get/post
authored
666 request(:post, uri, argument_to_hash(args, :body, :header, :follow_redirect), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
667 end
668
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
669 # Sends PUT request to the specified URL. See request for arguments.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
670 def put(uri, *args, &block)
29bd2d6 @nahi Handle argument Hash at new method: argument_to_hash
authored
671 request(:put, uri, argument_to_hash(args, :body, :header), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
672 end
673
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
674 # Sends DELETE request to the specified URL. See request for arguments.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
675 def delete(uri, *args, &block)
29bd2d6 @nahi Handle argument Hash at new method: argument_to_hash
authored
676 request(:delete, uri, argument_to_hash(args, :header), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
677 end
678
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
679 # Sends OPTIONS request to the specified URL. See request for arguments.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
680 def options(uri, *args, &block)
29bd2d6 @nahi Handle argument Hash at new method: argument_to_hash
authored
681 request(:options, uri, argument_to_hash(args, :header), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
682 end
683
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
684 # Sends PROPFIND request to the specified URL. See request for arguments.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
685 def propfind(uri, *args, &block)
29bd2d6 @nahi Handle argument Hash at new method: argument_to_hash
authored
686 request(:propfind, uri, argument_to_hash(args, :header), &block)
9192e7d * applied propfind/proppatch patch from an user. Thanks!
nahi authored
687 end
688
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
689 # Sends PROPPATCH request to the specified URL. See request for arguments.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
690 def proppatch(uri, *args, &block)
29bd2d6 @nahi Handle argument Hash at new method: argument_to_hash
authored
691 request(:proppatch, uri, argument_to_hash(args, :body, :header), &block)
9192e7d * applied propfind/proppatch patch from an user. Thanks!
nahi authored
692 end
693
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
694 # Sends TRACE request to the specified URL. See request for arguments.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
695 def trace(uri, *args, &block)
29bd2d6 @nahi Handle argument Hash at new method: argument_to_hash
authored
696 request('TRACE', uri, argument_to_hash(args, :query, :body, :header), &block)
b6b5e4f * http-access2 -> httpclient
nahi authored
697 end
698
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
699 # Sends a request to the specified URL.
700 #
701 # method:: HTTP method to be sent. method.to_s.upcase is used.
90174d9 * spell check.
nahi authored
702 # uri:: a String or an URI object which represents an URL of web resource.
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
703 # query:: a Hash or an Array of query part of URL.
704 # e.g. { "a" => "b" } => 'http://host/part?a=b'
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
705 # Give an array to pass multiple value like
c6d461b * added RDoc for httpclient/connection.rb. see #162.
nahi authored
706 # [["a", "b"], ["a", "c"]] => 'http://host/part?a=b&a=c'
87634ea @thodg Clean up doc for request
thodg authored
707 # body:: a Hash or an Array of body part. e.g.
708 # { "a" => "b" }
709 # => 'a=b'
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
710 # Give an array to pass multiple value like
87634ea @thodg Clean up doc for request
thodg authored
711 # [["a", "b"], ["a", "c"]]
712 # => 'a=b&a=c'.
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
713 # When the given method is 'POST' and the given body contains a file
87634ea @thodg Clean up doc for request
thodg authored
714 # as a value, it will be posted as a multipart/form-data. e.g.
715 # { 'upload' => file }
00c7985 @thodg Added custom multipart request, with doc.
thodg authored
716 # You can also send custom multipart by passing an array of hashes.
717 # Each part must have a :content attribute which can be a file, all
718 # other keys will become headers.
719 # [{ 'Content-Type' => 'text/plain', :content => "some text" },
720 # { 'Content-Type' => 'video/mp4', :content => File.new('video.mp4') }]
721 # => <Two parts with custom Content-Type header>
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
722 # See HTTP::Message.file? for actual condition of 'a file'.
25afc79 @nahi Change variable name: extheader -> header.
authored
723 # header:: a Hash or an Array of extra headers. e.g.
724 # { 'Accept' => '*/*' } or
725 # [['Accept', 'image/jpeg'], ['Accept', 'image/png']].
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
726 # &block:: Give a block to get chunked message-body of response like
727 # get(uri) { |chunked_body| ... }.
728 # Size of each chunk may not be the same.
729 #
730 # You can also pass a String as a body. HTTPClient just sends a String as
731 # a HTTP request message body.
732 #
733 # When you pass an IO as a body, HTTPClient sends it as a HTTP request with
5d49093 @nabeken Do not send request with Transfer-Encoding: chuncked when IO respond …
nabeken authored
734 # chunked encoding (Transfer-Encoding: chunked in HTTP header) if IO does not
735 # respond to :read. Bear in mind that some server application does not support
736 # chunked request. At least cgi.rb does not support it.
07569d5 @nahi Allow keyword style argument for HTTPClient#get, post, etc.
authored
737 def request(method, uri, *args, &block)
0623a18 @nahi Add keyword argument :follow_redirect: bool to get/post
authored
738 query, body, header, follow_redirect = keyword_argument(args, :query, :body, :header, :follow_redirect)
29bd2d6 @nahi Handle argument Hash at new method: argument_to_hash
authored
739 if [:post, :put].include?(method)
740 body ||= ''
741 end
742 if method == :propfind
743 header ||= PROPFIND_DEFAULT_EXTHEADER
744 else
745 header ||= {}
746 end
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
747 uri = urify(uri)
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
748 if block
749 filtered_block = proc { |res, str|
750 block.call(str)
751 }
b6b5e4f * http-access2 -> httpclient
nahi authored
752 end
0623a18 @nahi Add keyword argument :follow_redirect: bool to get/post
authored
753 if follow_redirect
754 follow_redirect(method, uri, query, body, header, &block)
755 else
756 do_request(method, uri, query, body, header, &filtered_block)
757 end
b6b5e4f * http-access2 -> httpclient
nahi authored
758 end
759
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
760 # Sends HEAD request in async style. See request_async for arguments.
761 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 @nahi Same keyword arg support for _async methods.
authored
762 def head_async(uri, *args)
763 query, header = keyword_argument(args, :query, :header)
764 request_async(:head, uri, query, nil, header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
765 end
766
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
767 # Sends GET request in async style. See request_async for arguments.
768 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 @nahi Same keyword arg support for _async methods.
authored
769 def get_async(uri, *args)
770 query, header = keyword_argument(args, :query, :header)
771 request_async(:get, uri, query, nil, header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
772 end
773
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
774 # Sends POST request in async style. See request_async for arguments.
775 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 @nahi Same keyword arg support for _async methods.
authored
776 def post_async(uri, *args)
777 body, header = keyword_argument(args, :body, :header)
778 request_async(:post, uri, nil, body || '', header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
779 end
780
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
781 # Sends PUT request in async style. See request_async for arguments.
782 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 @nahi Same keyword arg support for _async methods.
authored
783 def put_async(uri, *args)
784 body, header = keyword_argument(args, :body, :header)
785 request_async(:put, uri, nil, body || '', header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
786 end
787
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
788 # Sends DELETE request in async style. See request_async for arguments.
789 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 @nahi Same keyword arg support for _async methods.
authored
790 def delete_async(uri, *args)
791 header = keyword_argument(args, :header)
792 request_async(:delete, uri, nil, nil, header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
793 end
794
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
795 # Sends OPTIONS request in async style. See request_async for arguments.
796 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 @nahi Same keyword arg support for _async methods.
authored
797 def options_async(uri, *args)
798 header = keyword_argument(args, :header)
799 request_async(:options, uri, nil, nil, header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
800 end
801
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
802 # Sends PROPFIND request in async style. See request_async for arguments.
803 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 @nahi Same keyword arg support for _async methods.
authored
804 def propfind_async(uri, *args)
805 header = keyword_argument(args, :header)
806 request_async(:propfind, uri, nil, nil, header || PROPFIND_DEFAULT_EXTHEADER)
9192e7d * applied propfind/proppatch patch from an user. Thanks!
nahi authored
807 end
808
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
809 # Sends PROPPATCH request in async style. See request_async for arguments.
810 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 @nahi Same keyword arg support for _async methods.
authored
811 def proppatch_async(uri, *args)
812 body, header = keyword_argument(args, :body, :header)
813 request_async(:proppatch, uri, nil, body, header || {})
9192e7d * applied propfind/proppatch patch from an user. Thanks!
nahi authored
814 end
815
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
816 # Sends TRACE request in async style. See request_async for arguments.
817 # It immediately returns a HTTPClient::Connection instance as a result.
759e1a4 @nahi Same keyword arg support for _async methods.
authored
818 def trace_async(uri, *args)
819 query, body, header = keyword_argument(args, :query, :body, :header)
820 request_async(:trace, uri, query, body, header || {})
b6b5e4f * http-access2 -> httpclient
nahi authored
821 end
822
90174d9 * spell check.
nahi authored
823 # Sends a request in async style. request method creates new Thread for
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
824 # HTTP connection and returns a HTTPClient::Connection instance immediately.
825 #
826 # Arguments definition is the same as request.
25afc79 @nahi Change variable name: extheader -> header.
authored
827 def request_async(method, uri, query = nil, body = nil, header = {})
b6b5e4f * http-access2 -> httpclient
nahi authored
828 uri = urify(uri)
25afc79 @nahi Change variable name: extheader -> header.
authored
829 do_request_async(method, uri, query, body, header)
b6b5e4f * http-access2 -> httpclient
nahi authored
830 end
831
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
832 # Resets internal session for the given URL. Keep-alive connection for the
833 # site (host-port pair) is disconnected if exists.
b6b5e4f * http-access2 -> httpclient
nahi authored
834 def reset(uri)
835 uri = urify(uri)
836 @session_manager.reset(uri)
837 end
838
792bbbb * added RDoc to httpclient.rb. see #162.
nahi authored
839 # Resets all of internal sessions. Keep-alive connections are disconnected.
b6b5e4f * http-access2 -> httpclient
nahi authored
840 def reset_all
841 @session_manager.reset_all
842 end
843
844 private
845
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
846 class RetryableResponse < StandardError # :nodoc:
847 end
848
849 class KeepAliveDisconnected < StandardError # :nodoc:
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
850 attr_reader :sess
851 def initialize(sess = nil)
852 @sess = sess
853 end
e3ae9e2 * version: 2.1.2 -> 2.1.3-SNAPSHOT
nahi authored
854 end
855
25afc79 @nahi Change variable name: extheader -> header.
authored
856 def do_request(method, uri, query, body, header, &block)
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
857 conn = Connection.new
858 res = nil
cf55b01 * Cookie header is not set in authentication negotiation. do not reu…
nahi authored
859 if HTTP::Message.file?(body)
860 pos = body.pos rescue nil
861 end
3f2a996 * tested proxy + SSL against squid.
nahi authored
862 retry_count = @session_manager.protocol_retry_count
4d3c829 * refactoring: push down create_request invocation into do_request/d…
nahi authored
863 proxy = no_proxy?(uri) ? nil : @proxy
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
864 while retry_count > 0
cf55b01 * Cookie header is not set in authentication negotiation. do not reu…
nahi authored
865 body.pos = pos if pos
25afc79 @nahi Change variable name: extheader -> header.
authored
866 req = create_request(method, uri, query, body, header)
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
867 begin
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
868 protect_keep_alive_disconnected do
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
869 do_get_block(req, proxy, conn, &block)
870 end
871 res = conn.pop
872 break
873 rescue RetryableResponse
874 res = conn.pop
875 retry_count -= 1
876 end
877 end
878 res
879 end
880
25afc79 @nahi Change variable name: extheader -> header.
authored
881 def do_request_async(method, uri, query, body, header)
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
882 conn = Connection.new
883 t = Thread.new(conn) { |tconn|
1986c59 @nahi Applied the patch in #44: let Connection#pop raise the Exception when…
authored
884 begin
885 if HTTP::Message.file?(body)
886 pos = body.pos rescue nil
887 end
888 retry_count = @session_manager.protocol_retry_count
889 proxy = no_proxy?(uri) ? nil : @proxy
890 while retry_count > 0
891 body.pos = pos if pos
25afc79 @nahi Change variable name: extheader -> header.
authored
892 req = create_request(method, uri, query, body, header)
1986c59 @nahi Applied the patch in #44: let Connection#pop raise the Exception when…
authored
893 begin
894 protect_keep_alive_disconnected do
895 do_get_stream(req, proxy, tconn)
896 end
897 break
898 rescue RetryableResponse
899 retry_count -= 1
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
900 end
901 end
8aa3a34 @nahi Make sure to rescue an Exception. closes #44.
authored
902 rescue Exception
1986c59 @nahi Applied the patch in #44: let Connection#pop raise the Exception when…
authored
903 conn.push $!
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
904 end
905 }
906 conn.async_thread = t
907 conn
908 end
909
27d4279 * recover http_proxy/HTTP_PROXY environment variable support.
nahi authored
910 def load_environment
911 # http_proxy
912 if getenv('REQUEST_METHOD')
913 # HTTP_PROXY conflicts with the environment variable usage in CGI where
914 # HTTP_* is used for HTTP header information. Unlike open-uri, we
90174d9 * spell check.
nahi authored
915 # simply ignore http_proxy in CGI env and use cgi_http_proxy instead.
27d4279 * recover http_proxy/HTTP_PROXY environment variable support.
nahi authored
916 self.proxy = getenv('cgi_http_proxy')
917 else
918 self.proxy = getenv('http_proxy')
919 end
920 # no_proxy
921 self.no_proxy = getenv('no_proxy')
922 end
923
924 def getenv(name)
925 ENV[name.downcase] || ENV[name.upcase]
926 end
927
25afc79 @nahi Change variable name: extheader -> header.
authored
928 def follow_redirect(method, uri, query, body, header, &block)
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
929 uri = urify(uri)
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
930 if block
e3c1331 * a few performance improvements. see #191.
nahi authored
931 filtered_block = proc { |r, str|
932 block.call(str) if HTTP::Status.successful?(r.status)
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
933 }
934 end
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
935 if HTTP::Message.file?(body)
936 pos = body.pos rescue nil
937 end
938 retry_number = 0
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
939 while retry_number < @follow_redirect_count
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
940 body.pos = pos if pos
25afc79 @nahi Change variable name: extheader -> header.
authored
941 res = do_request(method, uri, query, body, header, &filtered_block)
b6b5e4f * http-access2 -> httpclient
nahi authored
942 if HTTP::Status.successful?(res.status)
943 return res
944 elsif HTTP::Status.redirect?(res.status)
4d3c829 * refactoring: push down create_request invocation into do_request/d…
nahi authored
945 uri = urify(@redirect_uri_callback.call(uri, res))
b6b5e4f * http-access2 -> httpclient
nahi authored
946 retry_number += 1
947 else
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
948 raise BadResponseError.new("unexpected response: #{res.header.inspect}", res)
b6b5e4f * http-access2 -> httpclient
nahi authored
949 end
950 end
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
951 raise BadResponseError.new("retry count exceeded", res)
b6b5e4f * http-access2 -> httpclient
nahi authored
952 end
953
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
954 def protect_keep_alive_disconnected
b6b5e4f * http-access2 -> httpclient
nahi authored
955 begin
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
956 yield
a620b5f @nahi Invalidate pooled sessions for the same destination when we got a Kee…
authored
957 rescue KeepAliveDisconnected => e
958 if e.sess
959 @session_manager.invalidate(e.sess.dest)
960 end
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
961 yield
b6b5e4f * http-access2 -> httpclient
nahi authored
962 end
963 end
964
25afc79 @nahi Change variable name: extheader -> header.
authored
965 def create_request(method, uri, query, body, header)
5fb4436 * Security fix introduced at 2.1.3.
nahi authored
966 method = method.to_s.upcase
25afc79 @nahi Change variable name: extheader -> header.
authored
967 if header.is_a?(Hash)
968 header = header.to_a
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
969 else
25afc79 @nahi Change variable name: extheader -> header.
authored
970 header = header.dup
b6b5e4f * http-access2 -> httpclient
nahi authored
971 end
972 boundary = nil
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
973 if body
25afc79 @nahi Change variable name: extheader -> header.
authored
974 _, content_type = header.find { |key, value|
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
975 key.downcase == 'content-type'
976 }
977 if content_type
978 if /\Amultipart/ =~ content_type
979 if content_type =~ /boundary=(.+)\z/
980 boundary = $1
981 else
982 boundary = create_boundary
983 content_type = "#{content_type}; boundary=#{boundary}"
25afc79 @nahi Change variable name: extheader -> header.
authored
984 header = override_header(header, 'Content-Type', content_type)
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
985 end
986 end
987 elsif method == 'POST'
988 if file_in_form_data?(body)
989 boundary = create_boundary
990 content_type = "multipart/form-data; boundary=#{boundary}"
991 else
992 content_type = 'application/x-www-form-urlencoded'
993 end
25afc79 @nahi Change variable name: extheader -> header.
authored
994 header << ['Content-Type', content_type]
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
995 end
b6b5e4f * http-access2 -> httpclient
nahi authored
996 end
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
997 req = HTTP::Message.new_request(method, uri, query, body, boundary)
25afc79 @nahi Change variable name: extheader -> header.
authored
998 header.each do |key, value|
83dc6ec * avoid duplicated request header line while HTTP access negotiation…
nahi authored
999 req.header.add(key, value)
b6b5e4f * http-access2 -> httpclient
nahi authored
1000 end
83dc6ec * avoid duplicated request header line while HTTP access negotiation…
nahi authored
1001 if @cookie_manager && cookie = @cookie_manager.find(uri)
1002 req.header.add('Cookie', cookie)
fad6e92 * post an IO body on the second request as well as the first request…
nahi authored
1003 end
b6b5e4f * http-access2 -> httpclient
nahi authored
1004 req
1005 end
1006
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
1007 def create_boundary
1008 Digest::SHA1.hexdigest(Time.now.to_s)
1009 end
1010
1011 def file_in_form_data?(body)
2253f95 * test coverage checked. see #189.
nahi authored
1012 HTTP::Message.multiparam_query?(body) &&
1013 body.any? { |k, v| HTTP::Message.file?(v) }
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
1014 end
1015
25afc79 @nahi Change variable name: extheader -> header.
authored
1016 def override_header(header, key, value)
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
1017 result = []
25afc79 @nahi Change variable name: extheader -> header.
authored
1018 header.each do |k, v|
e77f3e0 * easier file upload: add Content-Type: multipart/form-data when the…
nahi authored
1019 if k.downcase == key.downcase
1020 result << [key, value]
1021 else
1022 result << [k, v]
1023 end
1024 end
1025 result
1026 end
1027
b6b5e4f * http-access2 -> httpclient
nahi authored
1028 NO_PROXY_HOSTS = ['localhost']
1029
1030 def no_proxy?(uri)
1031 if !@proxy or NO_PROXY_HOSTS.include?(uri.host)
1032 return true
1033 end
1034 unless @no_proxy
1035 return false
1036 end
1037 @no_proxy.scan(/([^:,]+)(?::(\d+))?/) do |host, port|
1038 if /(\A|\.)#{Regexp.quote(host)}\z/i =~ uri.host &&
1039 (!port || uri.port == port.to_i)
1040 return true
1041 end
1042 end
1043 false
1044 end
1045
1046 # !! CAUTION !!
1047 # Method 'do_get*' runs under MT conditon. Be careful to change.
1048 def do_get_block(req, proxy, conn, &block)
1049 @request_filter.each do |filter|
1050 filter.filter_request(req)
1051 end
1052 if str = @test_loopback_response.shift
3050bde @nahi Add shortcut methods.
authored
1053 dump_dummy_request_response(req.http_body.dump, str) if @debug_dev
b6b5e4f * http-access2 -> httpclient
nahi authored
1054 conn.push(HTTP::Message.new_response(str))
1055 return
1056 end
47ba7a6 * [212] broke get/post with block. fixed. closes #173.
nahi authored
1057 content = block ? nil : ''
b6b5e4f * http-access2 -> httpclient
nahi authored
1058 res = HTTP::Message.new_response(content)
1059 @debug_dev << "= Request\n\n" if @debug_dev
1060 sess = @session_manager.query(req, proxy)
1061 res.peer_cert = sess.ssl_peer_cert
1062 @debug_dev << "\n\n= Response\n\n" if @debug_dev
1063 do_get_header(req, res, sess)
1064 conn.push(res)
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
1065 sess.get_body do |part|
e5efea5 @nahi Set response String encoding
authored
1066 set_encoding(part, res.body_encoding)
19a2f15 * reduce memory usage when calling HTTPClient#get with a block. app…
nahi authored
1067 if block
c9d999f * get_content with block yielded unexpected message body. closes #173.
nahi authored
1068 block.call(res, part)
19a2f15 * reduce memory usage when calling HTTPClient#get with a block. app…
nahi authored
1069 else
1070 content << part
1071 end
b6b5e4f * http-access2 -> httpclient
nahi authored
1072 end
9ada537 @nahi Make Keep-Alive test use threads properly.
authored
1073 # there could be a race condition but it's OK to cache unreusable
1074 # connection because we do retry for that case.
b6b5e4f * http-access2 -> httpclient
nahi authored
1075 @session_manager.keep(sess) unless sess.closed?
1076 commands = @request_filter.collect { |filter|
1077 filter.filter_response(req, res)
1078 }
1079 if commands.find { |command| command == :retry }
1080 raise RetryableResponse.new
1081 end
1082 end
1083
1084 def do_get_stream(req, proxy, conn)
1085 @request_filter.each do |filter|
1086 filter.filter_request(req)
1087 end
1088 if str = @test_loopback_response.shift
3050bde @nahi Add shortcut methods.
authored
1089 dump_dummy_request_response(req.http_body.dump, str) if @debug_dev
4fc7a2b * added tests for coverage. closes #189.
nahi authored
1090 conn.push(HTTP::Message.new_response(StringIO.new(str)))
b6b5e4f * http-access2 -> httpclient
nahi authored
1091 return
1092 end
1093 piper, pipew = IO.pipe
1094 res = HTTP::Message.new_response(piper)
1095 @debug_dev << "= Request\n\n" if @debug_dev
1096 sess = @session_manager.query(req, proxy)
1097 res.peer_cert = sess.ssl_peer_cert
1098 @debug_dev << "\n\n= Response\n\n" if @debug_dev
1099 do_get_header(req, res, sess)
1100 conn.push(res)
a176e7b * check test coverage on lib/httpclient/session.rb. see #189.
nahi authored
1101 sess.get_body do |part|
e5efea5 @nahi Set response String encoding
authored
1102 set_encoding(part, res.body_encoding)
0b7e801 @nahi avoid data corruption bug in async method.
authored
1103 pipew.write(part)
b6b5e4f * http-access2 -> httpclient
nahi authored
1104 end
1105 pipew.close
1106 @session_manager.keep(sess) unless sess.closed?
7c8ebd1 @nahi Avoid unused variable warnings from 1.9. closes #24.
authored
1107 _ = @request_filter.collect { |filter|
b6b5e4f * http-access2 -> httpclient
nahi authored
1108 filter.filter_response(req, res)
1109 }
1110 # ignore commands (not retryable in async mode)
1111 end
1112
1113 def do_get_header(req, res, sess)
1949938 @nahi Do not depend on Float#to_s behavior. closes #36.
authored
1114 res.http_version, res.status, res.reason, headers = sess.get_header
e5efea5 @nahi Set response String encoding
authored
1115 res.header.set_headers(headers)
17cbbf9 * refactoring around proxy handling. reduces created Objects. see …
nahi authored
1116 if @cookie_manager
b6b5e4f * http-access2 -> httpclient
nahi authored
1117 res.header['set-cookie'].each do |cookie|
1118 @cookie_manager.parse(cookie, req.header.request_uri)
1119 end
1120 end
1121 end
1122
1123 def dump_dummy_request_response(req, res)
1124 @debug_dev << "= Dummy Request\n\n"
1125 @debug_dev << req
1126 @debug_dev << "\n\n= Dummy Response\n\n"
1127 @debug_dev << res
1128 end
e5efea5 @nahi Set response String encoding
authored
1129
1130 def set_encoding(str, encoding)
1131 str.force_encoding(encoding) if encoding
1132 end
b6b5e4f * http-access2 -> httpclient
nahi authored
1133 end
Something went wrong with that request. Please try again.