Permalink
Browse files

First import/fork.

  • Loading branch information...
0 parents commit f3172f3abfe25fa871f33b2b592a04abbaaf5a52 unknown committed Jul 23, 2009
Showing with 2,403 additions and 0 deletions.
  1. +59 −0 History.txt
  2. +7 −0 Manifest.txt
  3. +54 −0 README.txt
  4. +103 −0 Rakefile
  5. +160 −0 lib/net_fix.rb
  6. +435 −0 lib/right_http_connection.rb
  7. +1,585 −0 setup.rb
@@ -0,0 +1,59 @@
+== 0.0.1 2007-05-15
+* 1 major enhancement:
+ * Initial release
+
+== 0.1.2 2007-06-27
+
+* No major changes.
+
+== 0.1.3 2007-07-09
+
+* No change.
+
+== 0.1.4 2007-08-10
+
+* r1442, todd, 2007-08-07 15:45:24
+ * # 373, Add support in right_http_connection for bailing out to a block while
+ reading the HTTP response (to support GET streaming...)
+
+* r1411, todd, 2007-08-03 15:14:45
+ * # 373, Stream uploads (PUTs) if the source is a file, stream, or anything
+ read()-able
+
+== 1.1.0 2007-08-15
+Initial public release
+
+== 1.2.0 2007-10-05
+
+* r1867, konstantin, 2007-10-05 06:19:45
+ * # 220, (re)open connection to server if none exists or connection params
+ have changed
+
+== 1.2.1
+
+* r2648, konstantin, 01-24-08 11:12:00
+ * net_fix.rb moved from right_aws gem to fix the problem with uploading the streamable
+ objects to S3
+
+* r2764, konstantin, 02-08-08 00:05:00 +03:00
+ * "RightAws: incompatible Net::HTTP monkey-patch" exception is raised if our net_fix
+ patch was overriden (by attachment_fu for example, to avoid this load attachment_fu
+ before loading the right_http_connection gem).
+
+== 1.2.2
+
+* r3524, konstantin, 2008-04-17 11:35:42 +0400
+ * Fixed a problem with incorrect error handling (connection retries always failed).
+
+== 1.2.3
+
+- Added support for setting retry & timeout parameters in the constructor
+- Improve handling of data streams during upload: if there is a failure and a retry, reset
+ the seek pointer for the subsequent re-request
+
+== 1.2.4
+
+* r4984, konstantin, 2008-08-11 14:49:18 +0400
+ * fixed a bug: <NoMethodError: You have a nil object when you didn't expect it!
+ The error occurred while evaluating nil.body_stream>
+
@@ -0,0 +1,7 @@
+History.txt
+Manifest.txt
+README.txt
+Rakefile
+lib/net_fix.rb
+lib/right_http_connection.rb
+setup.rb
@@ -0,0 +1,54 @@
+RightScale::HttpConnection
+ by RightScale, Inc.
+ www.RightScale.com
+
+== DESCRIPTION:
+
+Rightscale::HttpConnection is a robust HTTP/S library. It implements a retry
+algorithm for low-level network errors.
+
+== FEATURES:
+
+- provides put/get streaming
+- does configurable retries on connect and read timeouts, DNS failures, etc.
+- HTTPS certificate checking
+
+== SYNOPSIS:
+
+
+== REQUIREMENTS:
+
+- 2/11/08: If you use RightScale::HttpConnection in conjunction with attachment_fu, the
+ HttpConnection gem must be included (using the require statement) AFTER
+ attachment_fu.
+ This is due to a conflict between the HttpConnection gem and another
+ gem required by attachment_fu.
+
+
+
+== INSTALL:
+
+sudo gem install right_http_connection
+
+== LICENSE:
+
+Copyright (c) 2007-2008 RightScale, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,103 @@
+require 'rubygems'
+require 'rake'
+require 'rake/clean'
+require 'rake/testtask'
+require 'rake/packagetask'
+require 'rake/gempackagetask'
+require 'rake/rdoctask'
+require 'rake/contrib/rubyforgepublisher'
+require 'fileutils'
+require 'hoe'
+include FileUtils
+require File.join(File.dirname(__FILE__), 'lib', 'right_http_connection')
+
+AUTHOR = 'RightScale' # can also be an array of Authors
+EMAIL = "rubygems@rightscale.com"
+DESCRIPTION = "RightScale's robust HTTP/S connection module"
+GEM_NAME = 'right_http_connection' # what ppl will type to install your gem
+RUBYFORGE_PROJECT = 'rightscale' # The unix name for your project
+HOMEPATH = "http://#{RUBYFORGE_PROJECT}.rubyforge.org"
+DOWNLOAD_PATH = "http://rubyforge.org/projects/#{RUBYFORGE_PROJECT}"
+
+NAME = "right_http_connection"
+REV = nil # UNCOMMENT IF REQUIRED: File.read(".svn/entries")[/committed-rev="(d+)"/, 1] rescue nil
+VERS = RightHttpConnection::VERSION::STRING + (REV ? ".#{REV}" : "")
+CLEAN.include ['**/.*.sw?', '*.gem', '.config', '**/.DS_Store']
+RDOC_OPTS = ['--quiet', '--title', 'right_http_connection documentation',
+ "--opname", "index.html",
+ "--line-numbers",
+ "--main", "README",
+ "--inline-source"]
+
+# Suppress Hoe's self-inclusion as a dependency for our Gem. This also keeps
+# Rake & rubyforge out of the dependency list. Users must manually install
+# these gems to run tests, etc.
+# TRB 2/19/09: also do this for the extra_dev_deps array present in newer hoes.
+# Older versions of RubyGems will try to install developer-dependencies as
+# required runtime dependencies....
+class Hoe
+ def extra_deps
+ @extra_deps.reject do |x|
+ Array(x).first == 'hoe'
+ end
+ end
+ def extra_dev_deps
+ @extra_dev_deps.reject do |x|
+ Array(x).first == 'hoe'
+ end
+ end
+end
+
+# Generate all the Rake tasks
+# Run 'rake -T' to see list of generated tasks (from gem root directory)
+hoe = Hoe.new(GEM_NAME, VERS) do |p|
+ p.author = AUTHOR
+ p.description = DESCRIPTION
+ p.email = EMAIL
+ p.summary = DESCRIPTION
+ p.url = HOMEPATH
+ p.rubyforge_name = RUBYFORGE_PROJECT if RUBYFORGE_PROJECT
+ p.test_globs = ["test/**/test_*.rb"]
+ p.clean_globs = CLEAN #An array of file patterns to delete on clean.
+ p.remote_rdoc_dir = "right_http_gem_doc"
+
+ # == Optional
+ p.changes = p.paragraphs_of("History.txt", 0..1).join("\n\n")
+ #p.extra_deps = [] # An array of rubygem dependencies [name, version], e.g. [ ['active_support', '>= 1.3.1'] ]
+ #p.spec_extras = {} # A hash of extra values to set in the gemspec.
+end
+
+
+desc 'Generate website files'
+task :website_generate do
+ Dir['website/**/*.txt'].each do |txt|
+ sh %{ ruby scripts/txt2html #{txt} > #{txt.gsub(/txt$/,'html')} }
+ end
+end
+
+desc 'Upload website files to rubyforge'
+task :website_upload do
+ config = YAML.load(File.read(File.expand_path("~/.rubyforge/user-config.yml")))
+ host = "#{config["username"]}@rubyforge.org"
+ remote_dir = "/var/www/gforge-projects/#{RUBYFORGE_PROJECT}/"
+ # remote_dir = "/var/www/gforge-projects/#{RUBYFORGE_PROJECT}/#{GEM_NAME}"
+ local_dir = 'website'
+ sh %{rsync -av #{local_dir}/ #{host}:#{remote_dir}}
+end
+
+desc 'Generate and upload website files'
+task :website => [:website_generate, :website_upload]
+
+desc 'Release the website and new gem version'
+task :deploy => [:check_version, :website, :release]
+
+task :check_version do
+ unless ENV['VERSION']
+ puts 'Must pass a VERSION=x.y.z release version'
+ exit
+ end
+ unless ENV['VERSION'] == VERS
+ puts "Please update your version.rb to match the release version, currently #{VERS}"
+ exit
+ end
+end
@@ -0,0 +1,160 @@
+#
+# Copyright (c) 2008 RightScale Inc
+#
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be
+# included in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+#
+
+# Net::HTTP and Net::HTTPGenericRequest fixes to support 100-continue on
+# POST and PUT. The request must have 'expect' field set to '100-continue'.
+
+
+module Net
+
+ class BufferedIO #:nodoc:
+ # Monkey-patch Net::BufferedIO to read > 1024 bytes from the socket at a time
+
+ # Default size (in bytes) of the max read from a socket into the user space read buffers for socket IO
+ DEFAULT_SOCKET_READ_SIZE = 16*1024
+
+ @@socket_read_size = DEFAULT_SOCKET_READ_SIZE
+
+ def self.socket_read_size=(readsize)
+ if(readsize <= 0)
+ return
+ end
+ @@socket_read_size = readsize
+ end
+
+ def self.socket_read_size?()
+ @@socket_read_size
+ end
+
+ def rbuf_fill
+ timeout(@read_timeout) {
+ @rbuf << @io.sysread(@@socket_read_size)
+ }
+ end
+ end
+
+
+ #-- Net::HTTPGenericRequest --
+
+ class HTTPGenericRequest
+ # Monkey-patch Net::HTTPGenericRequest to read > 1024 bytes from the local data
+ # source at a time (used in streaming PUTs)
+
+ # Default size (in bytes) of the max read from a local source (File, String,
+ # etc.) to the user space write buffers for socket IO.
+ DEFAULT_LOCAL_READ_SIZE = 16*1024
+
+ @@local_read_size = DEFAULT_LOCAL_READ_SIZE
+
+ def self.local_read_size=(readsize)
+ if(readsize <= 0)
+ return
+ end
+ @@local_read_size = readsize
+ end
+
+ def self.local_read_size?()
+ @@local_read_size
+ end
+
+ def exec(sock, ver, path, send_only=nil) #:nodoc: internal use only
+ if @body
+ send_request_with_body sock, ver, path, @body, send_only
+ elsif @body_stream
+ send_request_with_body_stream sock, ver, path, @body_stream, send_only
+ else
+ write_header(sock, ver, path)
+ end
+ end
+
+ private
+
+ def send_request_with_body(sock, ver, path, body, send_only=nil)
+ self.content_length = body.length
+ delete 'Transfer-Encoding'
+ supply_default_content_type
+ write_header(sock, ver, path) unless send_only == :body
+ sock.write(body) unless send_only == :header
+ end
+
+ def send_request_with_body_stream(sock, ver, path, f, send_only=nil)
+ unless content_length() or chunked?
+ raise ArgumentError,
+ "Content-Length not given and Transfer-Encoding is not `chunked'"
+ end
+ supply_default_content_type
+ write_header(sock, ver, path) unless send_only == :body
+ unless send_only == :header
+ if chunked?
+ while s = f.read(@@local_read_size)
+ sock.write(sprintf("%x\r\n", s.length) << s << "\r\n")
+ end
+ sock.write "0\r\n\r\n"
+ else
+ while s = f.read(@@local_read_size)
+ sock.write s
+ end
+ end
+ end
+ end
+ end
+
+
+ #-- Net::HTTP --
+
+ class HTTP
+ def request(req, body = nil, &block) # :yield: +response+
+ unless started?
+ start {
+ req['connection'] ||= 'close'
+ return request(req, body, &block)
+ }
+ end
+ if proxy_user()
+ unless use_ssl?
+ req.proxy_basic_auth proxy_user(), proxy_pass()
+ end
+ end
+ # set body
+ req.set_body_internal body
+ begin_transport req
+ # if we expect 100-continue then send a header first
+ send_only = ((req.is_a?(Post)||req.is_a?(Put)) && (req['expect']=='100-continue')) ? :header : nil
+ req.exec @socket, @curr_http_version, edit_path(req.path), send_only
+ begin
+ res = HTTPResponse.read_new(@socket)
+ # if we expected 100-continue then send a body
+ if res.is_a?(HTTPContinue) && send_only && req['content-length'].to_i > 0
+ req.exec @socket, @curr_http_version, edit_path(req.path), :body
+ end
+ end while res.kind_of?(HTTPContinue)
+ res.reading_body(@socket, req.response_body_permitted?) {
+ yield res if block_given?
+ }
+ end_transport req, res
+ res
+ end
+ end
+
+end
Oops, something went wrong. Retry.

0 comments on commit f3172f3

Please sign in to comment.