Skip to content

Commit

Permalink
Merge branch 'httpclient_support'
Browse files Browse the repository at this point in the history
* httpclient_support: (36 commits)
  Regenerate gemspec
  Rename an internal module to make a more accurate historical reference :)
  Rename an internal method for clarity
  Rakefile: less sudo
  Rakefile: Add httpclient as a development dependency
  Rakefile: remove YARD as a development dependency, since we do a begin/rescue around the task definition
  Rakefile: update gem description for HTTPClient support
  Some RDoc syntax fixes in the README
  Add info to the README about HTTPClient support
  Move internal code out of the top-level module, so it only contains the end-user API
  Stop saving logged requests, so the GC can remove them
  Reorder methods so all the config methods are together again
  Refactor Samuel.load_drivers to be more concise
  Reorganize code into a consistent module/filesystem structure
  Extend loaded HTTP drivers using modules instead of dynamically requiring files
  Always load all LogEntry subclasses, now that they can be safely loaded without loading the HTTP drivers
  Check classes using strings, so this file can be safely required when Net::HTTP isn't loaded
  Make sure Net::HTTP requests are still logged when Net::HTTP raises during the connection stage
  Only load extensions for the HTTP drivers already loaded.
  We're not using benchmark anymore
  ...

Conflicts:
	README.rdoc
  • Loading branch information
chrisk committed Jan 1, 2010
2 parents 60e654a + ccffdf9 commit d5a056c
Show file tree
Hide file tree
Showing 17 changed files with 587 additions and 158 deletions.
50 changes: 32 additions & 18 deletions README.rdoc
@@ -1,13 +1,16 @@
= Samuel

Samuel is a gem for automatic logging of your Net::HTTP requests. It's named for
Samuel is a gem for automatic logging of your HTTP requests. It's named for
the serial diarist Mr. Pepys, who was known to reliably record events both
quotidian and remarkable.

Should a Great Plague, Fire, or Whale befall an important external web service
you use, you'll be sure to have a tidy record of it.

== Usage:
It supports both Net::HTTP and HTTPClient (formerly HTTPAccess2),
automatically loading the correct logger for the HTTP client you're using.

== Usage

When Rails is loaded, Samuel configures a few things automatically. So all you
need to do is this:
Expand All @@ -24,6 +27,14 @@ For non-Rails projects, you'll have to manually configure logging, like this:

If you don't assign a logger, Samuel will configure a default logger on +STDOUT+.

== HTTP Clients

When you load Samuel, it automatically detects which HTTP clients you've
loaded, then patches them to add logging. If no HTTP drivers are loaded when
you load Samuel, it will automatically load Net::HTTP for you. (So, if you're
using HTTPClient or a library based on it, make sure to require it before you
require Samuel.)

== Configuration

There are two ways to specify configuration options for Samuel: global and
Expand All @@ -44,23 +55,26 @@ configuration for a set of HTTP requests:

Right now, there are three configuration changes you can make in either style:

* +:labels+ - This is a hash with domain substrings as keys and log labels as
values. If a request domain includes one of the domain substrings, the
corresponding label will be used for the first part of that log entry. By
default this is set to <tt>{"" => "HTTP"}</tt>, so that all requests are
* <tt>:labels</tt> -- This is a hash with domain substrings as keys and log
labels as values. If a request domain includes one of the domain substrings,
the corresponding label will be used for the first part of that log entry.
By default this is set to <tt>{"" => "HTTP"}</tt>, so that all requests are
labeled with <tt>"HTTP Request"</tt>.
* +:label+ - As an alternative to the +:labels+ hash, this is simply a string.
If set, it takes precedence over any +:labels+ (by default, it's not set). It
gets <tt>"Request"</tt> appended to it as well -- so if you want your log to
always say +Twitter API Request+ instead of the default +HTTP Request+, you
can set this to <tt>"Twitter API"</tt>. I'd recommend using this setting
globally if you're only making requests to one service, or inline if you just
need to temporarily override the global +:labels+.
* +:filtered_params+ - This works just like Rails's +filter_parameter_logging+
method. Set it to a symbol, string, or array of them, and Samuel will filter
the value of query parameters that have any of these patterns as a substring
by replacing the value with <tt>[FILTERED]</tt> in your logs. By default, no
filtering is enabled.

* <tt>:label</tt> -- As an alternative to the <tt>:labels</tt> hash, this is
simply a string. If set, it takes precedence over any <tt>:labels</tt> (by
default, it's not set). It gets <tt>"Request"</tt> appended to it as well --
so if you want your log to always say <tt>Twitter API Request</tt> instead
of the default <tt>HTTP Request</tt>, you can set this to <tt>"Twitter
API"</tt>. I'd recommend using this setting globally if you're only making
requests to one service, or inline if you just need to temporarily override
the global <tt>:labels</tt>.

* <tt>:filtered_params</tt> -- This works just like Rails's
+filter_parameter_logging+ method. Set it to a symbol, string, or array of
them, and Samuel will filter the value of query parameters that have any of
these patterns as a substring by replacing the value with
<tt>[FILTERED]</tt> in your logs. By default, no filtering is enabled.

Samuel logs successful HTTP requests at the +INFO+ level; Failed requests log at
the +WARN+ level. This isn't currently configurable, but it's on the list.
Expand Down
10 changes: 5 additions & 5 deletions Rakefile
Expand Up @@ -7,18 +7,18 @@ begin
gem.name = "samuel"
gem.version = "0.2.1"
gem.summary = %Q{An automatic logger for HTTP requests in Ruby}
gem.description = %Q{An automatic logger for HTTP requests in Ruby. Adds Net::HTTP request logging to your Rails logs, and more.}
gem.description = %Q{An automatic logger for HTTP requests in Ruby, supporting the Net::HTTP and HTTPClient client libraries.}
gem.email = "chris@kampers.net"
gem.homepage = "http://github.com/chrisk/samuel"
gem.authors = ["Chris Kampmeier"]
gem.rubyforge_project = "samuel"
gem.add_development_dependency "shoulda"
gem.add_development_dependency "yard"
gem.add_development_dependency "mocha"
gem.add_development_dependency "httpclient"
gem.add_development_dependency "fakeweb"
end
rescue LoadError
puts "Jeweler (or a dependency) not available. Install it with: sudo gem install jeweler"
puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
end

require 'rake/testtask'
Expand All @@ -41,7 +41,7 @@ begin
end
rescue LoadError
task :rcov do
abort "RCov is not available. In order to run rcov, you must: sudo gem install rcov"
abort "RCov is not available. In order to run rcov, you must: gem install rcov"
end
end

Expand All @@ -54,6 +54,6 @@ begin
YARD::Rake::YardocTask.new
rescue LoadError
task :yardoc do
abort "YARD is not available. In order to run yardoc, you must: sudo gem install yard"
abort "YARD is not available. In order to run yardoc, you must: gem install yard"
end
end
24 changes: 11 additions & 13 deletions lib/samuel.rb
@@ -1,18 +1,21 @@
require "logger"
require "net/http"
require "net/https"
require "benchmark"
require "forwardable"

require "samuel/net_http"
require "samuel/request"
require "samuel/loader"
require "samuel/diary"
require "samuel/driver_patches/http_client"
require "samuel/driver_patches/net_http"
require "samuel/log_entries/base"
require "samuel/log_entries/http_client"
require "samuel/log_entries/net_http"


module Samuel
extend self

VERSION = "0.2.1"

attr_writer :config, :logger
attr_writer :logger, :config

def logger
@logger = nil if !defined?(@logger)
Expand All @@ -29,12 +32,6 @@ def config
Thread.current[:__samuel_config] ? Thread.current[:__samuel_config] : @config
end

def log_request(http, request, &block)
request = Request.new(http, request, block)
request.perform_and_log!
request.response
end

def with_config(options = {})
original_config = config.dup
nested = !Thread.current[:__samuel_config].nil?
Expand All @@ -48,7 +45,8 @@ def reset_config
Thread.current[:__samuel_config] = nil
@config = {:label => nil, :labels => {"" => "HTTP"}, :filtered_params => []}
end

end


Samuel.reset_config
Samuel::Loader.apply_driver_patches
29 changes: 29 additions & 0 deletions lib/samuel/diary.rb
@@ -0,0 +1,29 @@
module Samuel
module Diary
extend self

def record_request(http, request, time_requested)
@requests ||= []
@requests.push({:request => request, :time_requested => time_requested})
end

def record_response(http, request, response, time_responded)
time_requested = @requests.detect { |r| r[:request] == request }[:time_requested]
@requests.reject! { |r| r[:request] == request }
log_request_and_response(http, request, response, time_requested, time_responded)
end

private

def log_request_and_response(http, request, response, time_started, time_ended)
log_entry_class = case http.class.to_s
when "Net::HTTP" then LogEntries::NetHttp
when "HTTPClient" then LogEntries::HttpClient
else raise NotImplementedError
end
log_entry = log_entry_class.new(http, request, response, time_started, time_ended)
log_entry.log!
end

end
end
54 changes: 54 additions & 0 deletions lib/samuel/driver_patches/http_client.rb
@@ -0,0 +1,54 @@
module Samuel
module DriverPatches

module HTTPClient
def self.included(klass)
methods_to_wrap = %w(initialize do_get_block do_get_stream)
methods_to_wrap.each do |method|
klass.send(:alias_method, "#{method}_without_samuel", method)
klass.send(:alias_method, method, "#{method}_with_samuel")
end
end

def initialize_with_samuel(*args)
initialize_without_samuel(*args)
@request_filter << LoggingFilter.new(self)
end

def do_get_block_with_samuel(req, proxy, conn, &block)
begin
do_get_block_without_samuel(req, proxy, conn, &block)
rescue Exception => e
Samuel::Diary.record_response(self, req, e, Time.now)
raise
end
end

def do_get_stream_with_samuel(req, proxy, conn)
begin
do_get_stream_without_samuel(req, proxy, conn)
rescue Exception => e
Samuel::Diary.record_response(self, req, e, Time.now)
raise
end
end

class LoggingFilter
def initialize(http_client_instance)
@http_client_instance = http_client_instance
end

def filter_request(request)
Samuel::Diary.record_request(@http_client_instance, request, Time.now)
end

def filter_response(request, response)
Samuel::Diary.record_response(@http_client_instance, request, response, Time.now)
nil # this returns command symbols like :retry, etc.
end
end
end

end
end

42 changes: 42 additions & 0 deletions lib/samuel/driver_patches/net_http.rb
@@ -0,0 +1,42 @@
module Samuel
module DriverPatches

module NetHTTP
def self.included(klass)
methods_to_wrap = %w(request connect)
methods_to_wrap.each do |method|
klass.send(:alias_method, "#{method}_without_samuel", method)
klass.send(:alias_method, method, "#{method}_with_samuel")
end
end

def request_with_samuel(request, body = nil, &block)
Samuel::Diary.record_request(self, request, Time.now)

response, exception_raised = nil, false
begin
response = request_without_samuel(request, body, &block)
rescue Exception => response
exception_raised = true
end

Samuel::Diary.record_response(self, request, response, Time.now)

raise response if exception_raised
response
end

def connect_with_samuel
connect_without_samuel
rescue Exception => response
fake_request = Object.new
def fake_request.path; ""; end
def fake_request.method; "CONNECT"; end
Samuel::Diary.record_request(self, fake_request, Time.now)
Samuel::Diary.record_response(self, fake_request, response, Time.now)
raise
end
end

end
end
19 changes: 19 additions & 0 deletions lib/samuel/loader.rb
@@ -0,0 +1,19 @@
module Samuel
module Loader
extend self

def apply_driver_patches
loaded = { :net_http => defined?(Net::HTTP),
:http_client => defined?(HTTPClient) }

Net::HTTP.send(:include, DriverPatches::NetHTTP) if loaded[:net_http]
HTTPClient.send(:include, DriverPatches::HTTPClient) if loaded[:http_client]

if loaded.values.none?
require 'net/http'
apply_driver_patches
end
end

end
end
76 changes: 76 additions & 0 deletions lib/samuel/log_entries/base.rb
@@ -0,0 +1,76 @@
module Samuel
module LogEntries

class Base
def initialize(http, request, response, time_requested, time_responded)
@http, @request, @response = http, request, response
@seconds = time_responded - time_requested
end

def log!
Samuel.logger.add(log_level, log_message)
end


protected

def log_message
bold = "\e[1m"
blue = "\e[34m"
underline = "\e[4m"
reset = "\e[0m"
" #{bold}#{blue}#{underline}#{label} request (#{milliseconds}ms) " +
"#{response_summary}#{reset} #{method} #{uri}"
end

def milliseconds
(@seconds * 1000).round
end

def uri
"#{scheme}://#{host}#{port_if_not_default}#{path}#{'?' if query}#{filtered_query}"
end

def label
return Samuel.config[:label] if Samuel.config[:label]

pair = Samuel.config[:labels].detect { |domain, label| host.include?(domain) }
pair[1] if pair
end

def response_summary
if @response.is_a?(Exception)
@response.class
else
"[#{status_code} #{status_message}]"
end
end

def log_level
error? ? Logger::WARN : Logger::INFO
end

def ssl?
scheme == 'https'
end

def filtered_query
return "" if query.nil?
patterns = [Samuel.config[:filtered_params]].flatten
patterns.map { |pattern|
pattern_for_regex = Regexp.escape(pattern.to_s)
[/([^&]*#{pattern_for_regex}[^&=]*)=(?:[^&]+)/, '\1=[FILTERED]']
}.inject(query) { |filtered, filter| filtered.gsub(*filter) }
end

def port_if_not_default
if (!ssl? && port == 80) || (ssl? && port == 443)
""
else
":#{port}"
end
end
end

end
end

0 comments on commit d5a056c

Please sign in to comment.