Skip to content
Log outgoing HTTP requests in ruby
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
gemfiles basic rubocop fixes Apr 26, 2018
lib version bump 1.2.2; rubocop Mar 15, 2019
spec version bump 1.2.2; rubocop Mar 15, 2019
.gitignore support HTTP version 3 + update travis config Feb 18, 2018
.rspec updated rspec Sep 21, 2015
.rubocop.yml
.rubocop_todo.yml
.ruby-version ruby 2.4.2 Nov 2, 2017
.travis.yml bumped travis ruby versions 2.4.5 and 2.5.3 Jan 24, 2019
CHANGELOG.md
Gemfile
Gemfile.lock
Guardfile cleaned up and modernized specs (WIP) Dec 10, 2018
MIT-LICENSE
README.md
Rakefile
httplog.gemspec version bump 1.2.2; rubocop Mar 15, 2019

README.md

httplog

Gem Version Build Status Code Climate Release Version

Log outgoing HTTP requests made from your application. Helps with debugging pesky API error responses, or just generally understanding what's going on under the hood.

Requires ruby >= 2.3.

This gem works with the following ruby modules and libraries:

These libraries are at least partially supported, where they use one of the above as adapters, but not explicitly tested - YMMV:

In theory, it should also work with any library built on top of these. But the difference between theory and practice is bigger in practice than in theory.

This is very much a development and debugging tool; it is not recommended to use this in a production environment as it is monkey-patching the respective HTTP implementations. You have been warned - use at your own risk.

Installation

gem install httplog

Usage

require 'httplog' # require this *after* your HTTP gem of choice

By default, this will log all outgoing HTTP requests and their responses to $stdout on DEBUG level.

Notes on content types

  • Binary data from response bodies (as indicated by the Content-Type header)is not logged.
  • Text data (text/* and most application/* types) is encoded as UTF-8, with invalid characters replaced. If you need to inspect raw non-UTF data exactly as sent over the wire, this tool is probably not for you.

Configuration

You can override the following default options:

HttpLog.configure do |config|

  # Enable or disable all logging
  config.enabled = true

  # You can assign a different logger
  config.logger = Logger.new($stdout)

  # I really wouldn't change this...
  config.severity = Logger::Severity::DEBUG

  # Tweak which parts of the HTTP cycle to log...
  config.log_connect   = true
  config.log_request   = true
  config.log_headers   = false
  config.log_data      = true
  config.log_status    = true
  config.log_response  = true
  config.log_benchmark = true

  # ...or log all request as a single line by setting this to `true`
  config.compact_log = false

  # You can also log in JSON format
  config.json_log = false

  # Prettify the output - see below
  config.color = false

  # Limit logging based on URL patterns
  config.url_whitelist_pattern = nil
  config.url_blacklist_pattern = nil
end

If you want to use this in a Rails app, I'd suggest configuring this specifically for each environment. A global initializer is not a good idea since HttpLog will be undefined in production. Because you're not using this in production, right? :)

# config/environments/development.rb

HttpLog.configure do |config|
  config.logger = Rails.logger
end

You can colorize the output to make it stand out in your logfile, either with a single color for the text:

HttpLog.configure do |config|
  config.color = :red
end

Or with a color hash for text and background:

HttpLog.configure do |config|
  config.color = {color: :black, background: :yellow}
end

For more color options please refer to the rainbow documentation

Compact logging

If the log is too noisy for you, but you don't want to completely disable it either, set the compact_log option to true. This will log each request in a single line with method, request URI, response status and time, but no data or headers. No need to disable any other options individually.

JSON logging

If you want to log HTTP requests in a JSON format, set the json_log option to true. You can combine this with compact_log to only log the basic request metrics without headers and bodies.

Example

With the default configuration, the log output might look like this:

D, [2012-11-21T15:09:03.532970 #6857] DEBUG -- : [httplog] Connecting: localhost:80
D, [2012-11-21T15:09:03.533877 #6857] DEBUG -- : [httplog] Sending: GET http://localhost:9292/index.html
D, [2012-11-21T15:09:03.534499 #6857] DEBUG -- : [httplog] Status: 200
D, [2012-11-21T15:09:03.534544 #6857] DEBUG -- : [httplog] Benchmark: 0.00057 seconds
D, [2012-11-21T15:09:03.534578 #6857] DEBUG -- : [httplog] Response:
<html>
  <head>
    <title>Test Page</title>
  </head>
  <body>
    <h1>This is the test page.</h1>
  </body>
</html>

With compact_log enabled, the same request might look like this:

[httplog] GET http://localhost:9292/index.html completed with status code 200 in 0.00057 seconds

With json_log enabled:

[httplog] {"method":"GET","url":"localhost:80","request_body":null, "request_headers":{"foo":"bar"}, "response_code":200,"response_body":"<html>\n      <head>\n        <title>Test Page</title>\n      </head>\n      <body>\n        <h1>This is the test page.</h1>\n      </body>\n    </html>","response_headers":{"foo":"bar"},"benchmark":0.00057}

And with json_log and compact_log enabled:

[httplog] {"method":"GET","url":"localhost:80","response_code":200,"benchmark":0.00057}

Known Issues

Following are some known quirks and issues with particular libraries. If you know a workaround or have a suggestion for a fix, please open an issue or, even better, submit a pull request!

  • Requests types other than GET and POST have not been explicitly tested. They may or may not be logged, depending on the implementation details of the underlying library. If they are not for a particular library, please feel free to open an issue with the details.

  • When using OpenURI, the reading of the HTTP response body is deferred, so it is not available for logging. This will be noted in the logging statement:

      D, [2012-11-21T15:09:03.547005 #6857] DEBUG -- : [httplog] Connecting: localhost:80
      D, [2012-11-21T15:09:03.547938 #6857] DEBUG -- : [httplog] Sending: GET http://localhost:9292/index.html
      D, [2012-11-21T15:09:03.548615 #6857] DEBUG -- : [httplog] Status: 200
      D, [2012-11-21T15:09:03.548662 #6857] DEBUG -- : [httplog] Benchmark: 0.000617 seconds
      D, [2012-11-21T15:09:03.548695 #6857] DEBUG -- : [httplog] Response: (not available yet)
    
  • When using HTTPClient, the TCP connection establishment will be logged after the HTTP request and headers, due to the way HTTPClient is organized.

     D, [2012-11-22T18:39:46.031698 #12800] DEBUG -- : [httplog] Sending: GET http://localhost:9292/index.html
     D, [2012-11-22T18:39:46.031756 #12800] DEBUG -- : [httplog] Header: accept: */*
     D, [2012-11-22T18:39:46.031788 #12800] DEBUG -- : [httplog] Header: foo: bar
     D, [2012-11-22T18:39:46.031942 #12800] DEBUG -- : [httplog] Connecting: localhost:9292
     D, [2012-11-22T18:39:46.033409 #12800] DEBUG -- : [httplog] Status: 200
     D, [2012-11-22T18:39:46.033483 #12800] DEBUG -- : [httplog] Benchmark: 0.001562 seconds
    
  • Also when using HTTPClient, make sure you include httplog after httpclient in your Gemfile.

  • When using Ethon or Patron, and any library based on them (such as Typhoeus), the TCP connection is not logged (since it's established by libcurl).

  • Benchmarking only covers the time between starting the HTTP request and receiving the response. It does not cover the time it takes to establish the TCP connection.

  • When using REST Client, POST requests might be missing the requests data. See #54 for details.

Running the specs

Make sure you have the necessary dependencies installed by running bundle install. Then simply run bundle exec rspec spec. This will launch a simple rack server on port 9292 and run all tests locally against that server.

Contributing

If you have any issues with or feature requests for httplog, please open an issue on GitHub or fork the project and send a pull request. Please include passing specs with all pull requests.

Contributors

Thanks to these fine folks for contributing pull requests:

You can’t perform that action at this time.