Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Rack middleware for logging web requests to MongoDB
tree: abe51df99a
Failed to load latest commit information.
lib
spec Removed summary field from request log data since it only contained r…
.gitignore Added explicit sorting of requests when invoking the request_log:prin…
CHANGELOG.md Added explicit sorting of requests when invoking the request_log:prin…
Gemfile First commit
Gemfile.lock
README.md Added explicit sorting of requests when invoking the request_log:prin…
Rakefile Cleaning up Rakefile and specifying development dependencies in gemsp…
request_log.gemspec Cleaning up Rakefile and specifying development dependencies in gemsp…

README.md

Request Log

Request Log is a Rack middleware for logging web requests to MongoDB. Each web request becomes a document in MongoDB with fields like path, method, params, status, time, ip, runtime etc. The gem offers support for monitoring the time overhead (usually very small) that the logging incurs. The advantages of logging to MongoDB over logging to a plain text file are huge because of the query capabilities of MongoDB. Here is an example of what a log document can look like:

  method: "GET"
  path: "/"
  ip: "10.218.1.177"
  time: 2010-10-28 21:43:38 UTC
  params: {"hello_world"=>"1"}
  status: 200
  runtime: 0.000303

You can easily customize which fields are stored in the log.

Installation

Add gem dependencies with appropriate version numbers to your Gemfile (assuming you use Bundler):

  gem 'mongo', '~> version known to work'
  gem 'bson_ext', '~> version known to work'
  gem 'request_log', '~> version known to work'

Install with:

  bundle install

Note that it's up to your application how it wants to connect to MongoDB (if at all) and the suggested mongo and bson_ext gems are just suggestions.

Next you need to setup a MongoDB connection. Here is a MongoHQ example that in Rails would belong in config/initializers/request_log.rb:

  if ENV['MONGOHQ_URL']
      require 'uri'
      require 'mongo'      
      uri = URI.parse(ENV['MONGOHQ_URL'])
      connection = Mongo::Connection.from_uri(uri.to_s)
      RequestLog::Db.mongo_db = connection.db(uri.path.gsub(/^\//, ''))
  end

MongoDB recommends using capped collections for logging as they have good write performance. Here is an example of how to do log rotation when the size hits 20GB (this step is optional, note that the command may take a while):

  RequestLog::Db.mongo_db.create_collection("requests", :capped => true, :size => 20000000000)

Now setup the Middleware in your config.ru file:

  use RequestLog::Middleware

Here is an example of how you can customize the middleware:

  use RequestLog::Middleware,
      :logger => lambda { |data| RequestLog::Db.requests.insert(data.attributes.except(:runtime)) },
      :timeout => 0.5

In order to use the Rake tasks you need to make sure you have the MongoDB connection setup and that you require the tasks in your Rakefile, like this:

  require 'request_log'
  require 'config/initializers/request_log.rb' # The file where you setup the mongo db connection
  require 'request_log/tasks'

Accessing the logs

You can tail the log like this:

  rake request_log:tail

If you want to query the log and print a certain time period you can use request_log:print:

  rake request_log:print from="2010-10-28 17:06:08" to="2010-10-28 17:06:10" conditions='status: 200'

If you are using MONGOHQ, remember to set the MONGOHQ_URL environment variable.

Profiling

To monitor the time consumption and reliability of the MongoDB logging you can use the RequestLog::Profiler class. It records number of failed and successful loggings, average and maximum logging times etc. To persist the profiling information to MongoDB you can configure the profiler like this:

  RequestLog::Profiler.persist_enabled = true
  RequestLog::Profiler.persist_frequency = 1000 # persist profiling info every 1000 requests

The profiling info will then be written to a table (request_log_profiling) in the MongoDB database.

Something went wrong with that request. Please try again.