Skip to content

Commit

Permalink
Added explicit sorting of requests when invoking the request_log:prin…
Browse files Browse the repository at this point in the history
…t rake task to ensure chronological ordering. Added a missing require of timeout to make the tests pass. Moved README and CHANGELOG to markdown format. Bumped version to 0.1.2
  • Loading branch information
peter committed Oct 25, 2011
1 parent 213b9d5 commit 2b6f5b0
Show file tree
Hide file tree
Showing 8 changed files with 44 additions and 14 deletions.
1 change: 1 addition & 0 deletions .gitignore
Expand Up @@ -2,3 +2,4 @@ pkg/*
*.gem
.bundle
doc/*
.rvmrc
7 changes: 0 additions & 7 deletions CHANGELOG

This file was deleted.

13 changes: 13 additions & 0 deletions CHANGELOG.md
@@ -0,0 +1,13 @@
# CHANGELOG

## 0.1.2 (2011-10-25)

* Added explicit sorting by time to Db.filtered_requests method to ensure that the `request_log:print` rake task outputs requests in chronological order.

## 0.1.0

* Removed summary field from log data since it only contained redundant data and we want to trim the size of the log.

## 0.0.1

* Initial version
2 changes: 1 addition & 1 deletion Gemfile.lock
@@ -1,7 +1,7 @@
PATH
remote: .
specs:
request_log (0.1.0)
request_log (0.1.2)

GEM
remote: http://rubygems.org/
Expand Down
30 changes: 26 additions & 4 deletions README.rdoc → README.md
@@ -1,86 +1,108 @@
= Request Log
# Request Log

Request Log is a Rack middleware for logging web requests to MongoDB. Each web request becomes a document in MongoDB
with fields like path, method, params, status, time, ip, runtime etc.
The gem offers support for monitoring the time overhead (usually very small) that the logging incurs.
The advantages of logging to MongoDB over logging to a plain text file are huge because of the query
capabilities of MongoDB. Here is an example of what a log document can look like:

```
method: "GET"
path: "/"
ip: "10.218.1.177"
time: 2010-10-28 21:43:38 UTC
params: {"hello_world"=>"1"}
status: 200
runtime: 0.000303
```

You can easily customize which fields are stored in the log.

== Installation
## Installation

Add gem dependencies with appropriate version numbers to your Gemfile (assuming you use Bundler):

```
gem 'mongo', '~> version known to work'
gem 'bson_ext', '~> version known to work'
gem 'request_log', '~> version known to work'
```

Install with:

```
bundle install
```

Note that it's up to your application how it wants to connect to MongoDB (if at all) and the suggested
mongo and bson_ext gems are just suggestions.

Next you need to setup a MongoDB connection. Here is a MongoHQ example that in Rails would belong in config/initializers/request_log.rb:

```
if ENV['MONGOHQ_URL']
require 'uri'
require 'mongo'
uri = URI.parse(ENV['MONGOHQ_URL'])
connection = Mongo::Connection.from_uri(uri.to_s)
RequestLog::Db.mongo_db = connection.db(uri.path.gsub(/^\//, ''))
end
```

MongoDB recommends using capped collections for logging as they have good write performance. Here is an example of how to do log rotation when the size hits 20GB (this step is optional, note that the command may take a while):

```
RequestLog::Db.mongo_db.create_collection("requests", :capped => true, :size => 20000000000)
```

Now setup the Middleware in your config.ru file:

```
use RequestLog::Middleware
```

Here is an example of how you can customize the middleware:

```
use RequestLog::Middleware,
:logger => lambda { |data| RequestLog::Db.requests.insert(data.attributes.except(:runtime)) },
:timeout => 0.5
```

In order to use the Rake tasks you need to make sure you have the MongoDB connection setup and that you
require the tasks in your Rakefile, like this:

```
require 'request_log'
require 'config/initializers/request_log.rb' # The file where you setup the mongo db connection
require 'request_log/tasks'
```

== Accessing the logs
## Accessing the logs

You can tail the log like this:

```
rake request_log:tail
```

If you want to query the log and print a certain time period you can use request_log:print:

```
rake request_log:print from="2010-10-28 17:06:08" to="2010-10-28 17:06:10" conditions='status: 200'
```

If you are using MONGOHQ, remember to set the MONGOHQ_URL environment variable.

== Profiling
## Profiling

To monitor the time consumption and reliability of the MongoDB logging you can use the RequestLog::Profiler class.
It records number of failed and successful loggings, average and maximum logging times etc. To persist the profiling
information to MongoDB you can configure the profiler like this:

```
RequestLog::Profiler.persist_enabled = true
RequestLog::Profiler.persist_frequency = 1000 # persist profiling info every 1000 requests
```

The profiling info will then be written to a table (request_log_profiling) in the MongoDB database.
1 change: 1 addition & 0 deletions lib/request_log.rb
@@ -1,3 +1,4 @@
require 'timeout'
require 'request_log/version'
require 'request_log/db'
require 'request_log/data'
Expand Down
2 changes: 1 addition & 1 deletion lib/request_log/db.rb
Expand Up @@ -28,7 +28,7 @@ def self.filtered_requests(start_time, end_time, conditions = {})
start_time = Time.parse(start_time).utc if start_time.is_a?(String)
end_time = Time.parse(end_time).utc if end_time.is_a?(String)
time_condition = {"time" => {"$gt" => start_time, "$lt" => end_time}}
requests.find(time_condition.merge(parse_conditions(conditions)))
requests.find(time_condition.merge(parse_conditions(conditions))).sort([:time, :ascending])
end

def self.print_requests(start_time, end_time, conditions = {})
Expand Down
2 changes: 1 addition & 1 deletion lib/request_log/version.rb
@@ -1,3 +1,3 @@
module RequestLog
VERSION = "0.1.1"
VERSION = "0.1.2"
end

0 comments on commit 2b6f5b0

Please sign in to comment.