Skip to content
Easily record any metric in your system
Branch: master
Clone or download
Latest commit 8d5d0d7 Jun 3, 2011
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
autotest Record latencies Feb 21, 2011
lib Release v1.3.1 Jun 3, 2011
samples Release v1.2.0 Mar 29, 2011
spec
.gitignore Ignore Redis dump file Feb 21, 2011
.rvmrc Initial commit Feb 21, 2011
Gemfile
LICENSE
README.textile
Rakefile Initial hit implementation using both Memory and Redis versions Feb 21, 2011
counters.gemspec

README.textile

Counters

Easily record any metrics from anywhere within your code, using a very simple interface:

  • ping: When’s the last time we saw this thing?
  • hit: Increments a counter
  • magnitude: Measures numerical values
  • latency: Measures time intervals

Example

Let’s say you have a web crawler. There are a ton of things you can measure about your crawler: how many pages it processed (#hit), how many bytes you read (#magnitude), how long did it take to download the page (#latency), how long did it take to parse the raw HTML to a useable format (#latency).

require "uri"
require "open-uri"
require "counters"
require "nokogiri"

Counter = Counters::Redis.new(Redis.new, :namespace => "crawler", :base_key => "counters")

urls_to_crawl = ["http://blog.teksol.info/", "http://techcrunch.com/", "http://www.google.com/"]

while url = urls_to_crawl.pop
  Counter.ping "crawler"

  begin
    Counter.hit "urls_popped"

    puts "Fetching #{url}"
    raw_html = Counter.latency "download" do
      URI.parse(url).read
    end

    Counter.magnitude "bytes_in", raw_html.length

    parsed_html = Counter.latency "html_parsing" do
      Nokogiri::HTML(raw_html)
    end
  rescue
    Counter.hit "error"
  end
end

Other Backends

For testing purposes, there also exists a Counters::Memory. This would be good in test mode, for example. The counters are exposed through accessor methods returning a Hash.

You may log to a file, but be advised the file’s size grows very quickly. Counters are stored in the file, one per line, in an easily readable format.

$ irb -r counters
> Counter = Counters::File.new("counters.log")
 => #<Counters::File:0x00000101a18f18 @logger=#<Logger:0x00000101a18ef0 @progname=nil, @level=0, @default_formatter=#<Logger::Formatter:0x00000101a18ea0 @datetime_format=nil>, @formatter=#<Proc:0x00000101a18bd0@/Users/francois/Projects/counters/lib/counters/file.rb:15 (lambda)>, @logdev=#<Logger::LogDevice:0x00000101a18e28 @shift_size=1048576, @shift_age=0, @filename="counters.log", @dev=#<File:counters.log>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x00000101a18e00 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x00000101a18d88>>>>> 
> Counter.hit "crawler.page_read"
 => true
> Counter.magnitude "crawler.bytes_in", 9_921
 => true
> Counter.latency "crawler.processing" do sleep 0.3 ; end
 => true
> Counter.ping "crawler.alive"
 => true
> exit

$ cat counters.log
2011-02-21T09:46:21.296326000 - hit: crawler.page_read
2011-02-21T09:46:24.280388000 - magnitude: crawler.bytes_in 9921
2011-02-21T09:46:27.989183000 - latency: crawler.processing 0.3001821041107178s
2011-02-21T09:46:31.031969000 - ping: crawler.alive
2011-02-21T09:46:21.296326000 - hit: crawler.page_read
2011-02-21T09:46:24.280388000 - magnitude: crawler.bytes_in 13291
2011-02-21T09:46:27.989183000 - latency: crawler.processing 0.3123122982101s

You may also output your counters to StatsD. The only change that might be surprising is magnitudes are output as timer events. Magnitudes are used to record values (such as process RSS, packet sizes, etc).

Instantiate a StatsD instance:

Counter = Counters::StatsD.new("127.0.0.1", 8125, :namespace => "analyzer")

# Alternatively, use a URI:
Counter = Counters::StatsD.new(:url => "udp://127.0.0.1:8125", :namespace => "crawler")

See the file samples/crawler.rb for a more detailed example.

LICENSE

(The MIT License)

Copyright © 2008-2009 François Beausoleil (francois@teksol.info)

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
‘Software’), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

You can’t perform that action at this time.