Ent Historical Metrics

Sidekiq Enterprise v1.1+ has the ability to send queue processing metrics to Statsd for dashboards and historical reporting.

Configuration

In your initializer, add this:

# gem install statsd-ruby
statsd = Statsd.new('127.0.0.1', 8125)
statsd.namespace = 'myapp'

Sidekiq.configure_server do |config|
  # statsd must be something which quacks like a statsd-ruby instance
  # history is captured every 30 seconds by default
  config.save_history(statsd, 30)
end

Metrics

Sidekiq Enterprise sends the following metrics:

• sidekiq.processed - Number of job executions completed (success or failure)
• sidekiq.failures - Number of job executions which raised an error
• sidekiq.enqueued - Total Size of all known queues
• sidekiq.retries - Total Retries Size
• sidekiq.dead - Total Dead Size
• sidekiq.scheduled - Total Scheduled Size
• sidekiq.busy - Total Busy Size
• sidekiq.enqueued.#{x} - Current Size of queue x

The statsd namespace will be prepended to each metric, e.g. "myapp.sidekiq.enqueued".

Custom

You can add custom history metrics by passing a block to save_history which collects more metrics. Here we are adding the latency for the bulk and critical queues:

Sidekiq.configure_server do |config|
  config.save_history(statsd, 30) do |s|
    s.batch do
      %w(bulk critical).each do |qname|
        q = Sidekiq::Queue.new(qname)
        s.gauge("sidekiq.latency.#{q.name}", q.latency)
      end
    end
  end
end

The block will be passed a Statsd instance which quacks like the normal statsd client, e.g. dogstatsd-ruby.