Monitoring

Sidekiq comes with a Sinatra application that can display the current state of a Sidekiq installation.

1. Rails 3 and 4

Add `sinatra` (and `sprockets` if you are on Rails 3.0) to your Gemfile

# if you require 'sinatra' you get the DSL extended to Object
gem 'sinatra', '>= 1.3.0', :require => nil

Add the following to your `config/routes.rb`:

require 'sidekiq/web'
mount Sidekiq::Web => '/sidekiq'

1. Security

In a production application you’ll likely want to protect access to this information. You can use the constraints feature of routing (in the config/routes.rb file) to accomplish this:

1. Devise

Allow any authenticated `User`

# config/routes.rb
authenticate :user do
  mount Sidekiq::Web => '/sidekiq'
end

Same as above but also ensures that `User#admin?` returns true

# config/routes.rb
authenticate :user, lambda { |u| u.admin? } do
  mount Sidekiq::Web => '/sidekiq'
end

1. Authlogic

# lib/admin_constraint.rb
class AdminConstraint
  def matches?(request)
    return false unless request.cookies['user_credentials'].present?
    user = User.find_by_persistence_token(request.cookies['user_credentials'].split(':')[0])
    user && user.admin?
  end
end

# config/routes.rb
require "admin_constraint"
mount Sidekiq::Web => '/sidekiq', :constraints => AdminConstraint.new

1. Restful Authentication

Checks a `User` model instance that responds to `admin?`

# lib/admin_constraint.rb
class AdminConstraint
  def matches?(request)
    return false unless request.session[:user_id]
    user = User.find request.session[:user_id]
    user && user.admin?
  end
end

# config/routes.rb
require 'sidekiq/web'
require 'admin_constraint'
mount Sidekiq::Web => '/sidekiq', :constraints => AdminConstraint.new

1. Custom External Authentication

class AuthConstraint
  def self.admin?(request)
    return false unless (cookie = request.cookies['auth'])

    Rails.cache.fetch(cookie['user'], :expires_in => 1.minute) do
      auth_data = JSON.parse(Base64.decode64(cookie['data']))
      response = HTTParty.post(Auth.validate_url, :query => auth_data)

      response.code == 200 && JSON.parse(response.body)['roles'].to_a.include?('Admin')
    end
  end
end

# config/routes.rb
constraints lambda {|request| AuthConstraint.admin?(request) } do
  mount Sidekiq::Web => '/admin/sidekiq'
end

1. Standalone

Here’s an example `config.ru` for booting Sidekiq::Web in your choice of Rack server:

require 'sidekiq'

Sidekiq.configure_client do |config|
  config.redis = { :size => 1 }
end

require 'sidekiq/web'
run Sidekiq::Web

You can mount sidekiq to existing Rack (Sinatra) application as well:

require 'your_app'

require 'sidekiq/web'
run Rack::URLMap.new('/' => Sinatra::Application, '/sidekiq' => Sidekiq::Web)

---

If you do everything right, you should see this in your browser:


1. Standalone with GitHub OAuth

This configuration will allow access only to members of your GitHub organization. Start by [creating a new OAuth application](https://github.com/settings/applications/new). You will also need to create a secure session cookie:

openssl rand -base64 48

require 'sinatra_auth_github'

module Sidekiq
  class Web
    use Rack::Session::Cookie, :secret => ENV['RACK_SESSION_COOKIE']

    set :github_options, {
      :scopes    => "user",
      :client_id => ENV['GITHUB_KEY'],
      :secret    => ENV['GITHUB_SECRET']
    }

    register Sinatra::Auth::Github

    before do
      authenticate!
      github_organization_authenticate!(ENV['GITHUB_ORG'])
    end

    get '/logout' do
      logout!
    end
  end
end

1. Standalone with Basic Auth

require 'sidekiq'

Sidekiq.configure_client do |config|
  config.redis = { :size => 1 }
end

require 'sidekiq/web'

Sidekiq::Web.use Rack::Auth::Basic do |username, password|
  username == 'some_user' && password == 'some_pass'
end 

run Sidekiq::Web

1. Nagios

Below is a collection of nagios checks that includes [check_sidekiq_queue](https://github.com/wanelo/nagios-checks/blob/master/check_sidekiq_queue) script, which validates that a given queue depth is within a particular range. It’s a simple shell script that uses redis-cli command line tool, and does not have any dependency on ruby.

1. Scout


The [Sidekiq Monitor plugin](https://scoutapp.com/plugin_urls/7111-sidekiq-monitor) for [Scout](http://scoutapp.com), a hosted server monitoring service, reports key metrics like enqueued, processed, and failed jobs. Triggers can be configured to alert when the metrics reach specified thresholds. A Scout account is required to use the plugin.

1. Monitoring Queue Backlog

At The Clymb, we use a simple HTTP endpoint with Pingdom to check the size of our Sidekiq ‘default’ queue backlog. Put this in `config/routes.rb`:

require 'sidekiq/api'
match "queue-status" => proc { [200, {"Content-Type" => "text/plain"}, [Sidekiq::Queue.new.size < 100 ? "OK" : "UHOH" ]] }

Now when you hit http://yourapp.com/queue-status, the body of the response will be either ‘OK’ or ‘UHOH’. We have a Pingdom check every minute which fires off an email if the response == ‘UHOH’.

1. Monitoring Queue Latency

1. Using a custom end-point

If you throw a lot of jobs into the queue, you can get false positives when monitoring the queue backlog. Instead, monitor the queue latency. Queue latency is the difference between when the oldest job was pushed onto the queue versus the current time. This code will check that jobs don’t spend more than 30 seconds enqueued. Put this in `config/routes.rb`:

require 'sidekiq/api'
match "queue-status" => proc { [200, {"Content-Type" => "text/plain"}, [Sidekiq::Queue.new.latency < 30 ? "OK" : "UHOH" ]] }

Now when you hit http://yourapp.com/queue-status, the body of the response will be either ‘OK’ or ‘UHOH’.

1. Using the built-in dashboard

Sidekiq provides a JSON formatted dashboard at [http://yourapp.com/sidekiq/dashboard/stats](http://yourapp.com/admin/sidekiq/dashboard/stats). You get this :

{
  "sidekiq": {
    "processed": "12345",
    "failed": "56",
    "busy": "25",
    "enqueued": "178",
    "scheduled": "0",
    "retries": "0",
    "default_latency": "12",
  },
  "redis": {
    "connected_clients": "120",
    "uptime_in_days": "35",
    "used_memory_human": "602.31M",
    "used_memory_peak_human": "1.01G"
  }
}