Add global error handler feature

Changes.md

@@ -1,11 +1,13 @@
 3.0.0
 -----------
 
-Please see [Upgrading.md](Upgrading.md) for upgrade notes.
+Please see [Upgrading.md](Upgrading.md) for more comprehensive upgrade notes.
 
+- **Global Error Handlers** - blocks of code which process errors which
+  occur anywhere within Sidekiq, not just within middleware.
 - **Dead Job Queue** - jobs which run out of retries are now moved to a dead
   job queue.  These jobs must be retried manually or they will expire
-  after 6 months or 10,000 jobs.  The Web UI contains a Dead tab
+  after 6 months or 10,000 jobs.  The Web UI contains a "Dead" tab
   exposing these jobs.
 - **Remove official support for Ruby 1.9**  Things still might work but
   I no longer actively test on it.

Upgrading.md

@@ -22,3 +22,22 @@ changes a few data elements in Redis.  To upgrade cleanly:
   support policy is to support the current and previous major releases
   of Ruby and Rails.  As of February 2014, that's Ruby 2.1, Ruby 2.0, Rails 4.0
   and Rails 3.2.  I will accept PRs to fix issues found by users.
+
+## Error Service Providers
+
+If you previously provided a middleware to capture job errors, you
+should instead provide a global error handler with Sidekiq 3.0.  This
+ensures **any** error within Sidekiq will be logged appropriately, not
+just during job execution.
+
+```ruby
+if Sidekiq::VERSION < '3'
+  # old behavior
+else
+  Sidekiq.configure_server do |config|
+    config.error_handlers << Proc.new {|ex,context| ... }
+  end
+end
+```
+
+Your error handler must respond to `call(exception, context_hash)`.

lib/sidekiq.rb

@@ -4,7 +4,6 @@
 require 'sidekiq/client'
 require 'sidekiq/worker'
 require 'sidekiq/redis_connection'
-require 'sidekiq/util'
 require 'sidekiq/api'
 
 require 'json'
@@ -20,6 +19,7 @@ module Sidekiq
     :environment => nil,
     :timeout => 8,
     :profile => false,
+    :error_handlers => [],
   }
 
   def self.❨╯°□°❩╯︵┻━┻
@@ -117,6 +117,10 @@ def self.poll_interval=(interval)
     self.options[:poll_interval] = interval
   end
 
+  def self.error_handlers
+    self.options[:error_handlers]
+  end
+
 end
 
 require 'sidekiq/extensions/class_methods'

lib/sidekiq/cli.rb

@@ -310,7 +310,7 @@ def parse_options(argv)
     def initialize_logger
       Sidekiq::Logging.initialize_logger(options[:logfile]) if options[:logfile]
 
-      Sidekiq.logger.level = Logger::DEBUG if options[:verbose]
+      Sidekiq.logger.level = ::Logger::DEBUG if options[:verbose]
     end
 
     def write_pid

lib/sidekiq/exception_handler.rb

@@ -1,10 +1,29 @@
+require 'sidekiq'
+
 module Sidekiq
   module ExceptionHandler
 
+    class Logger
+      def call(ex, ctxHash)
+        Sidekiq.logger.warn(ctxHash) if !ctxHash.empty?
+        Sidekiq.logger.warn ex
+        Sidekiq.logger.warn ex.backtrace.join("\n") unless ex.backtrace.nil?
+      end
+
+      # Set up default handler which just logs the error
+      Sidekiq.error_handlers << Sidekiq::ExceptionHandler::Logger.new
+    end
+
     def handle_exception(ex, ctxHash={})
-      Sidekiq.logger.warn(ctxHash) if !ctxHash.empty?
-      Sidekiq.logger.warn ex
-      Sidekiq.logger.warn ex.backtrace.join("\n") unless ex.backtrace.nil?
+      Sidekiq.error_handlers.each do |handler|
+        begin
+          handler.call(ex, ctxHash)
+        rescue => ex
+          Sidekiq.logger.error "!!! ERROR HANDLER THREW AN ERROR !!!"
+          Sidekiq.logger.error ex
+          Sidekiq.logger.error ex.backtrace.join("\n") unless ex.backtrace.nil?
+        end
+      end
     end
 
   end

lib/sidekiq/processor.rb

@@ -139,7 +139,7 @@ def retry_and_suppress_exceptions(max_retries = 2)
           sleep(1)
           retry
         else
-          Sidekiq.logger.info {"Exhausted #{max_retries} retries due to Redis timeouts: #{e.inspect}"}
+          handle_exception(e, { :message => "Exhausted #{max_retries} retries"})
         end
       end
     end

test/test_exception_handler.rb

@@ -1,5 +1,4 @@
 require 'helper'
-require 'sidekiq'
 require 'sidekiq/exception_handler'
 require 'stringio'
 require 'logger'