Refactored as per our previous discussions. Is there anything you'd like me to add to to the wiki/readme for how to use this functionality?
Also, would monkeypatching be the best way to inject into Sidekiq::Extensions::DelayedClass?
sidekiq_logger do |logger|
logger.level = Logger::WARN
logger.execution_threshold = 5
logging controls on a per-worker basis
It's been months since we chatted. Could you summarize the changes here and what you're looking to accomplish? I'm old and my memory is fading...
We're running > 5k background jobs per minute, most of which are ephemeral and we don't need logging for. Paying for a hosted rsyslog service can get costly when most of the chatter from Sidekiq isn't necessary. The PR lets you customize the logger per-worker, to raise the log level (WARN only shows failures), and to optionally set an execution threshold to help debug jobs or queues that are taking a long time to complete.
What we did was remove the middleware that adds the "started" and "finished" logging and kept the log level at info for all workers. We do about 50-100k jobs per minute during normal daily peak load and had the same motivation regarding cloud logging. IMO I don't know if this is needed, instead just do
config.server_middleware do |chain|
I'd rather not remove all of the logging, seems that could introduce unintended consequences and will give us less visibility, especially for jobs or queues that are causing congestion among the worker pool.
I've gotta side with Jon here. This feels overly complicated for Sidekiq core.