Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sidekiq worker crashes when Twitter library throws an exception, even if exception is rescued #2092

Closed
scottmartinnet opened this issue Dec 17, 2014 · 13 comments
Closed

Sidekiq worker crashes when Twitter library throws an exception, even if exception is rescued #2092

scottmartinnet opened this issue Dec 17, 2014 · 13 comments

Comments

@scottmartinnet
Copy link

I'm experiencing a strange issue with the sidekiq workers. A task I want to run from within Sidekiq uses the standard Twitter library (https://github.com/sferik/twitter) to make a call to the Twitter API which may throw an exception, and I have code in place to rescue that exception and ignore it. But whether or not the exception is rescued, the worker crashes.

I've cut out anything non-essential, and can still reproduce the crash. The code the worker is running looks like this:

  def test_exception
    begin
      # Code that causes the exception
    rescue StandardError
      Rails.logger.info "caught"
    end
  end

And when I run it, I can verify that the "caught" output occurs, and the job should finish normally. But instead, I get a stack trace of the rescued exception and a crashed worker.

Here's a log of the execution:

2014-12-17T22:20:11.117Z 15771 TID-ourr1r16w INFO: Running in ruby 2.1.4p265 (2014-10-27 revision 48166) [x86_64-darwin14.0]
2014-12-17T22:20:11.117Z 15771 TID-ourr1r16w INFO: Sidekiq Pro 1.9.1, commercially licensed.  Thanks for your support!
2014-12-17T22:20:11.117Z 15771 TID-ourr1r16w INFO: Starting processing, hit Ctrl-C to stop
2014-12-17T22:20:11.279Z 15771 TID-oursq9v7s INFO: Booting Sidekiq 3.3.0 with redis options {}
2014-12-17T22:20:11.280Z 15771 TID-ourr2701w INFO: Booting Sidekiq 3.3.0 with redis options {}
2014-12-17T22:20:11.285Z 15771 TID-oursuvm6c StreamWorker JID-8934f210300d84eab155738f INFO: start
StreamWorker#perform: id: 133, test_exception
caught
Sorry, that page does not exist
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/response/raise_error.rb:22:in `on_complete'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:9:in `block in call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:63:in `on_complete'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:8:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/request/url_encoded.rb:14:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/request/multipart.rb:13:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/request/multipart_with_file.rb:15:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:253:in `run_request'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:118:in `post'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/client.rb:131:in `request'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/client.rb:103:in `post'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/api/utils.rb:91:in `objects_from_response'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/api/users.rb:231:in `block in users'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/core_ext/enumerable.rb:5:in `block (2 levels) in threaded_map'
2014-12-17T22:20:11.916Z 15771 TID-oursuvq54 Sidekiq::Batch::Lifecycle JID-6bbf021b529ce38e62d8745f INFO: start
#<Sidekiq::Batch::Lifecycle:0x007f8b3f487188>
2014-12-17T22:20:11.917Z 15771 TID-oursuvm6c StreamWorker JID-8934f210300d84eab155738f INFO: done: 0.632 sec
2014-12-17T22:20:11.919Z 15771 TID-oursq9v7s WARN: {:context=>"Manager#processor_died died"}
2014-12-17T22:20:11.919Z 15771 TID-oursq9v7s WARN: task was terminated
2014-12-17T22:20:11.919Z 15771 TID-oursq9v7s WARN: /Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:34:in `terminate'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:345:in `each'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:345:in `cleanup'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:329:in `shutdown'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:164:in `run'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:130:in `block in start'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in `block in initialize'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in `block in get_thread'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in `block in create'
2014-12-17T22:20:11.920Z 15771 TID-oursuulw8 Sidekiq::Batch::Lifecycle JID-a88877184505faf633b984cb INFO: start
#<Sidekiq::Batch::Lifecycle:0x007f8b3f471bd0>
2014-12-17T22:20:11.921Z 15771 TID-oursq9v7s WARN: {:context=>"Manager#processor_died died"}
2014-12-17T22:20:11.921Z 15771 TID-oursq9v7s WARN: task was terminated
2014-12-17T22:20:11.922Z 15771 TID-oursuvq54 Sidekiq::Batch::Lifecycle JID-6bbf021b529ce38e62d8745f INFO: done: 0.006 sec
2014-12-17T22:20:11.922Z 15771 TID-oursq9v7s WARN: /Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:34:in `terminate'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:345:in `each'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:345:in `cleanup'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:329:in `shutdown'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:164:in `run'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:130:in `block in start'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in `block in initialize'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in `block in get_thread'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in `block in create'
2014-12-17T22:20:11.924Z 15771 TID-oursuulw8 Sidekiq::Batch::Lifecycle JID-a88877184505faf633b984cb INFO: done: 0.003 sec

Research has revealed a few possible approaches, which I've investigated: I've verified that nothing is extending Exception. #1896 appears to describe a very similar problem, but I am catching the exception. Exceptions that do not originate in this Twitter library do not exhibit this behavior. I'm running sidekiq 3.3.0 with sidekiq-pro 1.9.1 and celluloid 0.16.0.
@mperham
Copy link
Collaborator

mperham commented Dec 17, 2014

Can you share your actual worker code? If your worker is streaming, how long does it block for?

@scottmartinnet
Copy link
Author

No, there's no streaming. I've contrived a very simple test case. You'll need to supply valid key/secret/token/token_secret values if you want to actually run it, but that's all you should need. This is using revision 6ad7fe327e61ea7dfddf4ef200c7bd7df9aff0ee of sferik/twitter.

class TestWorker
  include Sidekiq::Worker

  def perform
    begin
      service = ::Twitter::REST::Client.new(
                                            :consumer_key => ENV['TWITTER_CONSUMER_KEY'],
                                            :consumer_secret => ENV['TWITTER_CONSUMER_SECRET'],
                                            :oauth_token => TwitterAccount.first.authentication.credentials.token,
                                            :oauth_token_secret => TwitterAccount.first.authentication.credentials.secret
                                            )

      service.users(['lsdkjfldsjfljkdfjkl']) # A non-existent user (this is what triggers the exception)
    rescue StandardError
      Rails.logger.info 'caught'
    end
  end
end

Run from console:

TestWorker.perform_async

Output:

2014-12-17T22:47:35.751Z 18932 TID-oup74zj60 INFO: Running in ruby 2.1.4p265 (2014-10-27 revision 48166) [x86_64-darwin14.0]
2014-12-17T22:47:35.751Z 18932 TID-oup74zj60 INFO: Sidekiq Pro 1.9.1, commercially licensed.  Thanks for your support!
2014-12-17T22:47:35.751Z 18932 TID-oup74zj60 INFO: Starting processing, hit Ctrl-C to stop
2014-12-17T22:47:35.818Z 18932 TID-oup8wcla8 INFO: Booting Sidekiq 3.3.0 with redis options {}
2014-12-17T22:47:35.818Z 18932 TID-oup7585ho INFO: Booting Sidekiq 3.3.0 with redis options {}
2014-12-17T22:47:35.822Z 18932 TID-oup8wdzt4 TestWorker JID-b90fec6c8b09b3872067ca38 INFO: start
Sorry, that page does not existcaught
2014-12-17T22:47:36.259Z 18932 TID-oup8wdzt4 TestWorker JID-b90fec6c8b09b3872067ca38 INFO: done: 0.437 sec

/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/response/raise_error.rb:22:in `on_complete'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:9:in `block in call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:63:in `on_complete'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:8:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/request/url_encoded.rb:14:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/request/multipart.rb:13:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/request/multipart_with_file.rb:15:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:253:in `run_request'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:118:in `post'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/client.rb:131:in `request'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/client.rb:103:in `post'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/api/utils.rb:91:in `objects_from_response'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/api/users.rb:231:in `block in users'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/core_ext/enumerable.rb:5:in `block (2 levels) in threaded_map'
2014-12-17T22:47:36.262Z 18932 TID-oup8wcla8 WARN: {:context=>"Manager#processor_died died"}
2014-12-17T22:47:36.263Z 18932 TID-oup8wcla8 WARN: task was terminated
2014-12-17T22:47:36.263Z 18932 TID-oup8wcla8 WARN: /Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:34:in `terminate'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:345:in `each'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:345:in `cleanup'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:329:in `shutdown'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:164:in `run'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:130:in `block in start'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in `block in initialize'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in `block in get_thread'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in `block in create'
2014-12-17T22:47:36.264Z 18932 TID-oup8wcla8 WARN: {:context=>"Manager#processor_died died"}
2014-12-17T22:47:36.264Z 18932 TID-oup8wcla8 WARN: task was terminated
2014-12-17T22:47:36.264Z 18932 TID-oup8wcla8 WARN: /Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:34:in `terminate'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:345:in `each'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:345:in `cleanup'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:329:in `shutdown'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:164:in `run'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:130:in `block in start'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in `block in initialize'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in `block in get_thread'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in `block in create'

It also intermittently produces this different error output:

2014-12-17T22:45:40.324Z 18507 TID-oxbnt922w INFO: Running in ruby 2.1.4p265 (2014-10-27 revision 48166) [x86_64-darwin14.0]
2014-12-17T22:45:40.325Z 18507 TID-oxbnt922w INFO: Sidekiq Pro 1.9.1, commercially licensed.  Thanks for your support!
2014-12-17T22:45:40.325Z 18507 TID-oxbnt922w INFO: Starting processing, hit Ctrl-C to stop
2014-12-17T22:45:40.399Z 18507 TID-oxbpihcbo INFO: Booting Sidekiq 3.3.0 with redis options {}
2014-12-17T22:45:40.400Z 18507 TID-oxbo3crh4 INFO: Booting Sidekiq 3.3.0 with redis options {}
2014-12-17T22:45:40.405Z 18507 TID-oxbpiie7k TestWorker JID-fd059918361f97575188e3fb INFO: start
Sorry, that page does not existcaught
2014-12-17T22:45:40.853Z 18507 TID-oxbpiie7k TestWorker JID-fd059918361f97575188e3fb INFO: done: 0.448 sec

/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/response/raise_error.rb:22:in `on_complete'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:9:in `block in call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:63:in `on_complete'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:8:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/request/url_encoded.rb:14:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/request/multipart.rb:13:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/request/multipart_with_file.rb:15:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:253:in `run_request'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:118:in `post'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/client.rb:131:in `request'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/client.rb:103:in `post'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/api/utils.rb:91:in `objects_from_response'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/rest/api/users.rb:231:in `block in users'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/bundler/gems/twitter-6ad7fe327e61/lib/twitter/core_ext/enumerable.rb:5:in `block (2 levels) in threaded_map'
2014-12-17T22:45:40.855Z 18507 TID-oxbpigvw0 WARN: Sidekiq died due to the following error, cannot recover, process exiting
2014-12-17T22:45:40.856Z 18507 TID-oxbpigvw0 WARN:
2014-12-17T22:45:40.856Z 18507 TID-oxbpigvw0 ERROR: !!! ERROR HANDLER THREW AN ERROR !!!
2014-12-17T22:45:40.856Z 18507 TID-oxbpigvw0 ERROR: undefined method `backtrace' for nil:NilClass
2014-12-17T22:45:40.856Z 18507 TID-oxbpigvw0 ERROR: /Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/exception_handler.rb:10:in `call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/exception_handler.rb:20:in `block in handle_exception'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/exception_handler.rb:18:in `each'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/exception_handler.rb:18:in `handle_exception'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/launcher.rb:32:in `actor_died'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:15:in `block in call'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in `block in task'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in `block in task'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in `block in initialize'
/Users/smartin/.rbenv/versions/2.1.4/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in `block in create'

@mperham
Copy link
Collaborator

mperham commented Dec 17, 2014

Any chance you can try the latest twitter gem version? Looks like you're running on an older version. raise_error.rb looks suspicious but doesn't exist in the latest version.

@scottmartinnet
Copy link
Author

I have some dependencies that make this difficult, but if that's the only way to get around this problem, I can go that route. But I'd rather figure out this problem, because I can't ensure that it won't happen in other libraries.

If it's a matter of just wanting to see the source for that revision, you can do this:

git clone git@github.com:sferik/twitter.git
cd twitter
git checkout 6ad7fe327e61ea7dfddf4ef200c7bd7df9aff0ee .

@mperham
Copy link
Collaborator

mperham commented Dec 17, 2014

Can you run Sidekiq master? It's got a fix for errors which have no
backtrace. I think that might be part of your problem.

On Wed, Dec 17, 2014 at 3:39 PM, scottmartinnet notifications@github.com
wrote:

I have some dependencies that make this difficult, but if that's the only
way to get around this problem, I can go that route. But I'd rather figure
out this problem, because I can't ensure that it won't happen in other
libraries.

If it's a matter of just wanting to see the source for that revision, you
can do this:

git clone git@github.com:sferik/twitter.git
cd twitter
git checkout 6ad7fe327e61ea7dfddf4ef200c7bd7df9aff0ee .

—
Reply to this email directly or view it on GitHub
#2092 (comment).

@scottmartinnet
Copy link
Author

I experience the same crash under Sidekiq master.

@mperham
Copy link
Collaborator

mperham commented Dec 18, 2014

I'm at a loss to explain then. Can you reproduce the problem in a Ruby snippet so I can experiment?

@scottmartinnet
Copy link
Author

What do you need beyond the above TestWorker code?

@mperham
Copy link
Collaborator

mperham commented Dec 18, 2014

I can't run that code because I don't have a twitter dev account.

@scottmartinnet
Copy link
Author

I've sent you some testing credentials via email.

@mperham
Copy link
Collaborator

mperham commented Dec 18, 2014

Works fine for me, I need a reproducible scenario:

2014-12-18T18:18:51.801Z 54319 TID-ovq71ebc4 TwitterWorker JID-f0e50e85317d51a0aba23715 INFO: start
Faraday::Builder is now Faraday::RackBuilder.
2014-12-18T18:18:52.048Z 54319 TID-ovq71ebc4 TwitterWorker JID-f0e50e85317d51a0aba23715 INFO: caught Sorry, that page does not exist
2014-12-18T18:18:52.048Z 54319 TID-ovq71ebc4 TwitterWorker JID-f0e50e85317d51a0aba23715 INFO: done: 0.247 sec

@scottmartinnet
Copy link
Author

I upgraded the Twitter library to the latest version as you suggested earlier and the crash is no longer happening. It seems this is hard enough to reproduce that I won't be very likely to encounter it again. That's good enough for me.

Were you using the Twitter library revision I specified instead of their master? That's the only thing I can think of that could be substantially different between our setups.

@mperham
Copy link
Collaborator

mperham commented Dec 18, 2014

Yes, I was using that ref. Glad it seems to be fixed now.

On Thu, Dec 18, 2014 at 11:17 AM, scottmartinnet notifications@github.com
wrote:

I upgraded the Twitter library to the latest version as you suggested
earlier and the crash is no longer happening. It seems this is hard enough
to reproduce that I won't be very likely to encounter it again. That's good
enough for me.

Were you using the Twitter library revision I specified instead of their
master? That's the only thing I can think of that could be substantially
different between our setups.

—
Reply to this email directly or view it on GitHub
#2092 (comment).

@mperham mperham closed this as completed Dec 27, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mperham @scottmartinnet