Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SIGSEGV since updating to Ruby 2.1 #616

Closed
mipearson opened this issue Jan 9, 2014 · 6 comments
Closed

SIGSEGV since updating to Ruby 2.1 #616

mipearson opened this issue Jan 9, 2014 · 6 comments

Comments

@mipearson
Copy link

Looks like it's occuring on line 125 of delayed/backend/base.rb:

      def hook(name, *args)
        if payload_object.respond_to?(name)
          method = payload_object.method(name)
          method.arity == 0 ? method.call : method.call(self, *args) # SIGSEGV HERE
        end
      rescue DeserializationError
        # do nothing
      end

So, I'm guessing some fun deserialisation stuff.

Ruby trace follows:

$APPDIR/shared/bundle/ruby/2.1.0/gems/rollbar-0.10.14/lib/rollbar/delayed_job.rb:10: [BUG] Segmentation fault at 0x00000000000018
ruby 2.1.0p0 (2013-12-25 revision 44422) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0070 p:0029 s:0296 e:000294 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/rollbar-0.10.14/lib/rollbar/delayed_job.rb:10 [FINISH]
c:0069 p:---- s:0290 e:000289 CFUNC  :call
c:0068 p:0052 s:0285 e:000284 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:125
c:0067 p:0026 s:0279 e:000278 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:108
c:0066 p:0028 s:0276 e:000275 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:112 [FINISH]
c:0065 p:---- s:0273 e:000272 CFUNC  :call
c:0064 p:0010 s:0269 e:000268 LAMBDA $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60 [FINISH]
c:0063 p:---- s:0265 e:000264 CFUNC  :call
c:0062 p:0021 s:0261 e:000260 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65
c:0061 p:0111 s:0255 e:000254 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38
c:0060 p:0020 s:0249 E:001218 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:102
c:0059 p:0011 s:0246 e:000245 BLOCK  (eval):3
c:0058 p:0423 s:0244 e:000243 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/newrelic_rpm-3.7.1.180/lib/new_relic/agent/instrumentation/controller_instr
c:0057 p:0025 s:0229 E:0001a8 METHOD (eval):2
c:0056 p:0009 s:0224 e:000223 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:206
c:0055 p:0036 s:0222 E:0016f0 BLOCK  /usr/lib/ruby/2.1.0/timeout.rb:82 [FINISH]
c:0054 p:---- s:0217 e:000216 CFUNC  :catch
c:0053 p:0068 s:0213 E:001660 METHOD /usr/lib/ruby/2.1.0/timeout.rb:70
c:0052 p:0027 s:0203 E:0002b8 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:206
c:0051 p:0019 s:0201 e:000200 METHOD /usr/lib/ruby/2.1.0/benchmark.rb:294
c:0050 p:0023 s:0197 E:001510 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:205
c:0049 p:0009 s:0191 e:000190 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:267 [FINISH]
c:0048 p:---- s:0189 e:000188 CFUNC  :call
c:0047 p:0010 s:0184 e:000183 LAMBDA $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60 [FINISH]
c:0046 p:---- s:0180 e:000179 CFUNC  :call
c:0045 p:0021 s:0175 e:000174 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65
c:0044 p:0111 s:0169 e:000168 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38
c:0043 p:0029 s:0163 E:002000 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:267
c:0042 p:0007 s:0159 e:000158 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:189 [FINISH]
c:0041 p:---- s:0157 e:000156 CFUNC  :times
c:0040 p:0020 s:0154 e:000153 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:188
c:0039 p:0007 s:0148 e:000147 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:153
c:0038 p:0019 s:0146 e:000145 METHOD /usr/lib/ruby/2.1.0/benchmark.rb:294
c:0037 p:0013 s:0142 e:000141 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:152 [FINISH]
c:0036 p:---- s:0140 e:000139 CFUNC  :call
c:0035 p:0010 s:0136 e:000135 LAMBDA $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60 [FINISH]
c:0034 p:---- s:0132 e:000131 CFUNC  :call
c:0033 p:0021 s:0128 e:000127 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65
c:0032 p:0111 s:0122 e:000121 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38
c:0031 p:0014 s:0116 E:000170 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:151 [FINISH]
c:0030 p:---- s:0113 e:000112 CFUNC  :loop
c:0029 p:0007 s:0110 E:000cf8 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:150 [FINISH]
c:0028 p:---- s:0108 e:000107 CFUNC  :call
c:0027 p:0012 s:0104 e:000103 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/plugins/clear_locks.rb:7 [FINISH]
c:0026 p:---- s:0100 e:000099 CFUNC  :call
c:0025 p:0013 s:0096 e:000095 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78 [FINISH]
c:0024 p:---- s:0093 e:000092 CFUNC  :call
c:0023 p:0010 s:0089 e:000088 LAMBDA $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60 [FINISH]
c:0022 p:---- s:0085 e:000084 CFUNC  :call
c:0021 p:0010 s:0081 E:000cb8 LAMBDA $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78 [FINISH]
c:0020 p:---- s:0077 e:000076 CFUNC  :call
c:0019 p:0021 s:0073 e:000072 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65
c:0018 p:0111 s:0067 e:000066 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38
c:0017 p:0038 s:0061 E:000698 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:149
c:0016 p:0137 s:0058 e:000057 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:104
c:0015 p:0040 s:0052 e:000051 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:92 [FINISH]
c:0014 p:---- s:0049 e:000048 CFUNC  :call
c:0013 p:0061 s:0046 E:000868 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/application.rb:255 [FINISH]
c:0012 p:---- s:0044 e:000043 CFUNC  :call
c:0011 p:0061 s:0041 E:0026a8 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/application.rb:264
c:0010 p:0117 s:0036 e:000035 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/application.rb:296
c:0009 p:0209 s:0033 e:000032 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/controller.rb:73
c:0008 p:0009 s:0030 e:000029 BLOCK  $APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons.rb:197 [FINISH]
c:0007 p:---- s:0028 e:000027 CFUNC  :call
c:0006 p:0010 s:0025 e:000024 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/cmdline.rb:109
c:0005 p:0139 s:0020 E:0009c0 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons.rb:196
c:0004 p:0049 s:0014 E:000428 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:90
c:0003 p:0119 s:0009 e:000008 METHOD $APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:79
c:0002 p:0079 s:0004 E:001ed8 EVAL   script/delayed_job:6 [FINISH]
c:0001 p:0000 s:0002 E:000e88 TOP    [FINISH]

script/delayed_job:6:in `<main>'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:79:in `daemonize'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:90:in `run_process'
$APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons.rb:196:in `run_proc'
$APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/cmdline.rb:109:in `catch_exceptions'
$APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/cmdline.rb:109:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons.rb:197:in `block in run_proc'
$APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/controller.rb:73:in `run'
$APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/application.rb:296:in `start'
$APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/application.rb:264:in `start_proc'
$APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/application.rb:264:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/application.rb:255:in `block in start_proc'
$APPDIR/shared/bundle/ruby/2.1.0/gems/daemons-1.1.9/lib/daemons/application.rb:255:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:92:in `block in run_process'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:104:in `run'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:149:in `start'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `block in add'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `block (2 levels) in add'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/plugins/clear_locks.rb:7:in `block (2 levels) in <class:ClearLocks>'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/plugins/clear_locks.rb:7:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:150:in `block in start'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:150:in `loop'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:151:in `block (2 levels) in start'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:152:in `block (3 levels) in start'
/usr/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:153:in `block (4 levels) in start'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:188:in `work_off'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:188:in `times'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:189:in `block in work_off'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:267:in `reserve_and_run_one_job'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:267:in `block in reserve_and_run_one_job'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:205:in `run'
/usr/lib/ruby/2.1.0/benchmark.rb:294:in `realtime'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:206:in `block in run'
/usr/lib/ruby/2.1.0/timeout.rb:70:in `timeout'
/usr/lib/ruby/2.1.0/timeout.rb:70:in `catch'
/usr/lib/ruby/2.1.0/timeout.rb:82:in `block in timeout'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:206:in `block (2 levels) in run'
(eval):2:in `invoke_job_with_newrelic_transaction_trace'
$APPDIR/shared/bundle/ruby/2.1.0/gems/newrelic_rpm-3.7.1.180/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:339:in `perform_action_with_newrelic_trace'
(eval):3:in `block in invoke_job_with_newrelic_transaction_trace'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:102:in `invoke_job'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:112:in `block in invoke_job'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:108:in `rescue in block in invoke_job'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:125:in `hook'
$APPDIR/shared/bundle/ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:125:in `call'
$APPDIR/shared/bundle/ruby/2.1.0/gems/rollbar-0.10.14/lib/rollbar/delayed_job.rb:10:in `error'
@mipearson
Copy link
Author

I know this isn't much to go on at all - hoping that others who are experiencing this will see this and comment, see if there's a common factor.

@albus522
Copy link
Member

albus522 commented Jan 9, 2014

This appears to be an issue with how rollbar is trying to wrap DJ error reporting.

I suspect this doesn't do what they think it does:
https://github.com/rollbar/rollbar-gem/blob/master/lib/rollbar/delayed_job.rb#L10

@mipearson
Copy link
Author

Interesting. Worked under Ruby 2.0. cc @brianr

It's not easy for me to have Rails pull in the rollbar gem but omit it from delayedjob, but I'll have a go and see if that resolves things.

@brianr
Copy link

brianr commented Jan 9, 2014

Hm, looks like this might be caused by this bug in ruby: https://bugs.ruby-lang.org/issues/9315 - looks like there is a fix ready for whenever 2.1.1 is released.

@albus522 very possible; this code is over a year old and I don't remember exactly what the line is supposed to do. Is the example here the recommended way to add global error reporting for delayed_job? http://www.salsify.com/blog/delayed-jobs-callbacks-and-hooks-in-rails

@mipearson
Copy link
Author

Confirmed. Bug does not present when the delayed_job rollbar hook is removed (https://github.com/bikeexchange/rollbar-gem/tree/FixDJReporting)

Closing this & raising a new issue over in rollbar. Thanks for your help!

@mipearson mipearson mentioned this issue Jan 9, 2014
Closed
@albus522
Copy link
Member

@brianr that should work

@danbernier danbernier mentioned this issue Jan 27, 2014
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@albus522 @brianr @mipearson