Job was picked up after worker received TERM

[abezzub]

This weekend I was alerted that one of our jobs did not finish properly. It looks like Delayed Job picked up another job after receiving TERM signal. As I understand it is not supposed to happen:

https://github.com/collectiveidea/delayed_job/blob/master/lib/delayed/worker.rb#L151-L155

Logs:

May 15 13:41:11 app/worker.2: 2015-05-15T13:41:11-0700: [Worker(host:2ec6eceb-111f-49ff-b7b5-a3fa350236e2 pid:27)] 1 jobs processed at 16.2846 j/s, 0 failed
May 15 13:41:17 app/worker.2: 2015-05-15 13:41:16.948 26 LOG Stats: 6 req/s, in 3157 b/s, out 32720 b/s,query 23172 us
May 15 13:41:53 app/worker.2: [Worker(host:2ec6eceb-111f-49ff-b7b5-a3fa350236e2 pid:27)] Exiting...
May 15 13:41:53 app/worker.2: 2015-05-15T13:41:52-0700: [Worker(host:2ec6eceb-111f-49ff-b7b5-a3fa350236e2 pid:27)] Exiting...
May 15 13:41:54 app/worker.2: 2015-05-15 13:41:52.975 26 LOG Got SIGTERM, ignoring!
May 15 13:41:54 app/worker.2: [Worker(host:2ec6eceb-111f-49ff-b7b5-a3fa350236e2 pid:27)] Job ClassName#run_async_without_delay (id=16008133) RUNNING
May 15 13:41:54 app/worker.2: 2015-05-15T13:41:52-0700: [Worker(host:2ec6eceb-111f-49ff-b7b5-a3fa350236e2 pid:27)] Job ClassName#run_async_without_delay (id=16008133) RUNNING
May 15 13:41:55 app/worker.2:

I would expect Delayed Job to stop picking up new tasks after receiving TERM signal. Can someone help me look into this?

[abezzub]

May 15 13:41:54 app/worker.2: 2015-05-15 13:41:52.975 26 LOG Got SIGTERM, ignoring!

this messages comes from PGBouncer and Heroku support tells me that it means that PGBouncer ignored TERM signal so other processes can receive it successfully.

[albus522]

Closing stale