Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OSX only] newrelic make workers reaping indefinitely #70

Closed
jobwat opened this issue Apr 22, 2013 · 2 comments
Closed

[OSX only] newrelic make workers reaping indefinitely #70

jobwat opened this issue Apr 22, 2013 · 2 comments

Comments

@jobwat
Copy link

jobwat commented Apr 22, 2013

This may be probably related to issue #25, as a comment from there just lead me to realize newrelic was probably the cause...

The issue: resque-pool gets crazy after a little while...

⚡ bundle exec resque-pool
resque-pool-manager[webapp][63824]: Resque Pool running in development environment
 logging soap calls to log/soap.log in development
resque-pool-manager[webapp][63824]: started manager
resque-pool-manager[webapp][63824]: Pool contains worker PIDs: [63865, 63871, 63866, 63872, 63867, 63869]
resque-pool-worker[webapp][63865]: Starting worker Jos-Mac.local:63865:multis
after prefork 63865
resque-pool-worker[webapp][63866]: Starting worker Jos-Mac.local:63866:*resque-pool-worker[webapp][63869]: Starting worker Jos-Mac.local:63869:*
after prefork 63869
after prefork 63866
resque-pool-worker[webapp][63867]: Starting worker Jos-Mac.local:63867:*
after prefork 63867
resque-pool-worker[webapp][63872]: Starting worker Jos-Mac.local:63872:*
after prefork 63872
resque-pool-worker[webapp][63871]: Starting worker Jos-Mac.local:63871:*
after prefork 63871

[5-10min, loads of jobs done without a trouble....]

0resque-pool-manager[webapp][63824]: Reaped resque worker[63867] (status: 0) queues: 003_new_markets
resque-pool-manager[webapp][63824]: Reaped resque worker[63869] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64601]: Starting worker Jos-Mac.local:64601:*
after prefork 64601
resque-pool-worker[webapp][64603]: Starting worker Jos-Mac.local:64603:*
after prefork 64603
resque-pool-manager[webapp][63824]: Reaped resque worker[63872] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64607]: Starting worker Jos-Mac.local:64607:*
after prefork 64607
resque-pool-manager[webapp][63824]: Reaped resque worker[64601] (status: 0) queues: 003_new_markets
resque-pool-manager[webapp][63824]: Reaped resque worker[64603] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64610]: Starting worker Jos-Mac.local:64610:*
after prefork 64610
resque-pool-worker[webapp][64612]: Starting worker Jos-Mac.local:64612:*
after prefork 64612
resque-pool-manager[webapp][63824]: Reaped resque worker[64607] (status: 0) queues: 003_new_markets
resque-pool-manager[webapp][63824]: Reaped resque worker[63866] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64614]: Starting worker Jos-Mac.local:64614:*
after prefork 64614
resque-pool-worker[webapp][64616]: Starting worker Jos-Mac.local:64616:*
after prefork 64616
resque-pool-manager[webapp][63824]: Reaped resque worker[64610] (status: 0) queues: 003_new_markets
resque-pool-manager[webapp][63824]: Reaped resque worker[64612] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64618]: Starting worker Jos-Mac.local:64618:*
after prefork 64618
resque-pool-worker[webapp][64620]: Starting worker Jos-Mac.local:64620:*
after prefork 64620
resque-pool-manager[webapp][63824]: Reaped resque worker[64614] (status: 0) queues: 003_new_markets
resque-pool-manager[webapp][63824]: Reaped resque worker[64616] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64622]: Starting worker Jos-Mac.local:64622:*
after prefork 64622
resque-pool-worker[webapp][64624]: Starting worker Jos-Mac.local:64624:*
after prefork 64624
resque-pool-manager[webapp][63824]: Reaped resque worker[64618] (status: 0) queues: 003_new_markets
resque-pool-manager[webapp][63824]: Reaped resque worker[64620] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64626]: Starting worker Jos-Mac.local:64626:*
after prefork 64626
resque-pool-worker[webapp][64628]: Starting worker Jos-Mac.local:64628:*
after prefork 64628
resque-pool-manager[webapp][63824]: Reaped resque worker[64624] (status: 0) queues: 003_new_markets
resque-pool-manager[webapp][63824]: Reaped resque worker[64622] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64630]: Starting worker Jos-Mac.local:64630:*
after prefork 64630
resque-pool-worker[webapp][64632]: Starting worker Jos-Mac.local:64632:*
after prefork 64632
resque-pool-manager[webapp][63824]: Reaped resque worker[64626] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64635]: Starting worker Jos-Mac.local:64635:*
after prefork 64635
resque-pool-manager[webapp][63824]: Reaped resque worker[64628] (status: 0) queues: 003_new_markets
resque-pool-manager[webapp][63824]: Reaped resque worker[64630] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64638]: Starting worker Jos-Mac.local:64638:*
after prefork 64638
resque-pool-worker[webapp][64640]: Starting worker Jos-Mac.local:64640:*
after prefork 64640
resque-pool-manager[webapp][63824]: Reaped resque worker[64632] (status: 0) queues: 003_new_markets
resque-pool-worker[webapp][64642]: Starting worker Jos-Mac.local:64642:*
after prefork 64642

[CTRL+C]

^Cresque-pool-manager[webapp][63824]: INT: immediate shutdown (graceful worker shutdown)
resque-pool-manager[webapp][63824]: manager finished

that's not always been the case, it's working like a charm on linux servers, and the problem seems to have disappeared since I've removed newrelic instructions

I have no idea how to debug from there :|

@jobwat
Copy link
Author

jobwat commented Apr 23, 2013

Confirmed: it's newrelic gem

Even if you remove all related hooks, the gem will somehow mess with resque-pool...

@jobwat
Copy link
Author

jobwat commented Apr 23, 2013

I've commented it out in config/newrelic.yml and my world is back to bright :)

Under the development: section:

  # New Relic gem mess up with resque-pool under OSX.. - https://github.com/nevans/resque-pool/issues/70
  disable_resque: true

@jobwat jobwat closed this as completed Jun 6, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant