Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use resolv-replace #1258

Closed
mperham opened this issue Oct 17, 2013 · 11 comments
Closed

Use resolv-replace #1258

mperham opened this issue Oct 17, 2013 · 11 comments

Comments

@mperham
Copy link
Collaborator

mperham commented Oct 17, 2013

Sidekiq should pull in resolv-replace so we don't risk locking up MRI.

Relevant blog post.
@halorgium
Copy link
Contributor

@mperham do you think this is something which celluloid could also benefit from?
We do actually have an async DNS in celluloid-io, but not everyone will be using that.

@mperham
Copy link
Collaborator Author

mperham commented Oct 17, 2013

I'm not sure. That blog post is ancient so it's unclear if this is still a problem with MRI. The GIL is still there so it's possible that calls into libc's DNS resolution don't release the GIL. I don't want to cargo cult so it'd be nice to have someone knowledgable chime in.

On Oct 16, 2013, at 19:51, Tim Carey-Smith notifications@github.com wrote:

@mperham do you think this is something which celluloid could also benefit from?
We do actually have an async DNS in celluloid-io, but not everyone will be using that.

—
Reply to this email directly or view it on GitHub.

@mperham
Copy link
Collaborator Author

mperham commented Oct 17, 2013

This comment implies that it's still a problem with MRI resque/resque#1101 (comment)

@mperham
Copy link
Collaborator Author

mperham commented Oct 17, 2013

@tmm1 signed off on it. Good enough for me. efa3915

@mperham mperham closed this as completed Oct 17, 2013
@RaVbaker
Copy link

I'm using google-adwords-api gem in my workers and since upgrade from previous version (2.11.2) to 2.16.0 I'm getting errors:

ArgumentError: cannot interpret as DNS name: nil

Any idea why? When I downgraded to 2.15.2 everything is working and for now I will stay with it. But I think that you might want to know.

@mperham
Copy link
Collaborator Author

mperham commented Oct 30, 2013

It's hard to diagnose without a backtrace.

@RaVbaker
Copy link

Backtrace returned by sidekiq in gist.

@tmm1
Copy link

tmm1 commented Oct 30, 2013

FWIW, I was probably wrong about needing resolv-replace. Looks like rb_getaddrinfo() does resolution outside the GVL in ext/socket/raddrinfo.c.

@thibaudgg
Copy link
Contributor

Got same issue ArgumentError: cannot interpret as DNS name: nil with Mongoid/Moped, backtrace:

[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv.rb:1176:in `create`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv.rb:1022:in `generate_candidates`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv.rb:1047:in `resolv`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv.rb:513:in `each_resource`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv.rb:406:in `each_address`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv.rb:115:in `block in each_address`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv.rb:114:in `each`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv.rb:114:in `each_address`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv.rb:92:in `getaddress`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv.rb:43:in `getaddress`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv-replace.rb:10:in `getaddress`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/resolv-replace.rb:23:in `initialize`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/connection/socket/tcp.rb:20:in `block in initialize`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/connection/socket/connectable.rb:119:in `handle_socket_errors`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/connection/socket/tcp.rb:20:in `initialize`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/connection/socket/connectable.rb:153:in `new`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/connection/socket/connectable.rb:153:in `block in connect`
[PROJECT_ROOT]/vendor/ruby-2.0.0/lib/ruby/2.0.0/timeout.rb:66:in `timeout`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/connection/socket/connectable.rb:152:in `connect`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/connection.rb:56:in `connect`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:102:in `block in connect`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:130:in `block in connection`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/connection/pool.rb:160:in `with_connection`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:129:in `connection`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:102:in `connect`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:191:in `block in ensure_connected`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/executable.rb:25:in `execute`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:189:in `ensure_connected`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:577:in `block in flush`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:608:in `block in logging`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.0/lib/active_support/notifications.rb:159:in `block in instrument`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.0/lib/active_support/notifications/instrumenter.rb:20:in `instrument`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.0/lib/active_support/notifications.rb:159:in `instrument`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/instrumentable.rb:31:in `instrument`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:607:in `logging`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:576:in `flush`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:398:in `process`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/operation/read.rb:48:in `execute`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:639:in `read`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:87:in `command`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/node.rb:439:in `refresh`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/cluster.rb:166:in `block in refresh`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/cluster.rb:335:in `call`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/cluster.rb:335:in `block in refresh_peers`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/cluster.rb:334:in `each`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/cluster.rb:334:in `refresh_peers`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/cluster.rb:171:in `block in refresh`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/cluster.rb:178:in `each`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/cluster.rb:178:in `refresh`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/cluster.rb:135:in `nodes`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/cluster.rb:224:in `with_primary`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/query.rb:379:in `update`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/moped-2.0.0.beta3/lib/moped/query.rb:420:in `upsert`
[PROJECT_ROOT]/app/models/site_admin_stat.rb:26:in `upsert_stats`
[PROJECT_ROOT]/app/services/site_admin_stat_upserter.rb:16:in `upsert`
[PROJECT_ROOT]/app/services/site_admin_stat_upserter.rb:12:in `upsert`
[PROJECT_ROOT]/app/workers/stats_handler_base_worker.rb:21:in `_handle_al_event`
[PROJECT_ROOT]/app/workers/stats_handler_base_worker.rb:14:in `perform`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/processor.rb:48:in `block (3 levels) in process`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/chain.rb:119:in `call`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/chain.rb:119:in `block in invoke`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/librato-sidekiq-0.1.2/lib/librato-sidekiq/middleware.rb:49:in `call`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/chain.rb:121:in `block in invoke`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/newrelic_rpm-3.6.8.168/lib/new_relic/agent/instrumentation/sidekiq.rb:25:in `block in call`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/newrelic_rpm-3.6.8.168/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:324:in `perform_action_with_newrelic_trace`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/newrelic_rpm-3.6.8.168/lib/new_relic/agent/instrumentation/sidekiq.rb:21:in `call`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/chain.rb:121:in `block in invoke`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/server/active_record.rb:6:in `call`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/chain.rb:121:in `block in invoke`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/server/retry_jobs.rb:62:in `call`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/chain.rb:121:in `block in invoke`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/server/logging.rb:11:in `block in call`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/logging.rb:22:in `with_context`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/server/logging.rb:7:in `call`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/chain.rb:121:in `block in invoke`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/chain.rb:124:in `call`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/middleware/chain.rb:124:in `invoke`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/processor.rb:47:in `block (2 levels) in process`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/processor.rb:102:in `stats`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/processor.rb:46:in `block in process`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/processor.rb:83:in `do_defer`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/sidekiq-2.16.0/lib/sidekiq/processor.rb:37:in `process`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `public_send`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `dispatch`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/celluloid-0.15.2/lib/celluloid/calls.rb:122:in `dispatch`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/celluloid-0.15.2/lib/celluloid/actor.rb:322:in `block in handle_message`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/celluloid-0.15.2/lib/celluloid/actor.rb:416:in `block in task`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/celluloid-0.15.2/lib/celluloid/tasks.rb:55:in `block in initialize`
[PROJECT_ROOT]/vendor/bundle/ruby/2.0.0/gems/celluloid-0.15.2/lib/celluloid/tasks/task_fiber.rb:13:in `block in create`

@aackerman
Copy link
Contributor

With the release of Sidekiq 2.16.1, the changes to resolve-replace have been reverted. If you are able to upgrade it should solve your issue.

@thibaudgg
Copy link
Contributor

Yep done, thanks!

@gregburek gregburek mentioned this issue Feb 25, 2015
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@halorgium @thibaudgg @tmm1 @mperham @RaVbaker @aackerman