ruby-2.1.2/gems/eventmachine-1.0.3/lib/eventmachine.rb:187: [BUG] Segmentation fault at 0x00000000000000 #511
Comments
|
Here is how to reproduce the crash: $ ulimit -n 10000 The output can be seen on travis: |
|
I got pretty much the same segfault with eventmachine 1.0.3 on ruby 2.1.2: https://gist.github.com/ikataitsev/9a9c13d7d494ee27d780 EM process (which fetches a message from RabbitMQ using ruby-amqp gem, does some processing and writes it to the TCP socket) works fine for a few hours and then crashes with a segfault. I'd be happy to provide more details if needed. |
|
From what others have said, EventMachine is dead. So, I'm experimenting with Celluloid::IO. So far it's been a bit difficult to migrate the code but generally it seems more robust. Will be interesting to see how it scales. |
Invoker currently does not start on a new install because it will use RubyDNS 0.9.0 which was released on 10/23/14. This took an exceptionally long time to troubleshoot because of celluloid, but I finally tracked it down! Invoker's gemspec locks RubyDNS to `~> 0.7`, however [the author states](socketry/rubydns#28) that `0.x` releases are incompatible, but minor versions should be compatible. Here is the [offending change](socketry/rubydns@v0.8.5...v0.9.0#diff-d86035f01d9c6b072797c19b43eb736aL116) in 0.9.0. For the time being you can add the following to your own `Gemfile`: `gem 'rubydns', '~> 0.8.5'` Here is a trace of what happens when you run with RubyDNS 0.9.0: ``` → invoker start Procfile D, [2014-10-26T09:45:30.515304 #59891] DEBUG -- : Terminating 1 actor... vendor/bundle/gems/celluloid-0.16.0/lib/celluloid/calls.rb:39:in `check': wrong number of arguments (1 for 0) (ArgumentError) from vendor/bundle/gems/celluloid-0.16.0/lib/celluloid/calls.rb:24:in `dispatch' from vendor/bundle/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in `dispatch' from vendor/bundle/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in `block in invoke' from vendor/bundle/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in `block in task' from vendor/bundle/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in `block in task' from vendor/bundle/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in `block in initialize' from vendor/bundle/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in `block in create' from (celluloid):0:in `remote procedure call' from vendor/bundle/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in `value' from vendor/bundle/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in `method_missing' from vendor/bundle/gems/invoker-1.3.0/lib/invoker/power/powerup.rb:17:in `block in run' from vendor/bundle/bundler/gems/eventmachine-4d53154a9ea4/lib/eventmachine.rb:187:in `call' from vendor/bundle/bundler/gems/eventmachine-4d53154a9ea4/lib/eventmachine.rb:187:in `run_machine' from vendor/bundle/bundler/gems/eventmachine-4d53154a9ea4/lib/eventmachine.rb:187:in `run' from vendor/bundle/gems/invoker-1.3.0/lib/invoker/power/powerup.rb:13:in `run' from vendor/bundle/gems/invoker-1.3.0/lib/invoker/power/powerup.rb:7:in `block in fork_and_start' from vendor/bundle/gems/invoker-1.3.0/lib/invoker/power/powerup.rb:7:in `fork' from vendor/bundle/gems/invoker-1.3.0/lib/invoker/power/powerup.rb:7:in `fork_and_start' from vendor/bundle/gems/invoker-1.3.0/lib/invoker/process_manager.rb:72:in `run_power_server' from vendor/bundle/gems/invoker-1.3.0/lib/invoker/commander.rb:39:in `start_manager' from vendor/bundle/gems/invoker-1.3.0/lib/invoker/cli.rb:46:in `start' from vendor/bundle/gems/thor-0.19.1/lib/thor/command.rb:27:in `run' from vendor/bundle/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command' from vendor/bundle/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch' from vendor/bundle/gems/thor-0.19.1/lib/thor/base.rb:440:in `start' from vendor/bundle/gems/invoker-1.3.0/lib/invoker/cli.rb:12:in `start' from vendor/bundle/gems/invoker-1.3.0/bin/invoker:7:in `<top (required)>' from ./bin/invoker:16:in `load' from ./bin/invoker:16:in `<main>' ```
|
@ioquatix @ikataitsev That's a handy Travis job to repro this! Could I ask you to re-run it with eventmachine 1.0.4 and Ruby 2.1.5 to see if the situation persists? The eventmachine-segfault branch doesn't seem to be available anymore in your repo. |
|
Sidenote, I'm here to make eventmachine not dead: |
|
@sodabrew Sorry, that branch is long since gone. |
|
RubyDNS looks like a neat project. I'm sorry that eventmachine lagged long enough to push you away. Do you recall enough of the problem to describe how I could set up a repro? |
|
@sodabrew here it is https://github.com/ValiMail/rubydns/tree/eventmachine-segfault btw. I found Celluloid a much better platform for scalability and other reasons, it has it's problems too but in general feels like a much better platform for building highly concurrent systems. Even if EventMachine was fixed, I wouldn't consider moving back. Best of luck. |
|
Good news! This is resolved by #502! |
|
And it seems back in 1.0.7: This happens, again, when opening a lot of connections at the same time. |
|
@hsmade Could you open a new issue and include a C backtrace if you can get one? |
When creating a large number of UDP sockets:
The text was updated successfully, but these errors were encountered: