Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Random segfaults on Ruby 1.9 get worse with Ruby 2 #1746
We run a Rails application and are in the process of upgrading it from Ruby 1.9 to Ruby 2. During this process we noticed segmentation faults in the passenger log. We are not sure what is causing them and are unable to reproduce them outside of sending production traffic requests at the server. The traces that get dumped by passenger have a lot of information in them, but we are having difficulty narrowing down the issue. The traces happen on a variety of different files/gems/extensions/calls without a obvious pattern.
We tried Ruby 2.1, 2.2 and 2.3 and all of them significantly increase the rate of segmentation faults over 1.9. We have upgrade all of our gems with C extensions, and the passenger gem we are using to ensure we have all patches. We have tried smart and direct spawning. Despite all this, the large number of random segfaults persists.
We disabled garbage collection and this significantly reduced the segfaults, but we quickly ran out of memory and can't run in this configuration outside of a testing scenario.
Our C extension gems:
Sample of segfaults:
crash-watch output on app process that segfaulted:
Not sure if this is a passenger issue, but I wanted to seek advice on how to troubleshoot these issues. Is there a way to get better information out of crash-watch? Are there parts of the segfault dumps we should focus on to narrow down the issue?
This doesn't look like a Passenger issue. The crash output is coming from the gems/app; Passenger just forwards their stdout/stderr and is not crashing itself (so its crash protection and backtrace logging are not triggered).
You can use our support forum for these kinds of questions.
This post has some information on tracing similar issues, my best guess would be some kind of compiler type or flag issue.