Segfault on Ruby 2.5.0 and Rails 5.1.4 #652
Comments
|
Jeez... thanks. I'm inclined to say that it's in the If you replaced simplecov with |
|
I removed simplecov from my app, added in require 'coverage'
require 'minitest/autorun'
Coverage.start
Minitest.after_run { p Coverage.result }to my but no segfault. |
|
Ok weird discovery. With simplecov, I get the segfault when I run |
|
Strange... thanks a lot for trying that out. Could you please provide the crash report mentioned in the console output? I have no idea how this is happening we don't do anything too fancy so fear it's more or less trial and error to get it right/figure out what's the problem. I'm not sure but line 68 is this one config_path = Pathname.new(SimpleCov.root)So maybe try running that code at some point before and it's just the |
|
@chrismanderson interesting... might also be something to do with spring then if you're using it. Oh I love spring ;P It might also be that some of our root or command detection code goes rogue, produces weird output that then down the line leads to the segfault. I'm not big on minitest, but isn't there another way to run minitest test - like just executing them with ruby? Does it happen then? |
|
I do not use spring cause I enjoy my sanity :) so that shouldn't be the issue. I've attached the crash report. With my If I try to execute one test manually, using Pretty sure has happened for a while, one of the reasons I always use the rails running to run the suite and single tests. Running I've attached a crash report from the full suite run. Trying to go through the simplecov running to see if I can find more of where the failure happens. |
|
Ok so maybe it is the If configure I do not get a segfault (Simplecov doesn't really do anything, but still, no segfault). If I replace that with I do get a segfault. I would expect some long hash output, but not an exception. Not sure if I'm going down a rabbit hole in the wrong direction, but it is something at least. (And, if I do the same thing, puts Coverage in the at_exit block on Ruby 2.4.3, no segfault.) |
|
@chrismanderson try running not just at least the coverage library c implementation appears in the backtrace of the crash log so maybeeeee 🤷♂️ I think it'd be worthwhile opening a bug at ruby-lang. Generally speaking, we don't have a C-extensions or something so segfaults occuring are most likely ruby interpreter (and mostly coverage library) bugs as ruby code shouldn't be able to segfault the RubyVM. 2.5 has also seen some work on the coverage library so might be an error slipped through... |
|
Never knew you could preface Running # The path to this source file relative to the projects directory
def project_filename
@filename.sub(/^#{SimpleCov.root}/, "")
endI'll open an issue at ruby-lang. I didn't see any existing issues there; which is always disconcerting; but maybe there's some combination of the filelist/filename/content/etc that is causing the segfault. |
|
Seems like someone else did open an issue that I missed when searching. No activity yet, but fingers crossed. |
|
@chrismanderson could you please show me the values of like: and then show me the output? Thanks! |
|
It gets through most, if not all the files in my project. Sadly, ends on different ones each time if I run different test files or the whole suite (though the list is consistent when rerunning the same command). Whole suite One model test Different model Going to try later and get a sample project to recreate the issue. It's somewhat suspect that my two larger apps hit it, but a new sample project does not, so going to proceed down that path later today. |
|
Thanks for all the investigation. That all looks... oddly normal. The only odd thing that I spot is that you vendored your gems in |
|
I actually do have a filter turned on. One thing I just noticed is where the exception is happening. When I do as mentioned above, the actual exception is happening on either of the p lines (in this case |
|
Curious, trying setting |
|
Same exception when I do at the top of my |
|
How about, based on the stack trace require 'simpelcov'
def SimpleCov.at_exit
proc do
puts "I am a faked out at exist proc"
end
end
# and for fun
at_exit { puts "I'm being called before SimpleCov's at_exit" } |
|
That did not crash, outputted the text, though simplecov did not run. require 'simplecov'
SimpleCov.start 'rails' do
add_filter '.bundle'
end
def SimpleCov.at_exit
proc do
puts "I am a faked out at exist proc"
end
end
ENV['RAILS_ENV'] ||= 'test'
# ... rest of test_helper ...Same result basically as when I overrode SimpleCov.at_exit do
p "Testing finished"
end |
|
Well, that means it is for sure simplecov. How about in the trace I missed it actually ends with https://github.com/colszowka/simplecov/blob/f90a785f888b/lib/simplecov/source_file.rb#L85-L88 class SimpleCov::SourceFile
# The path to this source file relative to the projects directory
def project_filename
@filename.sub(/^#{SimpleCov.root}/, "")
end
endso SimpleCov::SourceFile.class_eval do
def project_filename
p [:debug, @filename]
p [:debug, SimpleCov.root]
File.basename(Dir.pwd)
end
end |
|
Basically the same result as putting puts into the gem itself above. Get the segfault during the puts itself (which obviously, is weird). (Line 8 translates to |
|
Is the last one a directory? Also, I thought My |
|
Also, Looks like it may be GC? Is this running in parallel by any chance? out of curiosity, here's some aggressive GC running refed in https://github.com/ko1/nakayoshi_fork/blob/master/lib/nakayoshi_fork.rb SimpleCov::SourceFile.class_eval do
def project_filename
h = {}
4.times{ # maximum 4 times
GC.stat(h)
live_slots = h[:heap_live_slots] || h[:heap_live_slot]
old_objects = h[:old_objects] || h[:old_object]
remwb_unprotects = h[:remembered_wb_unprotected_objects] || h[:remembered_shady_object]
young_objects = live_slots - old_objects - remwb_unprotects
# p [[live_slots, old_objects, remwb_unprotects], [young_objects]]
break if young_objects < live_slots / 10
disabled = GC.enable
GC.start(full_mark: false)
GC.disable if disabled
}
p [:debug, @filename, SimpleCov.root]
@filename.sub(/^#{SimpleCov.root}/, "")
end
end |
is random, which is always fun . Sometimes is segfaults on line 5, sometimes on 3. Sometimes it makes it through the debugs but then segfaults in simplecov (what's references I did have .bundle filtered, and added your list of filters to my .simplecov file but they still appear in the debug output. I then ran your GC code above and I think you might be right that it is GC. In this case the segfault occurred at
And crash report is attached. |
|
@chrismanderson How far do you think we can go before filing a bug report in Ruby? Need to find a minimal case. Can you move your bundle dir elsewhere? Maybe also try uncommenting What OSX version are you on? Is this only happening on your machine? Do you have any gems with C-extensions? Or have you using jemalloc or anything? I see looks suspicious, but I don't know much about it. I can see from the stack trace that the code is being called in a block. Do you have timeout running somewhere? How did you install your ruby? did you have to install what is Is it possible you're running out of disk space with too many open files? |
|
there were coverage changes https://bugs.ruby-lang.org/issues/13901 ... but I believe you confirmed that ignoring simplecov and just using the Coverage lib was fine require 'pp'
require 'coverage'
Coverage.start
at_exit { pp Coverage.result } |
|
Hey so this is a fun discovery, removed simplecov added in and got a segfault. Ran the suite a few more times, and got 8 segfaults and 2 runs that completed WITHOUT a segfault. So clearly seems to be a Ruby bug and likely something with GC that causes inconsistent results. Will open up a Ruby issue. |
|
👋 @bf4 we already have a bug filed with ruby-lang. Thanks for all the debugging help! https://bugs.ruby-lang.org/issues/14334 |
|
Depending on how that issue goes I might open up a new bug on ruby-lang now that I have a case that does not require simplecov at all. Going to see over the weekend too to get a sample project that recreates the issue as well. Always fun finding Ruby bugs! |
|
Yeah I updated the issue detailing that it's not related to simplecov and even if it'd be we don't have a c-extension - no way client ruby code should be able to cause a segfault. |
|
We were running into this segfault also (googleapis/google-cloud-ruby#1979) and were able to narrow it down. See https://bugs.ruby-lang.org/issues/14561 |
|
Thanks for doing that! Fingers crossed for a patch release! I'll close this issue though, there's nothing we can do than assure ruby-core this is a problem... |
|
Hey guys, I recently came across this same issue, running SimpleCov version 0.16.1 and Rails 5.1.5 and RSpec 3.7. I noticed that every time I wanted to run coverage locally it crashed with the SegFault metioned above... After trying to solve that issue I noticed that if I moved the SimpleCov config block from the Hope this is useful. |
|
@sebasjimenez10 You saved my day ;) |
Howdy! Updating two Rails 5.1.4 apps to Ruby 2.5.0 and getting a simplecov segfault on both of them.
Occurs after the minitest suite finishes. The segfault sometimes references different simplecov files (even when just rerunning the suite, which is fun) I've attached two crash reports from the same project. Working on coming up with a sample project to upload as well. Happy to provide any additional information.
Rails 5.1.4
Ruby 2.5.0p0
Minitest 5.11.3
Simplecov 0.15.1
Tried rebundling, reinstalling 2.5.0. Only change made to the project was updating the Ruby version to 2.5.0, bundling, then running the tests; that was enough to cause the segfault.
The text was updated successfully, but these errors were encountered: