Getting Rubinius::ToolSet::Runtime::CompileError from some of rspec-expectations files
#2934
Comments
We're getting odd compile errors from rubinius: rubinius/rubinius#2934
|
It also does not work with have.rb in this build. Also tried with eq.rb and eql.rb and got very the same error. |
The issue is described in: rubinius/rubinius#2934
|
I can confirm I'm getting this error on 2.2.5 as well: https://gist.github.com/sikachu/ff4ab66b900290ceea64 |
There was some problem with class autoloading in Rubinius, which can be workaround by just requiring that missing file. See rubinius/rubinius#2934
|
I managed to get the test pass by explicitly requiring that file in the test. See thoughtbot/appraisal@33775b4 and https://travis-ci.org/thoughtbot/appraisal/jobs/19821160 |
|
Is autoload not supported on rubinius? |
|
BTW, thanks @sikachu. It's nice in rspec-expectations to be able to use autoload so that it boots faster and doesn't bother loading matchers the user doesn't use, but we may have to switch it to not autoload on rubinius. |
|
@myronmarston It is as far as I'm aware of. It could be a race condition with the autoloading system, but it's hard to tell without some digging around. |
|
I did some digging around and it seems that autoloading is not thread-safe under Rubinius. Although I could not get rspec to fail I managed to whip together the following script: require 'rspec'
threads = []
100.times do
threads << Thread.new { sleep(rand); RSpec::Matchers::BuiltIn::Match }
end
threads.each(&:join)Running this script under Rubinius will work most of the time. However, once in a while the following error occurs: I suspect that a similar problem is causing the compiler to break. That is, N threads try to autoload the same file with slightly different timings. As a result the compiler might think that everything it needs is there when in reality it's not there yet. This is however just a theory, I'm not very familiar with the autoloading system (yet) so I can't tell for certain. |
|
Thanks for looking into this. We don't do any multithreading in RSpec (or in our test suite), though, so I doubt this is the root cause of our issue. |
|
The following is mostly a personal note / note for other contributors. I did some poking around in https://github.com/rubinius/rubinius/blob/master/kernel/common/autoload.rb as it looks prone to potential race conditions. For example, the I messed around with putting locks in place, both instance and class wide locks, but to no avail. I suspect that the actual race condition in rspec occurs earlier somewhere in the Rbx chain. It's also interesting that there are multiple Autoload instances present in the average Rbx process. The details are not clear to me so I'll assume this is intended. Either way, looking at this autoloading code I strongly feel it's not thead-safe and thus, sometimes, might cause problems. For example, two threads try to autoload a file and one sets something like Note that this is merely a theory. It could very well be something else, but it just has "autoloading" and "thread safety" written all over it. |
|
There are actually specs for autoload thread safety that we're passing and issues in this have been fixed before. It's not a simple / trivial case here. The locking logic happens around require requests here: https://github.com/rubinius/rubinius/blob/master/kernel/common/code_loader.rb#L31-L91. The spec for this can be found here: https://github.com/rubinius/rubinius/blob/master/spec/ruby/core/module/autoload_spec.rb#L457-L503 Also if RSpec runs single threaded, this is very like a different problem and not a concurrent autoload issue. |
|
Hi getting the "uninitialized constant Rubinius::ToolSets" error when building: I also installed the "rubinius-toolset" gem manually to make it work. "rake --trace" output: |
|
@ali-bugdayci That is a separate issue that should be resolved in 2.2.6. Please try said version and report it as a separate issue if the problem persists. |
|
@yorickpeterse I had a different problem with 2.2.6, hence tried 2.2.5, I am getting "Error libc not found" error on 2.2.6 build. I already have build-essentials package on ubuntu 12.10. By the way, the requirements section might need a change, since I apply them all: http://rubini.us/doc/en/getting-started/requirements/ |
|
@all-bugdayci Please report this problem in a separate issue as it's not related (from what I can tell) to the issue experienced by OP. |
|
We're getting this error in another spot which is not using autoload so it appears to be a general code loading issue and not just an autoload issue. This travis build (of rspec-core's 2-14-maintenance branch) got this failure: Notice that it's failing when using The file being loaded is quite small: |
|
Experienced the same problem while trying to run an application in our staging environment: https://gist.github.com/YorickPeterse/95bf9eeef16f0b417f4e Similar to the issue described by @myronmarston it does not seem to occur every time but instead semi-randomly. |
|
Digging into this again, I'm pretty darn sure this is a thread-safety problem. To help debug I patched diff --git a/kernel/common/module.rb b/kernel/common/module.rb
index 0d0e6a8..3d2152a 100644
--- a/kernel/common/module.rb
+++ b/kernel/common/module.rb
@@ -664,6 +664,9 @@ class Module
msg = "Missing or uninitialized constant: #{mod_name}#{name}"
end
+ puts "NameError: #{name}: #{msg} | Thread: #{Thread.current.instance_variable_get(:@thread_id)}"
+ puts caller
+
raise NameError.new(msg, name)
endRunning the script pasted above I sometimes (not always) get the following output: The first NameError is always there and doesn't break anything by the looks of it. The NameErrors after that one are only sometimes present. The amount of errors also varies, sometimes only 2, sometimes 5, etc. |
|
Another output sample where the script fails: |
|
And the last one: |
|
Hmm, is Anyhow, I searched through The first one isn't spawning threads; it's simply using thread local storage to be safe. For the second, I think it's using a thread there because I believe Could rubinius be doing some extra multi-threading under the covers as an implementation detail that could be causing this? |
|
@myronmarston The I'm not yet sure if the autoloading error is causing the compilation error, I'll need to do some further digging. At the moment I have two theories:
Rubinius does use threads for a bunch of things, e.g. the JIT runs in its own thread and so does part of the GC I believe. |
|
Patched Rubinius' diff --git a/kernel/common/autoload.rb b/kernel/common/autoload.rb
index d0ac53f..6ba95c4 100644
--- a/kernel/common/autoload.rb
+++ b/kernel/common/autoload.rb
@@ -14,6 +14,10 @@ class Autoload
attr_reader :thread
def initialize(name, scope, path)
+ if path.include?('rspec')
+ puts "Autoload.new(#{name.inspect}, #{scope.inspect}, #{path.inspect})"
+ end
+
@name = name
@scope = scope
@path = pathThis would display all the autoloading actions triggered in Rspec. The output is now as following: Seeing how the same constant is not autoloaded twice (unless I missed something) I'll assume that the actual registering of constants is thread-safe. One other possibility is that the order in which constants are loaded messes things up somehow. Note that the output above is the order in which constants are registered opposed to the load order. Locking in |
|
Using the following patch: diff --git a/kernel/common/autoload.rb b/kernel/common/autoload.rb
index d0ac53f..07f1778 100644
--- a/kernel/common/autoload.rb
+++ b/kernel/common/autoload.rb
@@ -31,6 +31,10 @@ class Autoload
# When any code that finds a constant sees an instance of Autoload as its match,
# it calls this method on us
def call(under, honor_require=false)
+ if path.include?('rspec')
+ puts "Autoloading#call with #{name} in #{scope} from #{path} under #{under}"
+ end
+
# We leave the Autoload object in the constant table so that if another
# thread hits this while we're mid require they'll be come in here and
# be held by require until @path is available, at which time they'll
@@ -49,6 +53,10 @@ class Autoload
end
def resolve
+ if path.include?('rspec')
+ puts "Autoloading#resolve with #{name} in #{scope} from #{path}"
+ end
+
unless @loaded && @thread == Thread.current
@loaded = true
@thread = Thread.currentThe output is now as following: Ran the script a few times and it seems that the amount of |
|
Hi there! I don't know if it's due to the same bug (at least we don't have the same exception raised) but we also have a thread-safety issue with autoloading. At the moment, Action Pack's test suite runs in parallel with Minitest's The problem is that most constants are autoloaded and we have a require 'minitest'
require 'minitest/autorun'
module A
module C; end
autoload :B, 'fake_file'
class B
autoload :C, 'c'
def self.bar
C.foo
end
end
end
class B < A::B; end
class FooTest < Minitest::Test
def self.run_one_method(klass, method_name, reporter)
Minitest.parallel_executor << [klass, method_name, reporter]
end
def test_b
B.bar
end
endThe module A
class B
class C
def self.foo; end
end
end
endThe interesting thing is that if we remove the |
|
@robin850 Hm, interesting. In my own experience autoload is definitely not thread-safe, even if we might claim otherwise. |
|
@yorickpeterse : Yep, it looks like. I don't know whether it could be useful but here's a more "ruby-ish" example that doesn't rely on Minitest to showcase the problem: https://gist.github.com/robin850/3eeaed2538f50a9887c2 ; you just need to run |
Since the `ForkingExecutor` class seems to be pretty slow on Rubinius due to DRb (c.f. http://git.io/xIVg), let's avoid running tests with it on this platform. Also, the `parallelize_me!` call make the suite to output a bunch of errors due to rubinius/rubinius#2934 since there are thread-safety problems with autoloading.
|
@robin850 - can you verify that your repro still fails on rbx-master and open a new issue with a link to your repro so we can track work there? |
|
The script @robin850 wrote still reproduces for me using rbx master. |
|
In commit b57399f I made some changes to hopefully finally make autoloading fully thread-safe (famous last words). @myronmarston and @robin850: could you try to run your scripts using said commit and see if the issue still persists? |
|
Hm, apparently the script discussed in #2934 (comment) still triggers a constant error. It appears this hydra grew a few extra heads. |
|
I would try if I had such a script but I was never able to repro locally. The errors I saw only happened on travis. |
|
As far as I can tell, RSpec::Matchers::Builtin contains no autoloads, and that's the constant that's going missing. I can even remove the last element and it still fails on JRuby and Rubinius. I'm not sure the script from #2934 (comment) is actually hitting any autoload-related bugs. |
|
I'm not sure if it explains the original issue, which is in a deeper namespace, but the logic for autoloading RSpec::Matcher is not thread-safe. See rspec/rspec-dev#121. It may not explain the original issue, but it does relieve Rubinius of any culpability in the RSpec::Matchers::BuiltIn error. |
|
@headius Hm, interesting. I spent some time looking at Rubinius' |
Yeah, it would be nice not to, but we haven't been able to come up with an alternate solution that both allows users to configure RSpec to use an alternates to rspec-expectations/rspec-mocks, while still making it "just work" for users who use the default stack. See my response on rspec/rspec-dev#121 for more detail. If you want to test the require 'rspec/matchers'
threads = []
100.times do
threads << Thread.new { sleep(rand); RSpec::Matchers::BuiltIn::Match }
end
threads.each(&:join)That'll avoid the
Yep, I don't think it explains the |
With current master (as of fbb3f1e), the given scripts are still failing unfortunately. |
…ad#call This fixes cases of autoload "errors" mentioned by @robin850 in #2934 (comment) and https://gist.github.com/robin850/3eeaed2538f50a9887c2 This is an explanation of the issue: We already have A::B been autoloaded by the main thread. When c.rb (see any of the previous links) is being autoloaded in another thread, class A::B is being opened using Rubinius.open_class_under (through Rubinius.open_class). Since module A has an Autoload entry for B in its constant_table, open_class_under tries to call #call on this Autload object, which returns nil because #resolve returns false (CodeLoader.require returns false if a feature is already been loaded). With nil returned, open_class_under decided to create a new Class object for B, which means the autoload entry it already had for :C is lost, resulting in constant A::B::C not being found.
|
After some digging, it may be not related to |
|
If I'm understanding the thread here correctly, this is a concurrency issue in RSpec (see referenced issue rspec/rspec-dev#121). |
For RSpec's travis builds we've started getting
Rubinius::ToolSet::Runtime::CompileErrorerrors for some files from rspec-expectations. It's inconsistent, though; originally it happened once or twice and appeared to be purely transient because the build would pass on retry, but now it's happening much more consistently.Note that it varies which file in rspec-expectations triggers the compile error; for example, in this build it's match.rb but in this build it's contain_exactly.rb.
These files are loaded lazily via autoload so the failure happens the first time the named matcher is used. After that, the constant remains undefined so the build gets "Missing or uninitialized constant: ..."
NameErrors for any later use of the matcher that failed to compile.For both of these specific build failure examples, travis reports this rubinius version:
The text was updated successfully, but these errors were encountered: