Getting "LoadError: no such file to load -- logger" when running bundler 1.5.0 against rubinius 2.2.1

[myronmarston]

This is happening even though the rubysl-logger gem is installed.

➜  rspec-mocks git:(master) gem list | grep logger
rubysl-logger (2.0.0)
➜  rspec-mocks git:(master) ruby -v
rubinius 2.2.1 (2.1.0 3ed43137 2013-11-17 JI) [x86_64-darwin12.5.0]
➜  rspec-mocks git:(master) bundle install
Fetching gem metadata from https://rubygems.org/.........
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Using builder (3.2.2)
Using archive-tar-minitar (0.5.2)

LoadError: no such file to load -- logger
Using rake (10.0.4)

LoadError: no such file to load -- logger

LoadError: no such file to load -- logger
Installing ffi (1.9.3)

LoadError: no such file to load -- logger
Unfortunately, a fatal error has occurred. Please see the Bundler troubleshooting documentation at http://bit.ly/bundler-issues. Thanks!
An exception occurred running /Users/myron/.gem/rbx/2.1.0/bin/bundle:

    exception class/object expected (TypeError)

Backtrace:

                          Kernel(Bundler::ParallelWorkers::UnixWorker)#raise at kernel/delta/kernel.rb:21
  Bundler::ParallelWorkers::Worker(Bundler::ParallelWorkers::UnixWorker)#deq at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/parallel_workers/worker.rb:35
                                      Bundler::Installer#install_in_parallel at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/installer.rb:325
                                                      Bundler::Installer#run at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/installer.rb:95
                                                  Bundler::Installer.install at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/installer.rb:15
                                                        Bundler::CLI#install at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/cli.rb:257
                                                           Thor::Command#run at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/vendor/thor/command.rb:27
                               Thor::Invocation(Bundler::CLI)#invoke_command at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/vendor/thor/invocation.rb:120
                                                               Thor.dispatch at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/vendor/thor.rb:363
                                       Thor::Base::ClassMethods(Class)#start at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/vendor/thor/base.rb:438
                                                          Bundler::CLI.start at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/cli.rb:10
                                                    { } in Object#__script__ at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/bin/bundle:22
                                                Bundler.with_friendly_errors at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/lib/bundler/friendly_errors.rb:5
                                                           Object#__script__ at /Users/myron/.gem/rbx/2.1.0/gems/bundler-1.5.0/bin/bundle:22
                                                         Kernel(Object)#load at kernel/common/kernel.rb:427
                                                           Object#__script__ at /Users/myron/.gem/rbx/2.1.0/bin/bundle:23
                                            Rubinius::CodeLoader#load_script at kernel/delta/code_loader.rb:66
                                            Rubinius::CodeLoader.load_script at kernel/delta/code_loader.rb:198
                                                     Rubinius::Loader#script at kernel/loader.rb:649
                                                       Rubinius::Loader#main at kernel/loader.rb:831

[brixen]

Bundler 1.5.0 added a new dependency on Logger. You can use 1.3.5 with Rubinius 2.2.2.

I'll release Rubinius 2.2.3 with new bootstrap dependencies for Bundler 1.5.0.

[myronmarston]

Thanks for looking into this.

You can use 1.3.5 with Rubinius 2.2.2.

I don't personally use Rubinius; this was on travis, where they upgraded to bundler 1.5.0 since it just came out and the rspec rubinius builds immediately started failing. I'm not aware of a way to force travis to use an older version of bundler than what they have installed, so until this is resolved I believe everyone's rubinius builds on travis will be broken.

In general, it concerns me that rubinius has taken a path that feels to me to be very fragile. Bundler choosing to use another piece of stdlib doesn't seem like something that should make it break on rubinius. It's become more and more of a burden to keep rbx builds on travis passing for the gems I maintain. Do you have plans to address this in a more holistic way or will we keep playing whack a mole with these stdlib/rubysl issues?

[cielavenir]

Perhaps it is related to https://travis-ci.org/cielavenir/multisax/builds/16064668
See, rbx-2.1.1 works, while 2.2.2 doesn't.

[yorickpeterse]

@myronmarston I believe @brixen is working on a patch for Bundler to make it work better with the rubysl Gem & friends which should solve these issues.

[brixen]

@myronmarston yes, there will be a patch to Bundler that gives it a set of gems it should not exclude. Rubinius works just fine except that Bundler removes all the gems that Rubinius has already installed. This is not specifically a fragility in Rubinius but a result of assumptions that Bundler makes. The unfortunate situation with RubyGems and Bundler being separate and partially competing projects will also hopefully be addressed in the near term.

I can understand that it's frustrating to have something break when you don't know what it is or what caused it. However, much of this is a consequence of deeply flawed infrastructure in the whole Ruby ecosystem. It won't get fixed until we start fixing it. If you've ever had two gems whose secondary or tertiary dependencies conflict, you know some of the pain.

It's not my intention to make it difficult to use or support Rubinius but it's also not my intention to continue sweeping these serious issues under the rug. Your patience and support while we try to improve the Ruby ecosystem is much appreciated.

[brixen]

@myronmarston as you can see here https://travis-ci.org/rubinius/bundler-canary/builds/16080883 the issue involves the --standalone argument to bundle install, which disables shared gems.

Curious, what's the reason for using --standalone?

[brixen]

@myronmarston another thing I just realized is that bundler-canary is failing with Bundler 1.3.5 with --standalone, so it's not related to Bundler 1.5 release at all. Or am I mistaken?

[brixen]

@myronmarston nope, I am mistaken, see https://travis-ci.org/rubinius/bundler-canary/jobs/16081410#L53 Not sure how Travis is forcing the Bundler version despite installing the gem. I'll look into that.

[brixen]

@myronmarston I got all the shenanigans with Bundler and Travis worked out and have a solid repro here https://travis-ci.org/rubinius/bundler-canary/builds/16090580 so I'll hopefully be able to keep tabs on this in the future.

[myronmarston]

Curious, what's the reason for using --standalone?

--standalone provides a file that sets up all the load paths to use the bundled gems w/o involving bundler or rubygems at runtime at all, which has a couple nice benefits:

• Vastly improves boot time of a short-lived ruby scripts (such as when running a single spec file). For example, in rspec-expectations, using MRI 1.9.3, running time bin/rspec spec/rspec/expectations_spec.rb (which uses the setup file generated by --standalone) reports it takes 137 ms on my machine. In constrast, time bundle exec spec/rspec/expectations_spec.rb reports it takes 969 ms. That's 7 times slower. We want each of our spec files to run and pass when run in isolation, so part of our travis builds is a loop that iterates over each spec file in a bash script and runs bin/rspec <path/to/file>. Saving 800 ms of boot time for each of these makes a huge difference in how long it takes our builds to run.
• At one point, we had a problem where a spec passed when run via bundle exec but failed when run w/o bundler, because the spec relied upon an extra piece of stdlib being loaded that bundler happens to load. Running w/o bundler at runtime helps surface issues so that we ensure our specs don't depend upon some undeclared dependency that bundler happens to load for us.

It won't get fixed until we start fixing it.

I'm not convinced that rubinius taking a new path with regards to stdlib will help us address these issues. The vast majority of us still target MRI. My experience with rubinius 2.0, and particularly this choice has been nothing but pain so far:

• When rubinius 2.0 came out, all of the rbx builds on all of my gems started failing because the rvm name for rubinius apparently changed away from rbx-18mode and rbx-19mode overnight with no warning or deprecation period. (My impression is that this broke the rubinius travis builds for every ruby project that has chosen to test against rubinius.) You blogged about the new setup so we changed .travis.yml in a bunch of projects to match.
• Things were still broken. A require 'test/unit' statement was causing a Could not find 'minitest' (~> 4.7) among 105 total gem(s) error, so I tried adding minitest to our Gemfile. Nope, still broken.
• I remembered reading something about rubinius spinning off rubysl and noticed there's a rubysl-test-unit gem, so I decide to add that to my Gemfile. It fixes the error I was having but now I get a new error from the cucumber part of our build trying to require enumerator. I decide to add rubysl to our Gemfile and it fixes it. One project down, a bunch more to go.
• For rspec-mocks, I immediately add rubysl to the Gemfile. It fails on a part of the build that tries to load test/unit. I'm confused by the fact that rubysl-test-unit is not a runtime dependency of rubysl (isn't it supposed to be the whole stdlib?) but try adding rubysl-test-unit, and it works. Another project down.
• For rspec-core, we try adding rubysl to the Gemfile. It fails while loading nokogiri when nokogiri requires racc/parser. Adding racc to our gemfile fixes the issue, but again: if racc is part of the stdlib, why didn't rubysl already pull it in?
• While troubleshooting Rubinius skips calling respond_to_missing? in some circumstances #2845, I decide to try the rubinius debugger. I try the instructions from the rubinius website but can't get it to work. I learned in no such file to load -- rubinius/debugger #2846 that I was supposed to install a gem for it, but the docs didn't mention that. (They do now, thanks to The debugger page at the website is out of date #2847 getting addressed -- glad to see that fixed). I had no idea, though: I thought the debugger was a built-in.
• Travis upgrades bundler to 1.5 (which is pretty reasonable) and the rbx builds that I just spent a bunch of time a couple weeks ago finally getting green are failing again because of this issue :(.

Having multiple good implementations is important for ruby's future so I want to see rubinius succeed, but in the past few months it's been nothing but a PITA for me. I'm not the only one that feels this way. For rubinius to succeed, I think it needs the ecosystem and tooling to work well with it, but as maintainer of one well-used piece of that ecosystem (RSpec), I'm very discouraged by the amount of effort required to keep our builds green on rubinius (multiplied by each of the individual repositories...). I still don't really understand what's going on with rubysl and why there are pieces of the stdlib that that doesn't provide. Also, for this issue I thought I could solve it by installing rubysl-logger, but that didn't work and I have no idea why :(.

It would be really great if you could streamline things for gem maintainers, and write up something explaining how it works so we understand how to troubleshoot issues when they arise. Right now, it feels like it's just a game of whack-a-mole.

[brixen]

@myronmarston thanks for writing up your long list of complaints. We have or are working on addressing all of them.

In fairness, many of them are not specifically Rubinius issues. When Travis changed the aliases for Rubinius, they delayed for a long time rolling out the new RVM support for binary installs, which would have mitigated a lot of the frustration by having a clear and simple replacement for projects testing on Rubinius. Rubinius was ready, but there wasn't anything we could do. The removal of the language modes in Rubinius was a correction of a bad choice that was only completely apparent in hind sight. The tradeoff for having to fix some broken travis builds is allowing us to devote much more attention to new features and compatibility by removing an architecture choice that was causing a great deal of complexity.

The reason that racc, psych, json, minitest (at least that list for now) are not dependencies of RubySL is because they have already been separated from the MRI standard library and are separately maintained. Some people have them already listed as gem dependencies. This pile of special cases and exceptions is a huge problem for Ruby. Right now, minitest 5.x and 4.x can't be used together but the test/unit library in MRI depends on minitest 4.x. That's why we can't list rubysl-test-unit as a dependency by default. There's no mechanism in RubyGems or Bundler to allow a project author to resolve these incompatible dependencies.

By making gems the single mechanism of packaging, distributing, and composing Ruby components, Rubinius and RubySL are trying to improve consistency, reliability, and maintainability of the Ruby ecosystem. It's not something everyone agrees with and there are a lot of components, like Travis, RubyGems, Bundler, RVM, other gems, etc. that Rubinius has no control over. We're trying to submit patches to RubyGems and Bundler, work with Travis and RVM, etc.

Finally, to --standalone, the fact that you even need to select this choice illustrates a fundamental flaw in RubyGems architecture that remains unfixed. RubyGems should never have been resolving dependencies at runtime. This, along with all the features that Bundler added are flaws in the RubyGems architecture that cause a great deal of pain. However, in general, RubyGems has been great for Ruby and it should be the mechanism for distributing and composing components of Ruby software. We're working on fixing the broken parts and making the whole system more consistent and reliable.

Fundamentally, there is no guarantee that your experience with any software will be pain free. I could point out the hundreds and hundreds of issues that people have had with RSpec over the years. But those issues don't detract from RSpec being perceived by some as useful software. In the end, whether you use Rubinius or not is your decision. We'll continue fixing issues and improving the technology available to people using Ruby. If you happen to have an interest in helping out, that's great. And there are a lot of avenues available to get help, like Twitter, the IRC channel, issues like this one, email. Waiting until you're frustrated by misunderstanding something isn't helping us or you. So please reach out if you need help. If you decide you're not interested, best of luck anyway.

[myronmarston]

@myronmarston thanks for writing up your long list of complaints. We have or are working on addressing all of them.

I'm glad to hear that.

The reason that racc, psych, json, minitest (at least that list for now) are not dependencies of RubySL is because they have already been separated from the MRI standard library and are separately maintained. Some people have them already listed as gem dependencies. This pile of special cases and exceptions is a huge problem for Ruby. Right now, minitest 5.x and 4.x can't be used together but the test/unit library in MRI depends on minitest 4.x. That's why we can't list rubysl-test-unit as a dependency by default.

This makes sense now that you explain it, but as far as I know there's no rubinius doc page explaining all these details about how it handles stdlib now. Currently, the burden is on gem maintainers to figure this out by trial and error.

Fundamentally, there is no guarantee that your experience with any software will be pain free.

I agree completely. I've never believed or stated otherwise.

I could point out the hundreds and hundreds of issues that people have had with RSpec over the years. But those issues don't detract from RSpec being perceived by some as useful software.

Yep, I'm well aware of issues RSpec users have had.

In the end, whether you use Rubinius or not is your decision.

I've never used rubinius and have no plans to use it in the foreseeable future. However, as maintainer of an important gem in the ruby ecosystem, it's important to the ruby ecosystem that we support rubinius...so even though you can say "having RSpec support rubinius or not is your decision", it would be unreasonable for me to choose not to. (Consider the repercussions for the community...for example: bundler's test suite uses RSpec, and it's important that bundler supports rubinius).

If you happen to have an interest in helping out, that's great. And there are a lot of avenues available to get help, like Twitter, the IRC channel, issues like this one, email.

Given I'm not a rubinius user and don't plan to start using it anytime soon, I think my limited open source time is better spent on the projects I already work on (primarily RSpec and VCR).

However, I am trying to help out in a limited capacity by reporting issues like this one, #2845 and #2846, and plan to continue to do so.

One thing that would be a HUGE help to non-rubinius using gem maintainers like myself is a documentation page explaining the repercussions of the differences rubinius has compared to other implementations for both gem maintainers and gem users. (I'm thinking primarily of the rubysl issues we're discussing, but there may be other differences I haven't run into yet). It would be VERY useful to have a page explaining which parts of the stdlib are available via gem 'rubysl', and which parts aren't (and why). This would help guide myself and other gem maintainers as we are trying to keep our builds green against rubinius, and also guide us in documenting for users how to use our gems on rubinius.

[brixen]

@myronmarston yes, I fully understand the need for more documentation around all these issues, I'm working on that.

I've released Rubinius 2.2.3 that fixes the load error in this issue. I'm working on a more robust fix in Bundler itself.

[myronmarston]

Thanks, @brixen. Looking forward to improved documentation :).