Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Double Loading Psych - Conflict with Ruby Bundled Version #66

Closed
tadman opened this Issue Jun 22, 2012 · 11 comments

Comments

6 participants

tadman commented Jun 22, 2012

Lately I've had nothing but trouble with Psych conflicting with the version that's built into Ruby. Using 1.9.3 and Psych 1.3.3 leads to the all too familiar " warning: already initialized constant" stream of errors that results when attempting to redefine what is already defined.

There was talk of introducing a fail-safe where Psych wouldn't load if already loaded, which you'd think would be as simple as adding something like this to the gem's main library file:

if (defined?(Psych))
  # Opportunity to warn or complain about already being loaded here...
end

Unlike most other gems where double-loading is almost always the fault of the developer, Psych is in the relatively unique position of being bundled with Ruby and being distributed as a commonly used gem. It's further complicated by the fact that it has C extensions that need to be loaded as well, and as far as I can tell, these extensions cannot be swapped out once loaded.

In short, allowing Psych to load twice is a bug in the library. It should, at the very least, warn when loaded twice. Ideally it should try and identify what caused it to be loaded prematurely, though this could be tricky to discern.

I'd also like to propose that the version of Psych shipped with Ruby be called something other than Psych so there's no risk of conflict in the future. It needs a different namespace so that it can co-exist with the gem version. This would side-step virtually all o the problems I've experienced and would eliminate a lot of confusion encountered by others if the issues filed here and questions asked in other places are any example.

Whatever YAML library is the default in Ruby should be given a name reflecting this status as "official".

@ghost

ghost commented Jul 10, 2012

👍

I want to share my full stack trace because I think it may lead more lost souls here.

/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundle: warning: already initialized constant ANY
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundle: warning: already initialized constant UTF8
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundle: warning: already initialized constant UTF16LE
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundle: warning: already initialized constant UTF16BE
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/stream.rb:12: warning: already initialized constant ANY
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/stream.rb:15: warning: already initialized constant UTF8
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/stream.rb:18: warning: already initialized constant UTF16LE
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/stream.rb:21: warning: already initialized constant UTF16BE
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/sequence.rb:42: warning: already initialized constant ANY
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/sequence.rb:45: warning: already initialized constant BLOCK
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/sequence.rb:48: warning: already initialized constant FLOW
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/scalar.rb:9: warning: already initialized constant ANY
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/scalar.rb:12: warning: already initialized constant PLAIN
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/scalar.rb:15: warning: already initialized constant SINGLE_QUOTED
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/scalar.rb:18: warning: already initialized constant DOUBLE_QUOTED
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/scalar.rb:21: warning: already initialized constant LITERAL
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/scalar.rb:24: warning: already initialized constant FOLDED
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/mapping.rb:16: warning: already initialized constant ANY
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/mapping.rb:19: warning: already initialized constant BLOCK
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/nodes/mapping.rb:22: warning: already initialized constant FLOW
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/visitors/visitor.rb:10: warning: already initialized constant DISPATCH
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/scalar_scanner.rb:8: warning: already initialized constant TIME
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/scalar_scanner.rb:11: warning: already initialized constant FLOAT
/Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/parser.rb:33:in `': superclass mismatch for class Mark (TypeError)
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/parser.rb:32:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych/parser.rb:1:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych.rb:7:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/1.9.1/psych.rb:7:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/activesupport-3.2.6/lib/active_support/ordered_hash.rb:2:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/activesupport-3.2.6/lib/active_support/ordered_hash.rb:2:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/activesupport-3.2.6/lib/active_support/ordered_options.rb:1:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/activesupport-3.2.6/lib/active_support/ordered_options.rb:1:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails/configuration.rb:2:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails/configuration.rb:2:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails/railtie.rb:2:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails/railtie.rb:2:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails/engine.rb:1:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails/engine.rb:1:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails/plugin.rb:1:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails/plugin.rb:1:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails/application.rb:3:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails/application.rb:3:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails.rb:10:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/railties-3.2.6/lib/rails.rb:10:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/actionpack-3.2.6/lib/action_controller/railtie.rb:1:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/actionpack-3.2.6/lib/action_controller/railtie.rb:1:in `'
    from /Users/david/dev/my_rails_app/config/application.rb:5:in `require'
    from /Users/david/dev/my_rails_app/config/application.rb:5:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /Users/david/dev/my_rails_app/config/environment.rb:2:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /Users/david/dev/my_rails_app/test/minitest_helper.rb:2:in `'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /Users/david/.rbenv/versions/1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'

lacker commented Aug 11, 2012

+1. It also seems impossible to use Psych 1.3.4 with Ruby 1.9.2 and the bundler. With 1.9.2-p180, even:

gem install psych --version 1.3.4
bundle exec irb

leads to "warning: already initialized constant VERSION" errors. This is a shame since the default Psych 1.0.0 in Ruby 1.9.2 has memory leaks.

Owner

tenderlove commented Aug 11, 2012

Has anyone tried to figure out why Psych is being loaded twice?

I don't want to add hacks around if it's loaded twice. That means you could have one version loaded when you expect a different one. I'd be fine raising an exception if Psych has already been loaded once, but that means your process will just quit.

I can help track this down, but I can't seem to reproduce the error on 1.9.3 (and I can't get 1.9.2 building on my machine). Could someone provide me with a way to reproduce this with 1.9.3? Or even better, figure out the root of the problem? Thanks.

@ghost

ghost commented Aug 11, 2012

I'm not getting the error any longer. I'm not sure how to reproduce -- or when it "went away" or got fixed. I'm running Mountain Lion, rbenv, and ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin12.0.0] now.

lacker commented Aug 11, 2012

In 1.9.2-p180, AFAICT the bundler is requiring Psych before your gemfile is actually loaded. So it uses the 1.0.0 Psych version which ships with 1.9.2-p180, which memory leaks. This happens before the bundle is set up. So if I try to include a newer version of Psych via gemfile, it loads twice.

Arguably this is a bundler problem rather than a Psych problem. The bundler logic does not seem to work well with gems that the bundler itself uses that are also provided with the base Ruby distribution. However, I'm not aware of other gems besides Psych that are dual-distributed like this. If the Psych shipped with Ruby was separate from the Psych provided in gems, these conflicts between pre-bundler and post-bundler code wouldn't exist.

Owner

tenderlove commented Aug 11, 2012

@lacker what version of bundler are you using? I sent a patch to them a while back that would this commit makes sure the latest gem is loaded (if it's on your system). That may fix the issue.

lacker commented Aug 13, 2012

I'm using 1.1.5 and that patch is included in my bundler. However, that patch does not appear to have the desired effect. Perhaps psych is getting required before lib/bundler.rb. I will investigate this more - thanks for pointing out that patch.

Owner

tenderlove commented Aug 24, 2012

I was able to reproduce this. IMO, it's a bug in bundler, so I filed an issue.

@croaky croaky referenced this issue in thoughtbot/laptop Feb 24, 2013

Closed

Use rbenv to change Ruby versions #70

Owner

tenderlove commented Jul 5, 2013

Supposedly this was fixed with bundler / rubygems, so I'm closing this issue.

@tenderlove tenderlove closed this Jul 5, 2013

frahugo commented Oct 3, 2013

In case someone would be reading this. My double-loading issue with Psych was caused by rubygems-bundler gem, version > 1.1.0. From 1.1.1, it requires 'yaml', and there the psych.rb provided with my ruby install gets loaded. Downgrading to 1.1.0 fixed my Psych problem.

Thanks @frahugo for pointing out the culprit. It seems they fixed the issue in v1.3.4: mpapis/rubygems-bundler#53

@geoffharcourt geoffharcourt added a commit to geoffharcourt/laptop that referenced this issue Dec 1, 2013

@croaky @geoffharcourt croaky + geoffharcourt Use rbenv to change Ruby versions
* Use Homebrew consistently for OS programs.
* Use ruby-build to install Rubies.
* Include auto-switching of Ruby in `~/.zlogin`.
* Upgrade Rubygems after Ruby is installed. This is to help avoid common
  Bundler/Psych issues:
  ruby/psych#66
* Use newest Bundler version in order to work with latest Rubygems.
* Use rbenv-gem-rehash so the shell automatically picks up binaries
  after installing gems with binaries.
8e572c4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment