Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't use require_relative in active_record_migrations.rb #6

Closed
dgutov opened this issue Sep 8, 2014 · 23 comments
Closed

Don't use require_relative in active_record_migrations.rb #6

dgutov opened this issue Sep 8, 2014 · 23 comments

Comments

@dgutov
Copy link

dgutov commented Sep 8, 2014

In our staging environment, it causes the ActiveRecordMigrations.load_tasks call to re-evaluate configurations.rb and thus revert to the default configuration (so loading of the Rakefile fails with something like "db/config.yml not found").

IIUC, because tasks/new_migration.rake has require 'active_record_migrations/configurations', and load_tasks loads that file.

For some reason, that doesn't happen on my (development) machine, but mixing require and require_relative is bad either way.

@rosenfeld
Copy link
Owner

Why is mixing require and require_relative bad? The effect should be absolutely the same:

echo "p 1" > a.rb
echo "require 'a'; require 'b'" > c.rb
echo "require_relative 'a'" > b.rb
ruby -I . c.rb

When you're able to reproduce the problem I'll be able to help you detecting why your configurations file is being loaded more than once. Regular Ruby makes no distinction with regards to files being loaded multiple times whether they are required with require or require_relative...

@rosenfeld
Copy link
Owner

IIUC, because tasks/new_migration.rake has require 'active_record_migrations/configurations', and load_tasks loads that file.

load_tasks doesn't load the configurations file. It loads tasks/new_migration, which requires active_record_migrations/configurations. This should not be a problem since it should simply ignore the require if the file has already been required...

@dgutov
Copy link
Author

dgutov commented Sep 8, 2014

The offending scenario, as I imagined, would go along these lines:

mkdir r
echo "p 1" > r/a.rb
echo "require 'a'" > r/b.rb
echo "require_relative 'r/a'; $: << File.expand_path('../r', __FILE__); require 'b'" > c.rb
ruby c.rb

Looking at $LOADED_FEATURES, though, yeah, both forms seem to have the same result. So all I know currently is that when I changed the statement in question to use the straight up require inside the installed gem (v4.1.2.1) in the problem system, the error went away. I'll ask the ops guy for access to it again tomorrow.

There's no reason to use require_relative here anyway, though, right?

@rosenfeld
Copy link
Owner

There's no need you probably meant. There's no need to use require_relative anywhere as far as I can tell but it's a convenient method if you ask me. I usually prefer require_relative for the local requires as it makes it trivial to understand what specific file is being loaded without worrying about the load path.

Maybe some other file in your project is requiring active_record_migrations/configurations using ActiveSupport's require_dependency or just load? I still think this is a bug in your project itself and if you could send it to me I could try to help you identifying where the problem is really lying on.

@rosenfeld
Copy link
Owner

By the way, I've released 4.1.5.1 early this morning... It shouldn't make any difference but maybe you'd like to know...

@dgutov
Copy link
Author

dgutov commented Sep 8, 2014

There's no need you probably meant.

"No real necessity", to put it more strictly. :)

I usually prefer require_relative for the local requires as it makes it trivial to understand what specific file is being loaded without worrying about the load path.

Yeah, okay. I've always figured using the require is the preferred way, all other things being equal, but maybe that's not true.

You're not using it very consistently, though: its require_relative 'active_record_migrations/configurations', but require "active_record_migrations/version" in the same file. And the other files in the project don't use require_relative at all.

Maybe some other file in your project is requiring active_record_migrations/configurations using ActiveSupport's require_dependency or just load?

I'm pretty sure they don't.

I still think this is a bug in your project itself and if you could send it to me

I'd be happy to, but it's proprietary. I might find out more tomorrow.

By the way, I've released 4.1.5.1 early this morning

I know, thanks.

@rosenfeld
Copy link
Owner

You're not using it very consistently, though: its require_relative 'active_record_migrations/configurations', but require "active_record_migrations/version" in the same file. And the other files in the project don't use require_relative at all.

You're right, I should update the version line to use require_relative as well. There are other places that I find require_relative confusing, specially if it starts with '..', so I prefer regular require in such cases.

In theory, if you have many directories in your load path, require_relative should be faster as it doesn't have to look-up on each possible path although I guess this wouldn't make any noticable performance difference for most projects.

@rosenfeld
Copy link
Owner

It's pretty easy to detect who is loading this file. Just puts caller in the top of the file. You'll know exactly what file is loading it.

@dgutov
Copy link
Author

dgutov commented Sep 9, 2014

Thanks for the tip. :)

Check this out:

$ bundle exec rake -T
loaded by: /srv/beepboop/current/vendor/ruby/1.9.1/gems/active_record_migrations-4.1.5.1/lib/active_record_migrations.rb:6:in `require_relative'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/active_record_migrations-4.1.5.1/lib/active_record_migrations.rb:6:in `<top (required)>'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `require'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `block in require'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:232:in `load_dependency'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `require'
/srv/beepboop/releases/35dfd0db09901ea4b144008a6811f5a1904d888b/Rakefile:2:in `<top (required)>'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/rake_module.rb:28:in `load'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/rake_module.rb:28:in `load_rakefile'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:687:in `raw_load_rakefile'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:94:in `block in load_rakefile'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:176:in `standard_exception_handling'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:93:in `load_rakefile'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:77:in `block in run'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:176:in `standard_exception_handling'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:75:in `run'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/bin/rake:33:in `<top (required)>'
/srv/beepboop/current/vendor/ruby/1.9.1/bin/rake:23:in `load'
/srv/beepboop/current/vendor/ruby/1.9.1/bin/rake:23:in `<main>'
loaded by: /srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `require'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `block in require'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:232:in `load_dependency'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:247:in `require'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/active_record_migrations-4.1.5.1/lib/active_record_migrations/tasks/new_migration.rake:4:in `<top (required)>'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:241:in `load'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:241:in `block in load'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:232:in `load_dependency'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/activesupport-4.1.5/lib/active_support/dependencies.rb:241:in `load'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/active_record_migrations-4.1.5.1/lib/active_record_migrations.rb:20:in `load_tasks'
/srv/beepboop/releases/35dfd0db09901ea4b144008a6811f5a1904d888b/Rakefile:8:in `<top (required)>'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/rake_module.rb:28:in `load'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/rake_module.rb:28:in `load_rakefile'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:687:in `raw_load_rakefile'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:94:in `block in load_rakefile'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:176:in `standard_exception_handling'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:93:in `load_rakefile'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:77:in `block in run'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:176:in `standard_exception_handling'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/lib/rake/application.rb:75:in `run'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/rake-10.3.2/bin/rake:33:in `<top (required)>'
/srv/beepboop/current/vendor/ruby/1.9.1/bin/rake:23:in `load'
/srv/beepboop/current/vendor/ruby/1.9.1/bin/rake:23:in `<main>'
rake aborted!
Errno::ENOENT: No such file or directory - db/config.yml
/srv/beepboop/current/vendor/ruby/1.9.1/gems/active_record_migrations-4.1.5.1/lib/active_record_migrations/configurations.rb:26:in `read'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/active_record_migrations-4.1.5.1/lib/active_record_migrations/configurations.rb:26:in `database_configuration'
/srv/beepboop/current/vendor/ruby/1.9.1/gems/active_record_migrations-4.1.5.1/lib/active_record_migrations.rb:26:in `load_tasks'
/srv/beepboop/releases/35dfd0db09901ea4b144008a6811f5a1904d888b/Rakefile:8:in `<top (required)>'
(See full trace by running task with --trace)

@dgutov
Copy link
Author

dgutov commented Sep 9, 2014

Looking at $LOADED_FEATURES before the load_tasks call, it has these items (note the different directories):

...
"/srv/beepboop/current/vendor/ruby/1.9.1/gems/active_record_migrations-4.1.5.1/lib/active_record_migrations/version.rb",
...
"/srv/beepboop/shared/vendor/ruby/1.9.1/gems/active_record_migrations-4.1.5.1/lib/active_record_migrations/configurations.rb",
"/srv/beepboop/current/vendor/ruby/1.9.1/gems/active_record_migrations-4.1.5.1/lib/active_record_migrations.rb",
...

current/vendor is a symlink to shared/vendor. And I guess require_relative picks up the "real" path to the current file (so it uses the real directory name), whereas require looks at the load path entries, which contain symlinked paths.

Unless you have any further suggestions, I guess we'll have to work around this on our side. Thanks for the help anyway.

@dgutov dgutov closed this as completed Sep 9, 2014
@rosenfeld
Copy link
Owner

Humm... it makes sense. I'll check with ruby-core mailing list if this could be detected somehow in the Ruby side to avoid issues like that. In the meanwhile I'll see if I can find some time later today to replace the require_relative with require and explain the reason when symlinks are used...

@rosenfeld
Copy link
Owner

Also found this ticket: https://bugs.ruby-lang.org/issues/5721

@dgutov
Copy link
Author

dgutov commented Sep 9, 2014

https://bugs.ruby-lang.org/issues/5721

I wonder if require_relative could resolve symlink if it's on the file, but keep the symlinked path if it's on one of the parent directories. That should support both the "intended behavior" and our situation. The question is mostly academical, though, since we're still on Ruby 1.9.3.

@rosenfeld
Copy link
Owner

It's not mostly academical just because it doesn't fix your current application since it should fix Ruby from now on if possible. I believe the proper way is to always point to the real path if possible. This is probably much easier to implement and guarantee it would always work. Let's see what they think about it once I discuss this behavior with them.

@rosenfeld rosenfeld reopened this Sep 9, 2014
@dgutov
Copy link
Author

dgutov commented Sep 9, 2014

I believe the proper way is to always point to the real path if possible.

Hm, yeah, that should work.

@rosenfeld
Copy link
Owner

Sorry but I'm not going to release a new version only due to this change but you can feel free to point to master in your Gemfile if you want.

@dgutov
Copy link
Author

dgutov commented Sep 9, 2014

Thanks, and no problem!

@rosenfeld
Copy link
Owner

Would you mind explaining further how to replicate the issue?

I tried this but didn't experience any troubles... Maybe this is a bug in MRI 1.9.3?

mkdir a
ln -s a b
echo "require_relative 'b'" > a/a.rb
echo "p 'b loaded'" > a/b.rb
ruby -I . -e 'require "b/a"'

I can't open a bug report in ruby-core unless I'm able to reproduce the problem.

@rosenfeld
Copy link
Owner

Taking a closer look at your stacktrace it seems the problem seems to be introduced by ActiveSupport::Dependencies. It overrides Object#require and seems to be causing the bug. Most likely this bug is specific to your project and how AS is used there. Ideally in production require should not be touched by ActiveSupport so most likely your application is still buggy. This is only a work-around on the real issue but you should expect more problems in your application side.

@dgutov
Copy link
Author

dgutov commented Sep 10, 2014

Ideally in production require should not be touched by ActiveSupport so most likely your application is still buggy.

Apparently, just loading ActiveRecord (which we use) drags ActiveSupport::Dependencies with it: https://github.com/rails/rails/blob/master/activerecord/lib/active_record/base.rb#L4
We don't even do require 'active_support/all'.

Anyway, AS doesn't seem to be the problem. Try this:

mkdir a
ln -s a b
echo "require_relative 'b'" > a/a.rb
echo "p 'b loaded'" > a/b.rb
echo "$: << File.expand_path('../b', __FILE__); require 'a'; require 'b'" > c.rb
ruby c.rb

And if I add a to the load path instead, the bug doesn't manifest.

@rosenfeld
Copy link
Owner

Great, I'm going to sleep now and will try this tomorrow morning and discuss with Ruby core team if I can reproduce it. Thank you!

@rosenfeld
Copy link
Owner

I've just created a ticket for it: https://bugs.ruby-lang.org/issues/10222

Thank you for the instructions on how to reproduce it.

@dgutov
Copy link
Author

dgutov commented Sep 10, 2014

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants