Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Top-level Gemfile in multi-gem project "leaks" dependencies #1668
I maintain several multiple-gem in a single git-repo projects. Introducing Bundler in these projects was of great utility for:
To enable the last in particular and keep things as easy to operate as possible, I'm using a single project top-level Gemfile. A much more prominent example of this style of Bundler usage is in rails:
https://github.com/rails/rails (see the top-level Gemfile)
There is a subtle flaw IMO in Bundler's behavior with these setups, in that any gem declared in any of the project gemspecs' is visible or
The following sample project demonstrates this problem in tests:
The alternative approach would be to use independent Gemfile's in each gem subdirectory. There are no dependency leaks in this case. See passing tests for the sample here:
I find myself needing to switch to this per-Gemfile style setup periodically to test that my gemspec dependencies are correct and to fix any dependency issues. I then find myself switching back to take advantage of (2) above, and given that a single top-level Gemfile makes the project easier to use with a single top-level
At minimum I hope to shed light on this "leak" issue. If you guys agree that this is Bundler's issue or the above feature is desirable, I will be happy to attempt an implementation.
Forgive me if I'm misunderstanding something here, but your example Gemfile explicitly declares that gems A, B, and C are all required. That means you have just told Bundler that all three of those gems, and every single child gem of all three of those gems, must be present and loadable once
Thanks for the quick response. I agree with your statement of fact. Here lies the gap between how bundler currently works and what I believe is desirable for a multi-gem project. See my sub-gemfiles sample branch (3rd link above). This is the only way to get completely correct dependency isolation. However by doing this, I lose:
Here's one alternate approach I've been using that allows you, I think, to "have your cake and eat it too".
The top-level Gemfile for the project is like this:
source 'https://rubygems.org' group :development do PROJECT_ROOT = File.expand_path('..', __FILE__) # install all gems needed by sub-libs %w[ lib_a lib_b ].each do |lib| eval File.read(File.join(PROJECT_ROOT, lib, "Gemfile")) end end
In effect, the top-level Gemfile simply reads in the Gemfiles of each subproject. Each subproject has its own gemfile. This ensures that when you are in a subproject, the Gemfile being used is the isolated one their with no extra dependencies. At the top level, you can use "bundler install" to install everything.
That's correct. Dependency isolation in Bundler is limited to per-Gemfile dependency trees, by design.
Rails (to use your "high profile" example from above) does in fact use the project Gemfile in this way, because the Gemfile corresponds to the entire Rails project, and not to any individual gem. The entire idea and point of Bundler is to manage your LOAD_PATH so that you can require any gem either listed at the top level of your Gemfile or depended on by one of those gems.
It sounds like what you're asking for is to be able to explicitly list a gem in your Gemfile, yet still be unable to load that gem. If that's true, I'm afraid we are unlikely to be willing to implement that, since that behaviour would be considered a critical bug.
If what you are after is simply letting your test processes make their own decisions about what gets required, you can easily disable Bundler's auto-require facility by adding
Thanks @myronmarston, I will definitely see how this plays as a practical workaround. I suspect though, that it is incremental: I will still have both top and sub-level Gemfile.lock's to contend with.
@indirect : I hope you will humor my making another attempt to make this issue and its consequences more clear:
Using the sub-gemfiles setup, in sub_b/lib/sub_b.rb, if I
By comparison, by using the sample master branch setup, with one top-level Gemfile: The same code will not fail during development. I will only catch the gemspec problem once I've released it, and a consumer gets a LoadError if hashie is not installed (for example, consumer doesn't also use sub_c, which has the declared hashie dependency).
So currently I'm forced to trade the full set of intended Bundler features and conveniences (described above) to get proper detection of gemspec dependency omissions. It would be better if these multi-gem projects could have all the features with one setup. If you think my above stated feature request is too much magic, or too much of a change without an explicit opt in, then how about if we could preserve the top level Gemfile but add something like the following in a Gemfile to each of the three gem sub-directories:
gemspec :path => '.', :name => 'sub_b', :restrains => '..'
Where the meaning of :restrains would be: use '../Gemfile(.lock)' to resolve/activate all dependencies that are actually listed in sub_b.gemspec, but no other dependencies listed at the top level. And no Gemfile.lock's would be created at the sub-level.
Alternatively this could be rolled into the single, top level Gemfile, i,e:
source :rubygems gemspec :path => 'sub_a', :name => 'sub_a', :constrain => true gemspec :path => 'sub_b', :name => 'sub_b', :constrain => true gemspec :path => 'sub_c', :name => 'sub_c', :constrain => true
With the same effect as above. Example: When I'm running rake/bundler/tests from the sub_b directory, then constrain dependencies to only gems listed in sub_b.gemspec plus transitive dependencies. I'd get the correct LoadError for
Okay, I think I see what you mean. You'd like Bundler to check the current working directory, and give you a subset of the parent directory's Gemfile based on which directory you are working from, right? That is definitely too much magic for me, sorry. :(
As "best practices for library maintainers", we suggest that the Gemfile.lock be added to .gitignore. I would add to that and suggest that you delete your Gemfile.lock before you do any pre-release testing, to effectively check and see if a vanilla
Hopefully the reasoning behind deleting your Gemfile.lock before any pre-release checks run makes sense, and I would love to know if you have any further thoughts on this issue. Thanks!