Add the ability to add a secondary Gemfile (that can be global to the system for example)

[carllerche]

• the information in the secondary gemfile are lower priority
• Do not include the secondary gemfile in a lock
• the secondary Gemfile should not effect the resolve of the main Gemfile. It is a superset on top of it.
• Add the ability to specify this secondary gemfile with an ENV variable, .bundle/config, or flag.

[qmx]

That solves my concern with OS-specific-gems, like autotest-growl

[dolzenko]

+1
Meanwhile, what's the best workaround to the issue? My .irbrc requires http://github.com/oggy/looksee from gem, by accident I discovered that the following lets me break out of the Bundler jail (at least in development mode):

Gem.refresh
Gem.activate('looksee')

so probably I can just detect bundler running and use this hack to require needed gem?

[indirect]

I just put wirble.rb into ~/.irb/wirble.rb, and I require it from there in my .irbrc file.

[dolzenko]

Right, but you can't just vendor a gem into your ~/.irb folder if it has native extensions (like the looksee gem does). Well maybe you can but this is going to be a mess, since you'd have to support compilation, different Ruby versions etc.

[indirect]

Bundler is (obviously, I hope) not going to support putting things into your ~/.irb folder. You could do it as a workaround, though, even if the gem has native extensions. Just copy the files from the installed gem's folder.

[dolzenko]

That doesn't scale when you have multiple extension incompatible Ruby versions (think rvm with 1.8 and 1.9) :)

[indirect]

If you're using rvm, I don't understand how this is a problem at all. You can just bundle the gem straight out of your rvm system gems. Just add it to the Gemfile, and it will use the one you already have installed automatically, and running bundle install will install it to your rvm system gems.

[dolzenko]

That is a problem because you will be adding developer specific gem to the project wide Gemfile, which is

1. repetitive
2. Gemfile may even not be owned by you

that's basically why I believe this feature request was submitted.

[indirect]

I completely agree, and that is why we have committed to adding this feature to bundler. You asked about workarounds, however, and that's what I have been talking about this entire time.

[rickmark]

I'd suggest Gemfile.local for the file...

It can be hacked together with some require 'Gemfile.lock' if File.exists?... until the implementation is ready

[Dykam]

I would like this feature, and the way penwellr suggested seems good. Then you can just ignore it in the vcs ignore settings.

[tomkurt]

Easy solution is just to require with full paths
require "/usr/local/lib/ruby/gems/1.8/gems/wirble-0.1.3/lib/wirble"

if the gem does relative requires like lookse use Dir.chdir("path') do ...;end

Dir.chdir("/usr/local/lib/ruby/gems/1.8/gems/looksee-0.2.1/lib") do
require "looksee/shortcuts"
end

[dolzenko]

I just needed to include "ruby-debug" gem and since it depends on a couple of another gems that approach won't work.
(what I had to do is to manually add all the required load paths with $LOAD_PATH.unshift but that's really awkward)

[pwnall]

Would like to add my use case to the thread: I'd like this feature so that my Rails deployment software can add the application server gem (thin / unicorn) to bundler's list of gems without messing with the Gemfile.

Right now, heroku asks developers to include "gem 'unicorn'" in their gem files. Ideally, the application's source code shouldn't be modified to reflect implementation details in the deployment environment.

[maxim]

+1, can't use irb-enhancing gems without polluting everyone's Gemfile, or creating mess with ugly workarounds.

[ticktricktrack]

+1, We're developing on OSX and Windows. We can either load additional Gemfiles OR put Gemfile.lock in our git repo. Both together would be nice.

[mynameisrufus]

I thought this would work:

https://gist.github.com/724422

But no because we need to commit the Gemfile.lock

I don't like the idea of changing the gems that the app 'actually' needs but for testing its seems like win!

[jtoy]

any progress updates on this?

[indirect]

No, not really. It's an extremely hard problem, and everyone who works on Bundler has dayjobs now. :\

[skojin]

my workaround https://gist.github.com/794915
add this method to .irbrc, allow include gems from rvm @global gemset, you should have bundler installed in this gemset

[rkh]

any updates on this?

@skojin that link is broken.

[skojin]

@rkh link works, copy/paste instead of click (should ends with 15)

[raggi]

module Bundler
  # Activate a gem from outside the Gemfile. Could be considered harmful.
  # Might be useful for .irbrc and friends. It's a slow activation, but
  # after activation, files from the activated gems will be available for
  # normal require.
  def self.activate_outside(*gems)
    # Bundler doesn't cripple this:
    Gem.source_index.refresh!
    # Or this:
    Gem.activate(*gems)
  ensure
    # Re-enable the bundler lockdown via bundlers #initialize hack
    Gem.send(:class_variable_set, :@@source_index, nil)
  end
end

[indirect]

Unfortunately, Gem.source_index is deprecated now, and is going to be removed. :(

[raggi]

I might recommend applying bundlers hacks to Gem.source_index singleton.

That would enable me to simply instance a new Gem::SourceIndex and even cache it, rather than the hack-back included above.

[raggi]

@indirect sure, i'll fix it when that goes and cripple_rubygems is updated.

[mschulkind]

If you just want to be able to require gems from the global gemset, here's an alternative that's working for me. Just stick this in your .irbrc :

# Add all gems in the global gemset to the $LOAD_PATH so they can be used even
# in places like 'rails console'.
if defined?(::Bundler)
  global_gemset = ENV['GEM_PATH'].split(':').grep(/ruby.*@global/).first
  if global_gemset
    all_global_gem_paths = Dir.glob("#{global_gemset}/gems/*")
    all_global_gem_paths.each do |p|
      gem_path = "#{p}/lib"
      $LOAD_PATH << gem_path
    end
  end
end

[phocke]

I tried all the solutions you proposed, but I still can't get over this issue

janlelis/irbtools#12 (comment)

any idea?

[kidpollo]

+1

[imajes]

+1 (for saving etc)

[simi]

@mattdbridges read #183 (comment)

[sheerun]

I understand that's because bundler would overwrite Gemfile.lock with dependencies defined in secondary Gemfile

[mdespuits]

@simi I understand that issue. Who would want that in a production environment? Besides, the idea is the global Gemfile is to be secondary to the app's Gemfile, thus making sure that if any conflicts between them arise, the primary one always wins. See then next two comments to understand what I was thinking about.

[amolpujari]

+1

I run a command(which is result of a gem install from Gemfile), now to run this command I obviously need to point to to a Gemfile having that gem, right?

But my this command tries to execute bundle pointing out to different Gemfile

So need a way to tell bundler that now point out to this new Gemfile, and do your job

http://stackoverflow.com/questions/16336964/chef-solo-rails-bundle-install-when-having-separate-gemfiles

[localredhead]

+1 . This would be useful

[amolpujari]

gems_requires 'path to other Gemfile from this one'

[brandondrew]

+1.
Gemfile.local seems like a good name for files that are specifically for a given project (but not kept in the repo). But if you want something that is global to all your project, but unique to you as a developer / kept out of repos, then I think ~/.Gemfile is a much better name. Both have use cases, so it seems useful to allow either to be used. (Though personally I'd probably only use the ~/.Gemfile, so I can have things like Jazz Hands in all my Rails projects.)

[simi]

@brandondrew you can achieve this behaviour by installing your gems manually (or by creating own meta development gem with only dev. dependencies and no code) and little tricks in $HOME/.irbrc. The problem with secondary Gemfile is written here #183 (comment).

[stephenmckinney]

@simi What if we documented some of these unsupported methods on a wiki page, caveat emptor? "Using gems in a development environment that are not in the gemfile" perhaps. Searching thru outdated gists is no fun.

If Bundler team thinks this is a horrible idea, I trust their judgment. This ticket can just be closed out if there is no intention of working on it. Personally, I'm less concerned about this being officially supported by Bundler and more concerned with just getting things done.

[djspinmonkey]

+1

I've read this whole thread, including #183 (comment), and I don't think that's actually a relevant point. Bundler already allows different sets of gems to be used in development environments than are used in production. That is exactly what groups are most commonly used for -- almost every Gemfile I've ever seen had at least a test group, and usually a development group, too.

So, the new thing that Gemfile.local would allow is not "differences between development and production gem environments", but rather "differences between individual development environments," which I think most people would agree is far less scary.

As somebody who works in a fairly heterogenous environment (both in terms of OS and preferred tooling), this feature would be very useful to me.

[shluboto]

+1
I would really like to use the 'minitest-reporters' gem if it's installed on my local machine.
Don't want to bother my colleagues, who prabably won't like it.

[fxn]

To better understand this proposal, how is bundler going to ensure dependencies are equal? Wouldn't a local Gemfile add more constraints that could end up in a different set of versions?

[shluboto]

You're totally right. I just ran into this problem as I tried to include a Gemfile.local via instance_eval within Gemfile.
As set in my local file 'minitest-reporters' relies on 'minitest < 5' and I want to specify the latest version in the Gemfile...
Too bad

[juanibiapina]

It could resolve dependencies as usual and only after that check if the local dependencies are met (plus any new ones).

If they are not able to resolve properly considering the restrictions of the non local gems, there could be a warning or even an error. The local file wouldn't change the resolved versions of the gems. It's like a lower priority and it becomes your responsibility to fix it or drop it.

[hoff2]

I'd like to +1 this with another use case. Not all Rails apps have only one production deployment, or if they have multiple, those are totally identical. I maintain a CMS app that has 50+ production deployments, each for a different client, and thus each has its own "theme" views and assets. A few of these clients also have some additional custom functionality added to the CMS and site via a plugin in vendor/plugins. Some of those plugins need certain gems. I need to be able to install those gems for those specific sites and have them work. If the Gemfile could include secondary Gemfiles from those plugins that would solve a big problem for me.

[indirect]

@centipedefarmer the ruby code to make this possible isn't actually very hard or complicated. that said, we don't want to make this an officially supported part of Bundler because it just explodes the ways that gemfiles and gems can go horribly wrong. Since Bundler is a small team of volunteers, and we don't have time to provide support for those complicated situations, it's up to people who need it to do it themselves.

[codybrouwers]

For anyone that ends up coming across this thread here what I've done, that seems to work but does still have one flaw:

1. Create a Gemfile.local file and add:

   # Include the regular Gemfile
    eval File.read('Gemfile')

2. Add any other gems you want to use to the group :development[, :test] (I wouldn't suggest doing this for any other environment)
3. Run bundle install --gemfile=Gemfile.local
4. You will now have a Gemfile.local.lock
5. You can now use the gem by running BUNDLE_GEMFILE=Gemfile.local bundle exec {{gem command}}
6. Or to make Bundler prioritize Gemfile.local gems, you can specify in {{app root}}/.bundle/config:
   BUNDLE_GEMFILE: "Gemfile.local"

The flaw in this is that since there are two Gemfile lock files, there isn't a way to ensure consistent versions between the two so something to be aware of. You can at least check for inconsistencies by not adding any additional gems to the Gemfile.local and comparing the two lock files, so it's not a dealbreaker for me.

[indirect]

Thanks for reporting back! I'm glad you've found something that works for your needs. 👍

[rlue]

The flaw in this is that since there are two Gemfile lock files, there isn't a way to ensure consistent versions between the two so something to aware of.
I use the following bash function to keep my lockfiles in sync:

function bundle() {
  bundle="$(type -P bundle)"

  if [ -r "$BUNDLE_GEMFILE" ] && [ -r Gemfile ] &&
     [ "$BUNDLE_GEMFILE" != Gemfile ] && 
     [[ "$1" =~ ^(|install|update)$ ]]; then 
    BUNDLE_GEMFILE=Gemfile "$bundle" "$@"
    cp Gemfile.lock "${BUNDLE_GEMFILE}.lock"
  fi

  "$bundle" "$@"
}

[bmulholland]

@rlue Thanks for that. I created a slight modification to support using a custom gemfile in some projects on the same machine but not others. This just means using the value from bundle config, if present, instead of an environment variable. Use bundle config --local gemfile Gemfile.local to set it for this project only.

function bundle() {
  bundle="$(type -P bundle)"

  gemfile=$BUNDLE_GEMFILE

  if [ -z "$gemfile" ]; then
    gemfile=`bundle config gemfile --parseable | sed -e 's/^gemfile=//'`
  fi

  if [ -r "$gemfile" ] && [ -r Gemfile ] &&
     [ "$gemfile" != Gemfile ] && 
     [[ "$1" =~ ^(|install|update)$ ]]; then 
    BUNDLE_GEMFILE=Gemfile "$bundle" "$@"
    cp Gemfile.lock "${gemfile}.lock"
  fi

  "$bundle" "$@"
}

And for ZSH:

function bundle() {
  gemfile=$BUNDLE_GEMFILE

  if [ -z "$gemfile" ]; then
    gemfile=$(command bundle config gemfile --parseable | sed -e 's/^gemfile=//')
  fi

  if [ -r "$gemfile" ] && [ -r Gemfile ] &&
     [ "$gemfile" != Gemfile ] &&
     [[ "$1" =~ ^(install|update)$ ]]; then
    BUNDLE_GEMFILE=Gemfile command bundle "$@"
    cp Gemfile.lock "${gemfile}.lock"
  fi

  command bundle "$@"
}

[rlue]

@bmulholland I tried this, but faced a strange bug:

When Gemfile.local is specified in the BUNDLE_GEMFILE environment variable, then

BUNDLE_GEMFILE=Gemfile bundle install

successfully updates Gemfile.lock.

When it's set in .bundle/config, however, that command does not update Gemfile.lock. An easy way to test this is to rm Gemfile.lock && bundle install — our custom bundle function will fail on cp Gemfile.lock Gemfile.local.lock.

(In fact, per issue #3081, you're not even supposed to be able to set BUNDLE_GEMFILE from within the Bundler config.)

[bmulholland]

@rlue That's odd - it's working for me. Bundler 1.16.1. Make sure your BUNDLE_GEMFILE environment variable is NOT set, since that will necessarily override the setting in bundle config.

[rlue]

Check and double-check. Wish I had the time to dig deeper.

Make sure your BUNDLE_GEMFILE environment variable is NOT set, since that will necessarily override the setting in bundle config.

Not according to this issue.

[andretf]

I'm using this aliases in my .zshrc:

alias be="bundle exec"
alias bel="BUNDLE_GEMFILE=Gemfile.local be"
alias bei="bundle && BUNDLE_GEMFILE=Gemfile.local bundle"

---

$ bei
$ bel guard

working good..

---

.gitignore_global:

Gemfile.local
Gemfile.local.lock