made binstubs from bundler overwrite gem generated stubs ... #289

merged 6 commits into from Jun 6, 2011

4 participants

Ruby enVironment Manager member

this should lower mess on disk, no need for ./bin in PATH, already available PATH components are used

Ruby enVironment Manager member

can you provide more details ? why it is wrong way ?

it only cleans the PATH, later on can be extended to allow disabling binstubs

Ruby enVironment Manager member

I see no way that could break things, bundler wrappers do the same thing as rubygems wrappers do, or did you found case when it breaks things ?

Ruby enVironment Manager member

I very much like the idea of them going to $GEM_HOME/bin, how do you force bundler to do that by default with --binstubs ?

Ruby enVironment Manager member

dmitryelastic: bundler should be taking care of it

wayne: I got tired of doing it blind and runned 'bundler --help install' it appears you can just pass path into --binstubs

I have tested it abit with running haml, seams all works well and we can obsolete calling 'bundle exec command' when using rvm

Ruby enVironment Manager member

dmitryelastic that was good point, thanks, I did analyzed all the code, and indeed it is not over, we need to extend RVM to provide per project BUNDLE_GEMFILE environment variable, I will need to talk with Wayne to clarify how to do that (unless I figure it by myself earlier)

just to make it clear ENV['BUNDLE_GEMFILE'] ||= ... means set it if not provided earlier, so providing it with RVM will solve the problem finally.

Ruby enVironment Manager member

@wayne: I know there is something wrong with the way I set BUNDLE_GEMFILE ... I guess you can fix this quite fast

@mpapis mpapis fix BUNDLE-INSTALL(1) BUNDLE-INSTALL(1)
       [1mbundle-install [22m- Install the dependencies specified in your Gemfile

       [1mbundle install [22m[--gemfile=GEMFILE]

                        [--path PATH] [--system]
                        [--without=GROUP1[ GROUP2...]]
                        [--local] [--deployment]

       Install  the  gems  specified  in your Gemfile(5). If this is the first
       time you run bundle  install  (and  a  [1mGemfile.lock  [22mdoes  not  exist),
       bundler will fetch all remote sources, resolve dependencies and install
       all needed gems.

       If a [1mGemfile.lock [22mdoes exist, and you have not updated your Gemfile(5),
       bundler  will fetch all remote sources, but use the dependencies speci-
       fied in the [1mGemfile.lock [22minstead of resolving dependencies.

       If a [1mGemfile.lock [22mdoes exist, and you  have  updated  your  Gemfile(5),
       bundler will use the dependencies in the [1mGemfile.lock [22mfor all gems that
       you did not update, but will re-resolve the dependencies of  gems  that
       you did update. You can find more information about this update process
       below under [4mCONSERVATIVE[24m [4mUPDATING[24m.

              The location of the Gemfile(5) that  bundler  should  use.  This
              defaults  to a gemfile in the current working directory. In gen-
              eral, bundler will assume that the location of the Gemfile(5) is
              also  the  project  root, and will look for the [1mGemfile.lock [22mand
              [1mvendor/cache [22mrelative to it.

              The location to install the gems in the bundle to. This defaults
              to the gem home, which is the location that [1mgem install [22minstalls
              gems to. This means that, by default, gems installed  without  a
              [1m--path  [22msetting  will  show  up  in  [1mgem list[22m. This setting is a
              [4mremembered[24m [4moption[24m.

              Installs the gems in the bundle to  the  system  location.  This
              overrides any previous [4mremembered[24m use of [1m--path[22m.

              A  space-separated  list of groups to skip installing. This is a
              [4mremembered[24m [4moption[24m.

              Do not attempt to connect to [[22m,  instead  using  just
              the  gems located in [1mvendor/cache[22m. Note that if a more appropri-
              ate platform-specific gem  exists  on  [[22m,  this  will
              bypass the normal lookup.

              Switches  bundler's  defaults  into  [4mdeployment[24m [4mmode[24m. Do not use
              this flag on development machines.

              Create a directory (defaults to [1mbin[22m)  containing  an  executable
              that  runs  in  the  context of the bundle. For instance, if the
              [1mrails [22mgem comes with a [1mrails [22mexecutable, this flag will create a
              [1mbin/rails  [22mexecutable  that  ensures  that all dependencies used
              come from the bundled gems.

       Bundler's defaults are optimized for development. To switch to defaults
       optimized  for  deployment,  use the [1m--deployment [22mflag. Do not activate
       deployment mode on development machines, as it will cause in  an  error
       when the Gemfile is modified.

       1.  A [1mGemfile.lock [22mis required.

           To ensure that the same versions of the gems you developed with and
           tested with  are  also  used  in  deployments,  a  [1mGemfile.lock  [22mis

           This  is  mainly  to  ensure  that  you remember to check your [1mGem-[0m
           [1mfile.lock [22minto version control.

       2.  The [1mGemfile.lock [22mmust be up to date

           In development, you can modify your Gemfile(5)  and  re-run  [1mbundle[0m
           [1minstall [22mto [4mconservatively[24m [4mupdate[24m your [1mGemfile.lock [22msnapshot.

           In  deployment, your [1mGemfile.lock [22mshould be up-to-date with changes
           made in your Gemfile(5).

       3.  Gems are installed to [1mvendor/bundle [22mnot your default  system  loca-

           In  development,  it's  convenient  to  share the gems used in your
           application with other applications and other scripts  run  on  the

           In  deployment, isolation is a more important default. In addition,
           the user deploying the  application  may  not  have  permission  to
           install  gems to the system, or the web server may not have permis-
           sion to read them.

           As a result, [1mbundle install --deployment [22minstalls gems to the  [1mven-[0m
           [1mdor/bundle  [22mdirectory  in  the  application. This may be overridden
           using the [1m--path [22moption.

       By default, bundler installs gems to the same location as [1mgem install[22m.

       In some cases, that location may not be writable by your Unix user.  In
       that case, bundler will stage everything in a temporary directory, then
       ask you for your [1msudo [22mpassword in order to copy  the  gems  into  their
       system location.

       From  your  perspective,  this  is  identical  to  installing them gems
       directly into the system.

       You should never use [1msudo bundle install[22m. This is because several other
       steps in [1mbundle install [22mmust be performed as the current user:

       +o   Updating your [1mGemfile.lock[0m

       +o   Updating your [1mvendor/cache[22m, if necessary

       +o   Checking out private git repositories using your user's SSH keys

       Of  these  three,  the  first  two  could theoretically be performed by
       [1mchown[22ming the resulting files to [1m$SUDO_USER[22m.  The  third,  however,  can
       only  be  performed by actually invoking the [1mgit [22mcommand as the current
       user. Therefore, git gems are downloaded and installed  into  [1m~/.bundle[0m
       rather than $GEM_HOME or $BUNDLE_PATH.

       As  a  result,  you  should run [1mbundle install [22mas the current user, and
       bundler will ask for your password if it is needed to put the gems into
       their final location.

       By  default, [1mbundle install [22mwill install all gems in all groups in your
       Gemfile(5), except those declared for a different platform.

       However, you can explicitly tell bundler  to  skip  installing  certain
       groups  with  the [1m--without [22moption. This option takes a space-separated
       list of groups.

       While the [1m--without [22moption will skip [4minstalling[24m the gems in the  speci-
       fied  groups, it will still [4mdownload[24m those gems and use them to resolve
       the dependencies of every gem in your Gemfile(5).

       This is so that installing a different set of groups on another machine
       (such  as  a  production  server) will not change the gems and versions
       that you have already developed and tested against.

       [1mBundler offers a rock-solid guarantee that the third-party code you are[0m
       [1mrunning in development and testing is also the third-party code you are[0m
       [1mrunning in production. You can choose to exclude some of that  code  in[0m
       [1mdifferent  environments,  but  you  will never be caught flat-footed by[0m
       [1mdifferent versions of third-party code being used in different environ-[0m

       For a simple illustration, consider the following Gemfile(5):

           source ""

           gem "sinatra"

           group :production do
             gem "rack-perftools-profiler"

       In  this  case,  [1msinatra  [22mdepends on any version of Rack ([1m>= 1.0[22m, while
       [1mrack-perftools-profiler [22mdepends on 1.x ([1m~> 1.0[22m).

       When you run [1mbundle install --without  production  [22min  development,  we
       look  at the dependencies of [1mrack-perftools-profiler [22mas well. That way,
       you do not spend all your time developing against Rack 2.0,  using  new
       APIs  unavailable  in Rack 1.x, only to have bundler switch to Rack 1.2
       when the [1mproduction [22mgroup [4mis[24m used.

       This should not cause any problems  in  practice,  because  we  do  not
       attempt  to  [1minstall [22mthe gems in the excluded groups, and only evaluate
       as part of the dependency resolution process.

       This also means that you cannot include different versions of the  same
       gem  in  different  groups,  because doing so would result in different
       sets of dependencies used in development and production. Because of the
       vagaries  of  the  dependency  resolution process, this usually affects
       more than just the gems you list in your Gemfile(5), and can  (surpris-
       ingly) radically change the gems you are using.

       Some  options  (marked  above  in  the  [4mOPTIONS[24m section) are remembered
       between calls to [1mbundle install[22m, and by the Bundler runtime.

       For instance, if you run [1mbundle install --without  test[22m,  a  subsequent
       call  to  [1mbundle  install  [22mthat  does not include a [1m--without [22mflag will
       remember your previous choice.

       In addition, a call to [1mBundler.setup [22mwill not attempt to make the  gems
       in  those  groups  available  on  the  Ruby load path, as they were not

       The settings that are remembered are:

              At runtime, this remembered setting will also result in  Bundler
              raising an exception if the [1mGemfile.lock [22mis out of date.

       [1m--path [22mSubsequent  calls  to  [1mbundle  install  [22mwill install gems to the
              directory originally passed to [1m--path[22m. The Bundler runtime  will
              look  for  gems  in that location. You can revert this option by
              running [1mbundle install --system[22m.

              Bundler will update the executables  every  subsequent  call  to
              [1mbundle install[22m.

              As  described  above,  Bundler  will  skip the gems specified by
              [1m--without [22min subsequent calls to  [1mbundle  install[22m.  The  Bundler
              runtime will also not try to make the gems in the skipped groups

       When you run [1mbundle install[22m, Bundler will persist the  full  names  and
       versions  of all gems that you used (including dependencies of the gems
       specified in the Gemfile(5)) into a file called [1mGemfile.lock[22m.

       Bundler uses this file in all subsequent calls to [1mbundle install[22m, which
       guarantees that you always use the same exact code, even as your appli-
       cation moves across machines.

       Because of the way dependency resolution works, even a seemingly  small
       change (for instance, an update to a point-release of a dependency of a
       gem in your Gemfile(5)) can result in radically  different  gems  being
       needed to satisfy all dependencies.

       As  a  result, you [1mSHOULD [22mcheck your [1mGemfile.lock [22minto version control.
       If you do not, every machine that checks out your repository (including
       your production server) will resolve all dependencies again, which will
       result in different versions of third-party code being used if  [1many  [22mof
       the  gems  in  the  Gemfile(5)  or  any of their dependencies have been

       When you make a change to the Gemfile(5) and then run  [1mbundle  install[22m,
       Bundler will update only the gems that you modified.

       In  other  words,  if  a  gem that you [1mdid not modify [22mworked before you
       called [1mbundle install[22m, it will continue to use the exact same  versions
       of all dependencies as it used before the update.

       Let's take a look at an example. Here's your original Gemfile(5):

           source ""

           gem "actionpack", "2.3.8"
           gem "activemerchant"

       In  this  case, both [1mactionpack [22mand [1mactivemerchant [22mdepend on [1mactivesup-[0m
       [1mport[22m. The [1mactionpack [22mgem depends on [1mactivesupport  2.3.8  [22mand  [1mrack  ~>[0m
       [1m1.1.0[22m,  while the [1mactivemerchant [22mgem depends on [1mactivesupport >= 2.3.2[22m,
       [1mbraintree >= 2.0.0[22m, and [1mbuilder >= 2.0.0[22m.

       When  the  dependencies  are  first  resolved,  Bundler   will   select
       [1mactivesupport  2.3.8[22m,  which satisfies the requirements of both gems in
       your Gemfile(5).

       Next, you modify your Gemfile(5) to:

           source ""

           gem "actionpack", "3.0.0.rc"
           gem "activemerchant"

       The [1mactionpack 3.0.0.rc [22mgem has  a  number  of  new  dependencies,  and
       updates  the [1mactivesupport [22mdependency to [1m= 3.0.0.rc [22mand the [1mrack [22mdepen-
       dency to [1m~> 1.2.1[22m.

       When you run [1mbundle install[22m,  Bundler  notices  that  you  changed  the
       [1mactionpack  [22mgem,  but not the [1mactivemerchant [22mgem. It evaluates the gems
       currently being used to satisfy its requirements:

       [1mactivesupport 2.3.8[0m
              also used to satisfy a dependency in  [1mactivemerchant[22m,  which  is
              not being updated

       [1mrack ~> 1.1.0[0m
              not currently being used to satify another dependency

       Because  you did not explicitly ask to update [1mactivemerchant[22m, you would
       not expect it to suddenly stop working after updating [1mactionpack[22m.  How-
       ever,  satisfying  the new [1mactivesupport 3.0.0.rc [22mdependency of action-
       pack requires updating one of its dependencies.

       Even though [1mactivemerchant [22mdeclares a very loose dependency that  theo-
       retically  matches  [1mactivesupport 3.0.0.rc[22m, bundler treats gems in your
       Gemfile(5) that have not changed as an atomic unit together with  their
       dependencies. In this case, the [1mactivemerchant [22mdependency is treated as
       [1mactivemerchant 1.7.1 + activesupport  2.3.8[22m,  so  [1mbundle  install  [22mwill
       report that it cannot update [1mactionpack[22m.

       To explicitly update [1mactionpack[22m, including its dependencies which other
       gems in the Gemfile(5) still depend on, run  [1mbundle  update  actionpack[0m
       (see [1mbundle update(1)[22m).

       [1mSummary[22m:  In  general,  after  making  a change to the Gemfile(5) , you
       should first try to run [1mbundle install[22m, which will  guarantee  that  no
       other  gems  in the Gemfile(5) are impacted by the change. If that does
       not work, run bundle update(1) [4mbundle-update.1.html[24m.

                                   May 2011                  BUNDLE-INSTALL(1)
Ruby enVironment Manager member

@wayneeseguin do not merge this patch yet, I will need to provide more patches, to make it bulletproof - usecase with calling gem install for gems from Gemfile after bundle install

Ruby enVironment Manager member

"if several different rails projects are using the same gemset" -- If someone is doing this then I have absolutely no compassion for them :)

@wayneeseguin wayneeseguin merged commit 927f045 into rvm:master Jun 6, 2011

I don't think it is good idea to manipulate with BUNDLE_GEMFILE - it broke my Jenkins continuous integration builds as BUNDLE_GEMFILE was not pointing to Gemfile in current build workspace. This automatic manipulation with Gemfile location can cause a lot of confusion.

Ruby enVironment Manager member

rsim, we are working on it to be a lot less intrusive


"if several different rails projects are using the same gemset" -- If someone is doing this then I have absolutely no compassion for them :)

I use RVM to manage which ruby version I'm using, but I use @global on all of them and manage my gems via bundler. Doesn't this put me in the compassionless bucket?

Ruby enVironment Manager member

1) using global is not a good idea, but as you use only it then it should work,
2) this patch was already reverted and I have created a gem that provides this or even better functionality

Ruby enVironment Manager member

@tsaleh Bundler currently does not honor the GEM_PATH hierarchy as it should. The Bundler team has agreed to alter this behavior and honor it. Until then stick with one project per gemset and all gems for it in the project specific gemset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment