Skip to content
This repository

Add :dest and :path sub-options to :git #396

Closed
dchelimsky opened this Issue June 22, 2010 · 33 comments

8 participants

David Chelimsky André Arko José Valim Hedgehog Stephan Schubert Danny Burkes Tim Uckun James Hu
David Chelimsky

I'd like the ability to specify both the :path and :git keys in the same gem declaration:

gem "rails", :path => "./where/i/want/to/store/it", 
  :git => "git://github.com/rails/rails.git"

During bundle install, if the repo exists at :path, use it. If not, clone from the :git location and store it at :path. During bundle update, do a git pull from the repo located at :path.

David Chelimsky

Here is the specific use case for this:

rspec-rails' development environment depends on rails being included in the bundle. It also requires that rails be stored in a local repo so that we can easily get from one version of rails to another by simply checking out the appropriate tag. This is stored ./vendor/rails.

The Gemfile in the root of the rspec-rails project includes:

gem "rails", :path => "./vendor/rails"

This leads to a catch 22, because when you first clone the rspec-rails repo and run "bundle install" the rails repo hasn't been cloned at ./vendor/rails yet. This can't be managed with a rake task, because the Rakefile operates in the same bundle, which won't load because the ./vendor/rails repo is not present. So the only way to get into a working state is to manually clone the rails repo, after which all is right with the world.

If bundler added support for :path and :git in the same declaration, bundle install would "just work." Yay.

André Arko
Owner

So :path by definition implies that you are saying "I'm going to manage this myself. Bundler, don't mess with it."

The reason I am wary of implementing this is that I had previously thought about path + git support, but it did the exact opposite: if the path is not present, it would use the git repo as a standard git gem instead. That's because a lot of people use :path locally but need to use :git when they deploy.

While I'm not sure about adding this to bundler without some more discussion, have you thought about creating a simple bootstrap.sh or similar file that checks out rails for you?

David Chelimsky

This isn't about me, it's about my users. I want rspec-rails development to be as easy as possible for contributors, and having to clone, bootstrap, and bundle install is 50% more steps than necessary :)

André Arko
Owner

Well, the bootstrap would obviously clone and then bundle install... so it's still a single step, just a different one than bundle install. :)

David Chelimsky

Sure, but git clone and bundle install are widely known and understood commands. It'd be nice to just rely on those. Also, if a person doesn't know to use the bootstrap script, then he'll just get an error when trying to run rake or bundle. What I did to work around this is check at the top of the rakefile if the repos are cloned already, and then raise a helpful error if not. I don't have any way of doing that with the bundle command.

So perhaps another solution would be to have a way to manage messages emitted by bundler.

gem "rails", :path => "./vendor/rails", :message => "./vendor/rails does not exist. Try running 'git clone git://github.com/rails/rails.git ./vendor/rails'

WDYT?

André Arko
Owner

I feel like managing the version of vendored rails in this fashion is outside the scope of what bundler is intended to do, but I'll continue to think about this. I may become persuaded.

André Arko
Owner

I'm also curious to hear your thoughts on this issue as it interacts with #231, which is a request to use :path in development and :git in production.

David Chelimsky

I'm uncomfortable with expecting different behavior in different environments. I think it would lead to confusion, especially when the local and remote HEAD commits are different.

André Arko
Owner

So how would you handle the case of local development and production deployment? Simply say "that's not allowed"? Suggest the hacky ENV workaround explained in #231?

David Chelimsky

Why not just do that by groups:

group :development do
  gem "mongoid", :path => "#{ENV['DEV_BUNDLE']}/mongoid"
end
group :production do
  gem "mongoid", :git => "git://github.com/durran/mongoid.git"
end
André Arko
Owner

Because unfortunately that doesn't really make sense -- --without doesn't actually remove groups from the Gemfile resolve, because that would break bundling entirely.

Using groups in that manner declares two gems, both named mongoid, one of which is at a path that may or may not exist, and the other is in git. How can bundler arbitrate between two contradictory gem declarations in a single manifest? :|

David Chelimsky

Couple of thoughts come to mind.

  1. :only and/or :except
gem "mongoid", :git => "git://github.com/durran/mongoid.git", :only => :production
gem "mongoid", :path => "local/path/to/mongoid", :except => :production
  1. :env key
gem "mongoid", :env => {
  :development => { :path => "local/path/to/mongoid" },
  :production => { :git => "git://github.com/durran/mongoid.git" }
}

Both have benefits and drawbacks. Just floating ideas.

José Valim
Collaborator

I need something like this for Devise. Devise depends on Rails and I have both checked out as a git repository in the same folder. Since I don't want bundler to checkout Rails once again, I do this in my Gemfile:

git "rails", :path => "../rails"

But that obviously won't work if you don't have Rails there. So it would be nice to be able to say: use this path, if not, check it out. For me, I don't care if rails will be checked out at "../rails" or BUNDLER_DIR, both would work great in my case.

André Arko
Owner

I talked this over with yehuda, and I think this is a really nice option to add to Bundler, but we should name it something else because it totally conflicts with the idea behind :path. How about this:

 gem "rails", :git => "...", :dest => "vendor/rails"

We could go with :dest, or :destination, or :clone_to, but I think I like :dest best. What do you guys think?

José Valim
Collaborator

I think David and I have different use cases. David wants to vendor it inside, I simply want to use that path/dest and, if not available, checkout the git repo (probably at the usual ~/.bundle location). In any case, :dest as option sounds great!

André Arko
Owner

I think I am proposing that :path + :git be used for José's case (try this path, if it's not there, then clone this git repo as usual), and :dest + :git be used for David's case (try this path, if it's not there, then clone this git repo into that path). I will of course add documentation for these options to the git page of gembundler.com, since those uses aren't exactly obvious. Hopefully, though, the combination of :path and :git I am proposing is such that nothing will break.

José Valim
Collaborator

Awesome! Victory! Success!

David Chelimsky

Both use cases say "try this path, then do something else". Where they differ is in the "do something else" part, not in the "path" part, and the difference is the destination. José's case is to store it in the bundle, mine is to store it in the :path. So how about this:

gem "rails", :git => "..", :path => "vendor/rails", :dest => :bundle
gem "rails", :git => "..", :path => "vendor/rails", :dest => :path

The default could be :bundle, so you José can just say:

gem "rails", :git => "..", :path => "vendor/rails"

WDYT?

David Chelimsky

Either that, or something like this:

gem "rails", :path => "vendor/rails", :alt => { :git => "..", :dest => :bundle }

gem "rails", :path => "vendor/rails", :alt => { :git => "..", :dest => :path }

Very explicit.

André Arko
Owner

I guess they don't seem like the same thing to me because in the one case, :path is the main option, and :git is a fallback in case :path doesn't work. In the other case, :git is the main option, and :dest is a sub-option to :git, like :ref is.

I'm not sure yet how to make that clear in the API, though.

José Valim
Collaborator

"I think I am proposing that :path + :git be used for José's case (try this path, if it's not there, then clone this git repo as usual), and :dest + :git be used for David's case (try this path, if it's not there, then clone this git repo into that path)."

+1 for this approach.

Hedgehog

Is it not better to make any src-dest and fallback behavior explicit, unlimited and consistent between :path and :git?
Allow both :git and :path to accept a:

  • string (specific)
  • hash (specific)
  • array (common)

Current behavior to remain (call these git-strings and gem-strings use cases):

gem "nokogiri", :git => "git://github.com/tenderlove/nokogiri.git"
gem "rails", :path => "./vendor/rails"

Inroduce a 'Git-hash'

git_hash= { :git => "<git-string>", :dest => "./where/i/want/to/store/it", :ref => "...", etc.}

Also allow a 'Gem-hash':

gem_hash= { :path => "gem-string", :dest => "./where/i/want/to/store/it", :version => "...", etc.}

In both hashes :dest => :bundle and :dest => :path are valid, defaulting to the current behavior.

try this path, if it's not there, then clone this git repo as usual
etc.

gem "rails", :path => gem_hash
gem "rails", :path => ["./vendor/rails", git_hash, gem_hash, "./other/path"]

gem "rails", :git => git_hash
gem "rails", :git => ["./vendor/rails", git_hash2, gem_hash2, "./other/path", etc.]

Note that when the :git/:path array element contains a string, it behaves just as the current deafult:

gem "rails", :path => "./vendor/rails"

Or when the string is any valid Git URI:

gem "nokogiri", :git => "git://github.com/tenderlove/nokogiri.git"

Specifically

  • When :git => points to a hash a valid git-hash is expected.
  • When :path => points to a hash a valid gem-hash is expected.

Finally, the :git/:path behave identicaly when given an array, and are able to process:

  • git-strings (unchanged from current behavior)
  • gem-strings (unchanged from current behavior)
  • git-hashes
  • gem-hashes.

I believe this solves dchelimsky's:

I'd like the ability to specify both the :path and :git keys in the same gem declaration

and

store it in the :path

and also josevalim's:

try this path, if it's not there, then clone this git repo as usual

and

store it in the bundle

André Arko
Owner

Your "gem hash" doesn't make any sense. Why would you ever combine :path and :dest? That is a totally meaningless combination. Your "git hash" is what I'm already planning on doing, without the hash. No reason to make things more complicated if it's not needed.

Hedgehog

Fair enough, drop it. From my reading of the above comments fallback behavior was really valued, and it'd be nice to have that behave consistently between :path and :git. Admittedly in my proposal it is the case that when given an array both :path and :git behave as if they are aliases.
Personally I alwsays thought :path was ambiguous, :dest is more explict, but it seems too late now - or should :dest be introduced and :path deprectade, but retained unitl 1.1? I suspect the vote would be to keep it, then introduce :dest at 1.x and try unring the :path bell later ;)

Hedgehog

As my comment above makes clear, the function of :path is not unambiguous to users. :path cannot be deprecated without some replacement and I think I've decoded what :path is supposed to do and it is not as generic as its name suggests it is....

On reflection I think the correct way to address the 'issues' with :git and :path is in Issue #577.
I've seen some of josevalim and dchelimsky efforts so I know they are not dim - if they are having issues then I am not alone and I suspect most others will too.

indirect: please think then count to 100 before dismissing and closing Issues #577, your perfromance in both Issues #319 and #320 didn't leave an impression of taking thoughtful care in closing tickets - hopefully it was just a bad week.

Stephan Schubert
jazen commented June 21, 2011

+1 for the "Try :path first, use :git otherwise" or "Different sources in different groups" solutions..

This issue is really annoying when developing a main application and several gems at the same time :/

Danny Burkes

What's the status on this? From the comments, it looks like it has been fallow for almost a year. Is this planned?

André Arko
Owner

The status is "on the list of things that would be nice to implement when we have time". The core team has limited time and is spending most of it on more universally applicable issues like speeding up bundle install. If someone writes a patch for this, I will try to get it in. If no one writes a patch for this, I will add it to Bundler when I have time to do that.

Tim Uckun

Just like to interject this idea.

There should be a :host option for gems. This way each developer can specify their own path to the gem and also specify different gems for different machines whether production or development.

James Hu

Wasn't this feature supposed to be scheduled for 1.1 as per this blog post (scroll to bottom)? http://www.cowboycoded.com/2010/08/10/using-2-sources-for-a-gem-in-different-environments-with-bundler/

+1 from me, this feature is something that would help gem development immensely! Hope it gets implemented soon.

As of right now, I'm doing it like this:

require 'rails'

if Rails.env == "production"
    gem "foo", :git => "/path/to/repo"
else
    gem "foo", :path =>  "/path/to/gem"
end
Tim Uckun
André Arko
Owner
André Arko
Owner

Whoa, didn't realize this ticket was still open. :) As of version 1.2, Bundler allows you to use a git repo that is checked out onto your machine without editing your Gemfile via a particular bundle config setting. Feel free to check out the git locals documentation for more information.

André Arko indirect closed this January 28, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.