Skip to content
This repository

Discuss ability to specify branches, specific commits, etc #107

Closed
satazor opened this Issue October 23, 2012 · 58 comments
André Cruz
Owner

The objective of this issue is that people can give some feedback so we can converge towards a solution and an implementation.

Follow up of: #105 and #8

People often find bugs on their dependencies and the typical thing to do is to fork it, fix it, make a PR and use the fork until its accepted (or discarded :P).
This workflow does not directly integrates with bower because the fork would have to contain a new tag with the fixes. The new tag is a must because bower always uses tags and only fallbacks to commits when the repo has no tags at all.

So:

  • How can users target a specific commit?
  • How can users tell bower to always fetch the latest commit, and discard the versions/tags
  • How can users specify a branch while installing?
  • How can users mix this together (e.g.: target X branch and the last commit of it)

One question that arises.. how will the prune/conflict/most suitable version of a shared dependency work out if targeting specific commits/latest commits?

//cc @fat @maccman @addyosmani @sindresorhus @aeosynth @kirkstrobeck

Tobias Birmili

Here is an example which demonstrates what I could think is nice:

{
  'somePackage': '1.2.x', # works like we know it 
  'my-lib-1': 'git://url.to.the/repo#v1.2.3' # a tag
  'my-lib-2': 'git://url.to.some/repo#a4bd91337' # that specific commit
  'my-lib-3': 'git://url.to.other/repo#gh-pages' # always the latest commit on gh-pages
  'my-lib-4': 'git://url.to.other/repo' # always the latest commit on master (default)
}

Because a git checkout treats tags, branches and commit-hashes the same we can consider them the same, too.

While installing I would just also interpret the thing after the hash, so the user has all the possibilities he needs.

And mixing those things together doesn't really make sense with git, if you ask me. Everything is just a pointer to one commit, may it be a branch, tag or commit-hash.

The question with the shared dependency is a though one though… I don't know if bower should do any magic there or just tell the user that libA needs version X of libB which the user has as a "custom" git source…

Kirk Strobeck

@toabi I agree with that spec. I believe in the matter of the shared dependency it should just tell the user.

jorin jorin-vogel referenced this issue in remotestorage/remotestorage.js November 03, 2012
Closed

No pre-built version in the repo #141

André Cruz
Owner

I do agree with almost everything @toabi.

Still, If we change bower install jquery or bower install git://github.com/components/jquery.git to install the latest commit (on master) bower will loose the ability to find the most suited version for the project.

This is an advanced feature and should be used carefully.

Well not necessarly because the component.json would be still be read and the version contained in it would be assumed. But still..

Tobias Birmili

What we could do is: We won't change that the url or package name without a version specifier (#) fetches the latest version, but still the most suited one.

If a user wants the latest commit, then he can simply put git://bla.tld/foo#master there for example. Tadam.

André Cruz
Owner

Agree

André Cruz
Owner

@fat can you give your input?

Jacob
Owner

my opinion is if you need to specify individual commits or branches you're doing it wrong.

Everything should be built around tags. People develop unstable projects in master all the time, and pulling the latest code from the lastest commit in master when you specify a project just isn't safe.

Also, relying on tags will allow us to optimize downloads in the future around tars.

do not change bower to install jquery from the latest commit

lol, jquery itself is unstable in master. cmon.

Maxime Fabre

That is very nice and all but you're kind of talking about an imaginary wonderland where all repos are all clean and tagged with ideal versions conventions and all. This is not the case, Github is a gigantic websites, there are hundreds of components one could want to use and you cannot rely on the idea that everyone will adopt Bower and bow to your conventions. You need to provide gracious fallbacks and means to use repos not using neither tags nor components.json files.

André Cruz
Owner

I have to agree with @Anahkiasen.

While most of the time you won't target specific commits/last commit on branch, it's useful for the workflow described in my post:

People often find bugs on their dependencies and the typical thing to do is to fork it, fix it, make a PR and use the fork until its accepted (or discarded :P).
This workflow does not directly integrates with bower because the fork would have to contain a new tag with the fixes. The new tag is a must because bower always uses tags and only fallbacks to commits when the repo has no tags at all.
Tobias Birmili

I also think that it would be much better to make bower able to work with every repository one can find instead of just blessed ones.

In my opinion bower should be a simple but powerful "We can download all kind of client-side dependencies from everywhere™ and even do magic if people provide proper metadata." - If it would just be a "Manager for blessed libraries with component.json/tags", it would be sad. And I could just go and use http://jamjs.org

Paul Irish
Owner
Addy Osmani
Collaborator

^ :+1:

Jacob
Owner

hm… I think you guys are approaching this the wrong way – the goal shouldn't be to make bower magically consume everything which remotely resembles a package. It should be to educate authors to improve their repos and publish bower consumable packages.

Historically that's how package managers have worked… and there's a reason for that.

Package managers need to rely on package maintainers to tell us when a package is reliable – not the other way around.

Imagine building a high scale application like twitter.com. Or google search. Would you want google search relying on a commit sha?

the idea is kinda hilarious to me, and for that reason i'm completely against it. because people will do it.

what if i rebase my commit history? or delete my branch? it's just too vulnerable.

@paulirish every package manager i know requires a publish step unfortunately. However, I agree it would be nice if you could do it from the github interface – maybe try bringing it up with one of your github homies?

Jacob fat closed this November 12, 2012
Jacob fat reopened this November 12, 2012
Jacob
Owner

sorry didn't mean to close…

Sindre Sorhus

Also, relying on tags will allow us to optimize downloads in the future around tars.

You can use commits to fetch tars.

what if i rebase my commit history? or delete my branch? it's just too vulnerable.

GitHub will still keep around the tar.


So the thing is that a lot of projects rely on anything being in master as useable, especially small projects, and don't bother tagging. While this is not best practice, we have to face reality. We should absolutely not advice user to do their projects this way, but I'm for making the best of the situation and supporting it. Face it, users are going to find ways to misuse your product wether much you try to prevent it.

Nicolas Gallagher
Owner

This discussion seems particularly GitHub-centric at the moment.

So the thing is that a lot of projects rely on anything being in master as useable, especially small projects, and don't bother tagging.

If you're not specifying a fixed commit-ish, then you're not relying on a known, fixed version.

users are going to find ways to misuse your product...

That doesn't mean that you should make it easy for them to do so.

Jacob
Owner

GitHub will still keep around the tar.

wuh? really?

If i have mybranch… and i delete it with git push origin :mybranch… you can download a tar of my deleted branch?

and if i git rebase -i – squash a bunch of commits into a single commit… you can download a tar of a commit i previously pushed?

@sindresorhus can you tell me how to do that… out of curiosity

Nicolas Gallagher
Owner

AFAIK, GitHub does eventually do some clean up, at least w.r.t. the commits you can access by URL…so I'd assume they eventually do with the tars too.

Sindre Sorhus

If you're not specifying a fixed commit-ish, then you're not relying on a known, fixed version.

Ok, should have been clear. I'm not for being able to specify branch, only a specific commit.

AFAIK, GitHub does eventually do some clean up, at least w.r.t. the commits you can access by URL…so I'd assume they eventually do with the tars too.

No idea, they might, but it has worked with a lost commit of grunt for months now, but I see the point on not relying on some undocumented feature. But I think GitHub also keep around commits from deleted branches for some comparison purposes or something.

@fat You can download specific commit using this url: https://github.com/twitter/bower/archive/4f5f36e4b3.tar.gz

But that is moot anyway, since you should never ever rebase something that is pushed.

Nicolas Gallagher
Owner

Yep. Personally, I think it makes more sense to stay focused on git features within the context of package management.

Jacob
Owner

should never ever rebase

i think you may have just joined me in the "imaginary wonderland"

+1 to necolas… it's ok to optimize around github, but not to build features which only work on github.

Kirk Strobeck

seems like a healthy compromise could be to support latest master with latest

{
  'my-lib-5': 'git://url.to.other/repo#latest'
}
Nicolas Gallagher
Owner

seems like a healthy compromise could be to support latest master with latest

What's the use case for specifying a moving pointer rather than a commit-ish?

Sindre Sorhus

i think you may have just joined me in the "imaginary wonderland"

I've always been there. Only thing that keeps me sane these days.

it's ok to optimize around github, but not to build features which only work on github.

@fat Uhm, you were the one suggestion using tars:

Also, relying on tags will allow us to optimize downloads in the future around tars.

What's the use case for specifying a moving pointer rather than a commit-ish?

Lazyness? (carelessness?). Relying on master will eventually break your project. I think we should steer the discussion away from that. What people actually need is to be able to specify a commit sha to be able to work around lazy non-tagging maintainers.

Jacob
Owner

Uhm, you where the one suggestion using tars

Tars as an optimization – not as an all or nothing.

Jacob
Owner

Honestly, i'm done discussing this. If you want to push it through, push it through.

Francisco Facioni

+1 for parsing branches too. A lot of projects are not using tags and there is no easy way to include them. This is a showstopper for using bower for every frontend dependency.

Alex MacCaw
Owner

fwiw, I completely agree with @fat here.

Nami-Doc

+1 to allow branches/commits as versions

It'd be really useful in several cases :
You may want to try out the last bugfix to test out if your usecase is resolved, to try out some feature, etc
Maybe the project maintainer isn't tagging and you want to use the latest version disponible
or maybe you need a very certain version to try something (used as something like bisect).

Adam Biggs

+1

Keep default behaviour the same as it is now, but imho there should be some way to target specific commits for testing/debugging (don't care about targeting branches though).

I totally agree this feature shouldn't be used in production, but sometimes during dev you can't get work done without a specific commit of some dependency.

E.g.: I sent a pull request to SlexAxton/require-handlebars-plugin which has been merged into master, but hasn't been tagged yet. Our app's UI is broken without this minor fix.

To get it through Bower, I had to point to my fork and push up a new tag (for only 2 loc changed). This feels like overkill when I could just go...

"require-handlebars-plugin": "SlexAxton/require-handlebars-plugin#73a4399319"

... and then set it back to "require-handlebars-plugin": "~0.4.1" when the next tag comes out.

Scott Andrews

I'm new to Bower, just downloaded it for the first time this morning, and am in the process of defining component.json files for my projects. Sounds easy enough, until I try to test my work. Whoops, I can only install from a tag. I don't want to tag my code just to test a config before a release, that seems horribly backwards.

In my opinion any git refspec should be installable: tags, commits, branches, whatever. I fully agree that depending on a branch for production code is a bad idea, however, it's incredibly useful for testing changes. Dumb people will do dumb things, it's not the tool's job to be their nanny.

André Cruz
Owner

@scothis Since you are new to bower there is two things that you should know.

  • Bower works without tags. If a repo has no tags, bower will fetch the latest commit. But as soon as a repo has a tag, it won't let you fetch specific commits anymore.
  • Bower has a link command, similar to npm, which helps you debug stuff easily. Check the help of the command for more info.
Scott Andrews

@satazor thanks for the quick reply. In this case I wanted to test the components.json file and it's dependencies. bower link doesn't resolve dependencies, so that's a no go...

I don't want to hijack this thread, so I'll post a question elsewhere if I can't figure this out. I may just chance a tag and fix it later if it's borked.

André Cruz
Owner

@robdodson thing is that the default should be what bower does now, which is checking out tags. But in the end, we should give the ability to target branches, like master.

Rob Dodson

@satazor yeah I'm in a situation now where I'm trying to create two repos, one depending on the other using bower. I'm in the dev stage with both of them, trying to figure out the structure. Having to rely on tags is making it really annoying for development. I'm planning to dev with submodules and then switch to bower when we have our structure worked out. It would be nicer if I could target the master branch at least in this really rough 'sketching' phase.

P.J. Onori

I'm in the same boat. I could really use the ability to specify a branch. +1 for this.

Duran R

having the ability to specify branches would be great +1.

Adam Biggs

FYI, I found that you can target a branch using GitHub's archive service.

So to target the latest commit in master, you can go:

"dependency_name": "https://github.com/user/repo/archive/master.zip"
Rob Dodson

Interesting. It doesn't seem to work with my private repos but it does work for public repos. Unfortunately we have to work in private so we're outta luck there.

Greg Smith

+1 on being able to specify branches / hashes.

If I'm going to use Bower I'd like to manage all my third party dependencies through my components.json file. Where I'm currently comfortable using the semantic versioning and Bower's registry for major libraries that are well maintained, the idea of having to rely on the latest commit on master of a smaller repository makes me cringe.

In the case of hashes, I understand that some moron could rebase their repository and blow away the hash I was depending on, but at least in that case Bower would fail loudly. I prefer this option to a scenario where I can't depend on my software working because I may have grabbed an unvetted version of a dependency that previously worked fine.

Enrique Paredes

Imho we are all almost on the same page, authors should care about his repos and developers should have the ability to specify were to subscribe to. @toabi's syntax seems fair enough to me.

Please remember that bower aims to be The unopinionated package manager™ so let developers opinion about how to screw his project, with messy repos, rebased commits and so on.

Making strong documentation on good practices is good enough to me. Encourage in tag versioning but unforced to use it.

@satazor can you decide with way to go, so we can let our code talks instead our words?

André Cruz
Owner

Yes I will schedule this addition to 0.8.0 milestone.

Gabriel uglymunky referenced this issue in millermedeiros/requirejs-hogan-plugin February 07, 2013
Closed

Please tag latest revision #9

Balázs Reé

I had a look at this issue, and I am a bit confused. What is the status of specifying git branches as dependancies? I am under the impression that this is not yet implemented anywhere, but I may be wrong.

@fat, I understand you are saying that depending on the top of a git branch is a bad practice. The reason why I would find this useful at all, because I am thinking of a specific use case. In this, there is an ongoing development of several packages. Releasing or tagging this packages after each commit would be very unpractical. Still, I would like to enable all developers working on the project, that they get the newest development on each update. Naturally, after the product gets released, it should not depend on unreleased versions of its dependancies, so depending on branches would only occur before releasing, so I agree on this 100%, but still I would like to have a way of development installation.

Provided that you agree on this use case, but at the same time you think that bower should not support this case by allowing dependancies to git branches, then, how would you suggest to support this workflow in a sensible way?

Rob Dodson

yeah my team is basically living @reebalazs 's example. We're using git submodules but being able to target branches for development will be huge.

Michael Grassotti
Nicolas Gallagher
Owner

IMO, developing against a moving target is a bad way to go about things. Even when your package is undergoing rapid iteration, other packages that are depending on it should point to specific commits and undergo their own iteration upon a stable foundation. You wouldn't need to cut a new tag each time if you could reference specific commits (a la npm). But you also don't need to update the package reference for every commit because you're not going to be significantly breaking a package's API or design features every single commit. This leaves it up to a team to communicate when significant changes have taken place in their package, and when authors of a separate package should take a moment to adapt it to deal with the changes that have taken place in one of its dependencies.

Allowing people to specific branch pointers is inevitably going to leak into production and kind of makes a mockery of the very idea of managing package versions. It means that I can't even check out your package (even during "early development") one day and then be sure (when I resolve my project's dependencies) that it will still work the next, despite remaining unchanged itself. One of your dependencies specified with a branch pointer might have changed in a way that blows up your entire package.

AFAICT, this is analogous to creating a feature branch and deciding for yourself when to merge master back into it. You do this at consciously decided moments, e.g., when the team lets you know that master is undergoing significant change and you shouldn't let your feature diverge too far.

I'm happy to be convinced otherwise. But at the moment, I can't see how being able to specify branches is a good thing.

Rob Dodson

@necolas so you're saying don't do branches but do do specific commits yes? Like how git submodules work? If so I'm fine with that as well. I just need something that isn't exclusively tags.

Josh Iverson

I guess I am confused, is bower suppose to protect you as a developer from bad packages? Or try to make repos follow some rule? First off specifically only checking for tags just won't work in a development cycle, you can't keep tagging on every commit that is just unrealistic.

But why put these arbitrary limitations on what someone puts in their repo for others to use? If someone adds in their component.json file that points to a specific branch, so be it, and if I use their package and it starts breaking my shit ....well I probably won't use it for very long or just fork and fix it myself. Either way let us decide, if you supply the gun and I shoot myself, you are not responsible.

As of right now the development team I work on don't use bower because of 2 reasons (the 3rd would be nice to have):
1. Not letting us just point to a branch or specific commit
2. Let us ignore files unneeded instead of pulling the whole repo(I know this is better with the ignore feature but still needs to be a bit more robust)
3. Let the developer installing the package supply our own component.json file so we can specific what we need(enhancement? bower install jquery mycomponent.json)

David DeSandro
Owner

:-1: for now on this feature. In short, I see more problems created over problems solved.

As a reminder, granpappies fat & maccman have already weighed in. necolas's above comment is spot-on.

In addition...


One of Bower's biggest and best features is how it resolves dependencies and versioning. Adding more granular controls like commit-specific versions or more abstract controls like latest-commit to branch would significantly complicate how versions are resolved.

Yes we could add additional feature that commits/branches are only for local development, and then add documentation about why we strip commit/branch versions from the public registry, but then we're adding more features to solve for a previously added feature.


The bigger issue here may not be Bower's feature-set, but with the practice of software development with Bower.

I'm currently working my own Bower-ified project. Unlike some who have chimed in this thread, I'm finding Bower's requirement of explicit versions to be beneficial to my development workflow. This is just me working on the project. My project has a lot dependencies. Even as one person, who has it all in my head, I can't keep track of everything. Working with explicit versions feels a lot more stable than just working off of master for 10+ repos.

If a larger, over-arching project exposes a problem in a dependency, they I go back and address that problem solely in the dependency. A test gets added to the dependency's project. The problem is resolved. The dependency's version is updated. Then bower update back in the larger. If I've properly identified and isolated the problem, I can rest assure that the associated problem will be resolved in the larger project.

Explicit versioning encourages me to keep separate projects separate and un-muddled.

Rob Dodson

@desandro npm already offers this feature and most developers don't seem to be abusing it. In my personal experience it has come in handy when I'm trying to develop a node module. I would be totally fine with bower exploding if this feature was improperly used. As @jiverson stated, that's a problem which would resolve itself when the lib author got an inbox full of complaints.

Rob Dodson

I have a much better understanding of how bower works than when this discussion started 3 months ago. In those 3 months I used git submodules instead of bower because I was in a similar situation to @adambiggs

E.g.: I sent a pull request to SlexAxton/require-handlebars-plugin which has been merged into master, but hasn't
been tagged yet. Our app's UI is broken without this minor fix.

In my case the author had pushed up a broken tag and disappeared for 6 months. Our options were:

  1. Fork the repo, create our own tag and handcode the github url into component.json
  2. Throw bower away and switch to git submodules so we could specify the correct commit for 1 library

We chose option 2 because we thought it would give us the most flexibility if this happened again. It sucks to manage (in our case 14) submodules on a long running project and they're really confusing for team members who aren't savvy with git. Better than doing it all manually but I was hurting for bower during that time. If I could have pointed at the proper commit then I wouldn't have had to throw away my entire package manager. Because really that's what you have to do when one of your bower dependencies isn't properly tagged. You could choose option 1 and keep using bower but that means for the life of that release you better make sure your fork doesn't go anywhere.

Since a tag is really just a commit hash I don't see a huge problem with giving developers the ability to specify a commit as a last resort. It's not a best practice but what recourse do I have in the example I stated? At the end of the day I still had to point at a commit, it was just a lot more work to do so.

I agree with @necolas that pointing at a branch is a bad idea since it could just keep updating. Even if you're just developing, working with commit hashes is a better approach. That's how submodules do it and it keeps things stable.

André Cruz
Owner

I will create a new issue to allow specific commits. With it, we solve the following:

People often find bugs on their dependencies and the typical thing to do is to fork it, fix it, make a PR and use the fork until its accepted (or discarded :P).
This workflow does not directly integrates with bower because the fork would have to contain a new tag with the fixes. The new tag is a must because bower always uses tags and only fallbacks to commits when the repo has no tags at all.

Moving pointers (latest commit on a branch) will not be supported for now. If people are undergoing a short development cycle, bower link can be used to get around it.
This issue will be reopen if necessary.

Thanks all for the useful feedback!

André Cruz satazor closed this February 24, 2013
Balázs Reé

Thanks for all the replies, sorry for not getting back earlier. Github notifications do not work well for me.

I still have the impression that we are talking about two use cases, not one. I do agree that in the first use case, in which the goal is to support the installation of stable released version and their dependancies, it is indeed not desirable to allow a stable version to depend on a moving target.

However there is a second use case in which the developer wants to depend on a continually updating package. Yes, a moving target is not a problem: in this use case, the developer wants exactly this to happen. Moreover, it is essential that the automation is done in the same way (with the same tool and configuration) then it will happen in the final, stable release, because at one point the unstable packages will get released, and the second use case would transform to the first use case. One would not want to switch the toolset just at this point.

Also: targeting specific commits does not offer a solution to the need I am trying to describe, so I am not reasoning for or against that use case.

To extend the focus a little more: I am working for ten or more years in Python, where 'setuptools' was the old package manager. It was far from being perfect, it lacked many features. So, the developers created workarounds to be able to work with the workflow they wanted. One of these solutions was the usage of 'zc.buildout' which is a build tool around setuptools. It also has a recipe called 'mr.developer' that can support working with development packages directly from repositories. The new, remade package manager 'pip' however already supports the use case of installing from git and friends right away. And it does not try to restrict the developers to not use branches and go back to the workarounds. And I believe that many of us Python developers shares the feeling: how great would it have been to have this feature in setuptools earlier.

So, back to bower: I will look for a solution to automate the installation of development packages, maybe with the help of 'bower link', but that can also be automated from a single configuration similar to 'component.json'. I am sure that such a solution is or will be possible in some sensible way. And I still have the opinion that it would be more fortunate if bower could support this use case directly and right away, without the need of a second tool or a workaround. A meaningful restriction (such as, a 'stable' package can not depend on a 'branch' package) may make sense here, but I do not think that bower's complete restriction of the support of branch packages is the best decision.

Again, thanks for the elaborate replies, I hope I can soon find a way to 'scratch my itch' :)

Scott Andrews scothis referenced this issue from a commit in scothis/wire March 18, 2013
Scott Andrews Support bower transitive dependency management
Adding a component.json descriptor that bower uses to resolve transitive
dependencies. These dependencies are independent of Node's package.json
dependencies, but should be kept in sync with package.json changes.
e6f50bd
Eric Lanehart pushred referenced this issue in SparkartGroupInc/pagination June 07, 2013
Merged

Bower vs. Component #1

Jarrod Payne jpdesigndev referenced this issue in LeaVerou/prefixfree August 06, 2013
Closed

Bower is complaining #165

Hugh Winkler

I'm unclear why referencing a branch is not supported. The only reasons I have seen above, seem to say "because developers ought not do that." That's pretty opinionated, and just the sort of attitude I dislike in a project. Have I got it right, or was there some technical barrier?

André Cruz
Owner

@hwinkler you can install a branch in Bower > 1.0.0: bower install xxx#foo-branch.

Hugh Winkler

Oh, thank you much! Will try.

Steffen Müller steffenmllr referenced this issue in js-coder/cookie.js August 19, 2013
Open

Adds a bower.json package description #16

Sergei Dorogin

How to install a package from a branch AND with a tag?
For instance, I have master branch with dev version and release branch with all releases. Some people want to live with dev version and use default master branch. There they have tags with versions to stick on a version for some time (if dev version is broken) - like "1.2.3-snapshot-131209". But other people want to live only on release branch. Release branch also have tags (1.2.3, 1.2.4, 1.3.0 and so on). If they will install as bower install xxx#release then they will get the latest version from release branch. But how to specify a tag in this branch (i.e. a particular version)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.