Implement 'shrinkwrap' feature #505

Open
necolas opened this Issue May 27, 2013 · 188 comments

Projects

None yet
@necolas
Member
necolas commented May 27, 2013

Bower could do with something like npm's shrinkwrap feature to lock down specific versions for deploys.

https://npmjs.org/doc/shrinkwrap.html

This would probably be simpler than what npm has to do, as we'd just need to create a config file to represent the currently installed versions.

Related #416

@sindresorhus
Member

I want us to do this right, meaning checksum checking for everything to ensure validity. Npm doesn't currently implement this, but are planning to.

@adambiggs

πŸ‘ This would be a really great feature.

@thomaswelton

Must say πŸ‘ in comparison to the npm and composer install I run bower is super slow. And is a real pain.
I want my repos to be practically empty, and the diffs and commits to be clean. But after going through a phase of committing my bower_components as suggested here #416 I really hated it. Looking forward to this. πŸ˜„

@rogeriopradoj

πŸ‘ for sure it will be an excellent feature!

I'm not completely sure how npm works on this, but the way Composer does totally fits my needs.

[OFF-TOPIC]
I'm really happy that the guys in https://github.com/RobLoach/component-installer have built an way to install frontend components via Composer. This is way we can use the composer.lock while the bower.lock (or something close) is not ready. It's a pity there are too few components https://github.com/components/ available, but at least Twitter Bootstrap is there.

@jasoncrawford

+1. However, I think there's a simple workaround for now. Since Bower's dependency tree is flat, you can just specify exact versions of projects in your bower.json. E.g., specify Bootstrap "2.3.2" instead of "~2.3.2" or "2.3.x" or whatever you do. Then bower install will always install the same versions.

(Of course, the flat dependency tree leeds to version conflict problems, not sure what the long-term solution to that is.)

@necolas
Member
necolas commented Jun 6, 2013

It's not quite a solution, because your immediate dependencies will have non-fixed version numbers for their dependencies. So even if you lock the versions down, every time bower install is performed, you might get different versions of dependency-dependencies installed.

@jasoncrawford

Ah, good point. I guess you would have to take all the dependencies (recursively all the way down the tree) and explicitly include them at the top level.

@ostera
ostera commented Jun 11, 2013

πŸ‘

@rubennorte

I'm joining a little late this discussion, but I'd like to suggest a way to implement this.

I think this could be done the way Bundle does with Gemfile and Gemfile.lock. We could specifiy dependencies in bower.json, install them with bower install or bower update and then a bower.lock (or a more appropiate name) file would be created in the same directory with all the dependencies referencing the exact commit that has been used.

bower.json example:

{
  "name": "my-component",
  "version": "0.1.0",
  "ignore": [
    "**/.*",
    "node_modules",
    "components",
    "bower_components",
    "test",
    "tests"
  ],
  "dependencies": {
    "jquery": "~1.9.0",
    "bootstrap": "~2.3.0",
    "underscore": "~1.4.0"
  }
}

bower.lock example:

{
  "dependencies": {
    "jquery": "#28d946384799abcd7608c3beda898a0a11ac88d7",
    "bootstrap": "#d9b502dfb876c40b0735008bac18049c7ee7b6d2",
    "underscore": "#484bdb43ec4a9dd6a40e60a2d25317bec7aeb43f"
  }
}

Doing so we could deploy exactly the same version of the components we have installed in our development environment without the need of including them in the VC. This could not be a problem for some people, but many components, like jquery-mobile (16 MB), jquery-ui (4.9 MB) or bootstrap (5.1 MB) are including the whole repo as the component (ignoring the ignore option to exclude documentation, tests, binary files, etc.) and making your repo needlessly big.

I think this could be an easy to implement option and that it should be taken into account. I also expect to take some time to implement this and submit a pull request. Meanwhile I'll be watching this thread waiting for your suggestions.

Thanks.

@mikaelkaron

Would it be better that a final version instead of the commit in the lock file?

Meaning that ~1.2 may result in the exact version 1.2.3 and that's the version that would go into the lock file.

@satazor
Member
satazor commented Aug 6, 2013

@mikaelkaron as long as we implement checksums, I think it's better to use the version instead of the commit. It's also faster for github endpoints to download versions, instead of checking out commits.

@roelvanduijnhoven

This would be great to make sure development and production versions are identical!

The activity in this discussion suggest that such a feature is not being worked on at the moment. What can be done to make sure steps are made in the right direction?

@twisk
twisk commented Oct 23, 2013

πŸ‘

@necolas
Member
necolas commented Oct 23, 2013

What can be done to make sure steps are made in the right direction?

Kick off an implementation design and some code for the feature :)

@stefanpenner

one issue i see with npm's shrinkwrap over bundlers lockfile, is that the shrinkwrap is not maintained and used to re-validated in all scenarios. This makes it annoying to maintain, and error-prone. I would strongly suggest modeling this around how Gemfile.lock workflow.

@julien-c

Or composer.lock

@szimek
szimek commented Oct 28, 2013

+1 for the Bundler-like approach with bower.lock file.

Is it currently possible to programmatically check (e.g. when running grunt server) if user has the current versions of libraries installed?

@hxgdzyuyi

πŸ‘

@evillemez

+1 for some type of bower.lock. This feature is really important for maintenance.

@kof
kof commented Nov 18, 2013

+1, please implement it, currently we are forced to put all the bower components into version control to ensure nothing will change on deployment.

@DavidMikeSimon

+1 this is crucial for deployment

@shouze
shouze commented Nov 21, 2013

πŸ‘ it's a death or life question

@jeremyFreeAgent

πŸ‘

@fzaninotto

πŸ‘

@faceleg
Member
faceleg commented Nov 28, 2013

This would be like cherries after a lovely sunset.

@kossnocorp

If we want to have it in the bower we need to decide how it should behave because here is a problem: bower allows to specify package as direct url and there is no way to lock version of package installed this way.

Possible solutions:

  1. Force user to add packages installed via direct url to VCS.
  2. Create separated directory with such packages and urge developer to commit them.
  3. Show warning when user installing package via direct url and explain why it's dangerous.
  4. Create community supported dump where we can keep copy of such packages.

Personally I'm prefer 2) but we can start with 3).

What do you think guys?

I'm really interested in this feature.

/cc @satazor, @maccman, @fat.

@greg0ire
  1. would be better IMO, because 2) urges developer to do something that is considered bad practice. Also there is another solution:

  2. when installing create a checksum of the downloaded package, and warn when installing on another machine from the bower.lock if the newly downloaded package does not match the checksum.

Note that 5) is not incompatible with 3)

@kossnocorp

@greg0ire this is bad because it can happen on production server and updated file can contain errors or changed API.

@greg0ire

@kossnocorp how about doing 3) and 5) then ? Warn in both cases ? Also, in 3) does the warning stop the installation process or not (I assumed it does not) ?

@kof
kof commented Nov 28, 2013

checksum should be used in any case, with or without version ... so version actually doesn't matter ... just in case there is no version and somebody tries to install after checksum has changed without to update .lock file ... then it should be inpossible to install and warning should come in place ...

@kossnocorp

@greg0ire 3) + 5) it good for start but this is worst way to solve this problem but it's easy to implement.

@szimek
szimek commented Nov 28, 2013

I might be missing something important here, but haven't libraries like Bundler already solved these problems?

I can specify something like

gem 'nokogiri', :git => 'git://github.com/tenderlove/nokogiri.git', :branch => '1.4' 

and it works using sha as its "version".

@greg0ire

@szimek : I think that direct url means : http://some.server.com/jquery-latest.js, but maybe I'm wrong.

@kossnocorp

@szimek bower allows to install packages via direct link (and it's not necessary link to raw on github). For example: https://jqueryrotate.googlecode.com/files/jQueryRotate.js.

@szimek
szimek commented Nov 28, 2013

@greg0ire @kossnocorp Thanks. Maybe it's going to be simpler to remove such feature :) Yeah, it makes sense for JS libraries.

@kossnocorp

@szimek actually this is good point, let's call it 6). Less is more and removing this feature may be good motivation for library authors. Btw it's possible only with changing major version of bower.

@rogeriopradoj

Composer PHP allows this kind of just one file url install via repository:

Example of a composer.json with a one file url (works for zip files as well, that will be automatically unzipped):

{
    "repositories": [
        {
            "type": "package",
            "package": {
                "name": "bootswatch/united",
                "version": "2.1.1",
                "dist": {
                    "url": "http://bootswatch.com/united/bootstrap.min.css",
                    "type": "file"
                }
            }
        }
    ],

    "require": {
        "bootswatch/united": "2.1.1"
    }
}

And composer.lock stores some hashing, is it correct @Seldaek?

My vote is for something like this in bower too.

@greg0ire

@rogeriopradoj : do you know how Composer behaves when the content at the given url changes (and the hash is not matched) ?

@rogeriopradoj

@greg0ire, I'm not sure, sorry... That's why I called @Seldaek, one of the core developers of Composer.

@johanneswuerbach

Wouldn't using the HTTP_ETag as version work in most cases? NPM also allows to specify urls, but they recommend to use git urls.

https://npmjs.org/doc/cli/npm-shrinkwrap.html

Shrinkwrap files only lock down package versions, not actual package contents. While discouraged, a package author can republish an existing version of a package, causing shrinkwrapped packages using that version to pick up different code than they were before. If you want to avoid any risk that a byzantine author replaces a package you're using with code that breaks your application, you could modify the shrinkwrap file to use git URL references rather than version numbers so that npm always fetches all packages from git.

If you wish to lock down the specific bytes included in a package, for example to have 100% confidence in being able to reproduce a deployment or build, then you ought to check your dependencies into source control, or pursue some other mechanism that can verify contents rather than versions.

@evillemez

Yeah, I'm kinda with option 2 as a fallback. In the case of a git url, you can lock against a specific commit, so that should stay relatively safe.

In the case of a package w/ raw bytes, I don't see it as bad to store those in a separate directory, maybe bower_components_static/, committed into your repository, with checksums stored in the lock. Yes, it's bad practice to save deps in your repository - but it's worse practice to deploy with deps you can't trust. It's a decent compromise - and bower could at least throw warnings when the checksums change.

@kossnocorp

@johanneswuerbach it's impossible to get needed version of package by etag when url content was changed so it's not suitable for this case.

@evillemez πŸ‘

@kossnocorp

So here is draft plan:

  1. bower_components goes to .gitignore,
  2. when new package added to bower.json or bower.lock.json doesn't exist then bower install will create or update bower.lock.json,
  3. in other cases bower will read bower.lock.json and install specified versions for all packages (sha for git end-point, precise version for bower package),
  4. if package is installed via direct url then etag will be specified as version and copy of downloaded package will be saved to bower_components/freeze.

.gitignore:

bower_components/**/*
!bower_components/freeze

What do you think, guys?

@rogeriopradoj

@kossnocorp sounds good

@tomByrer
tomByrer commented Dec 2, 2013

bower allows to specify package as direct url and there is no way to lock version of package installed this way.

There are many direct URLs that specify versions.
However, even if the version numbers are the same, there is no guarantee the code is exactly the same (eg a CDN uploader may have used a different minifier that causes bugs, repo manager changed files but forgot to change version, etc).
So 2 (host dependents) is the only dependable solution. (Kinda like how installing new software installs many DLLs you are likely to already have).

What is the point of NOT ignoring bower_components/freeze please?

@evillemez

@tomByrer I think that's the location @kossnocorp is suggesting for implementing option 2, so those dependencies would be committed into your project repo.

@kossnocorp

Any chance to get approval from core maintainers of bower?

@sindresorhus
Member

@kossnocorp i'm not sure how it should work to be honest, but i would look at how npm/pip/ruby implements it before starting something yourself. There's a reason it has taken npm so long to implement. It's not as easy as you would think.

As I've commented earlier, it has to have checksum checking to guarantee a package haven't changed.

@kossnocorp

@sindresorhus sure thing, I'm understand that this is not trivial feature, but it's really needed to push bower to next level. We have to start with something that at least works what can be adjusted and polished later.

As I've commented earlier, it has to have checksum checking to guarantee a package haven't changed.

Btw I can start with this feature, because currently bower list shows what package is installed even if only .bower.json exists in package dir. So I can add it to list command. What do you think?

@pavarnos

+1. This feature would be very awesome. thanks! (the attraction is avoiding having dependancy source, tests, build artifacts inside my own repo)

@afeld
afeld commented Dec 12, 2013

I've been following this thread for a long time because it's a feature I had wanted, but I'm now starting to feel πŸ‘Ž about it. It sounds like it would be a fair amount of work, and doesn't really help in the case where a dependency gets yanked. If you want to ensure that a project has identical copies of all packages all the way down, just check them into the repository. FWIW this is what we do at GitHub for gems, Bower packages, NPM modules, etc.

@timcharper

I've been following this thread for a long time because it's a feature I had wanted, but I'm now starting to feel about it. It sounds like it would be a fair amount of work, and doesn’t really help in the case where a dependency gets yanked.

True, but dependency-yanked = build failure (more desirable over build being wrong). Dependencies don’t get yanked that frequently.
If you want to ensure that a project has identical copies of all packages all the way down, just check them into the repository.

I don’t like this option as it leads to repo-bloat, quite quickly. I’d really prefer to not check my dependencies in to my source control. A basic angular project with karma, foundation, angularjs has about 27MB of dependencies.

@sonata82

If you want to ensure that a project has identical copies of all packages all the way down, just check them into the repository.

I don’t like this option as it leads to repo-bloat, quite quickly. I’d really prefer to not check my dependencies in to my source control. A basic angular project with karma, foundation, angularjs has about 27MB of dependencies.

I would like to second this: checking in dependencies is not a good idea, especially for projects that are bound to be dependencies for other projects.

@evillemez

@sonata82 @timcharper I think the initial goal @kossnocorp was aiming for was to store deps in the repo only in the case where there is not a corresponding source repository. Angular, Karma and Foundation have git repos - you would be specifying their git urls, and bower would lock against a specific commit for that repository. You wouldn't store those deps in your repo.

But, in the case of some random jQuery plugin you discovered that's being hosted on another server with no corresponding repo - for that dependency you would store it in your repo. But bower would at least be able to compare against the original url and warn you when/if the contents have changed. Frankly, that's the only safe way to do it. I understand that storing dependencies in your repository sucks, it really does. But have you ever deployed an app that ended up with dependencies that weren't what you tested against? That's a lot worse.

Edit: Also, whether or not you actually store those dependencies in your repo is up to you. Bower could put in the lock a hash of the contents for the url you want. That way, it can reinstall, but if the contents had somehow changed you would at least be notified. Generally speaking, libraries would not want to store their dependencies. End user applications would definitely want to store them, however.

@evillemez

One thing that might be useful is to look at how composer allows specifying alternate repositories. Rather than simply specifying a dependency that points to a url - you define a repository that describes the package you want. So in your deps list, you still specify the dependency/version you want. This lets you take care of packages that aren't registered at packagist.org (composer's default repository). It also allows you to seamlessly host your own repository of packages.

This doesn't solve the issue of what to do if a dependency changes from under you. But, most of the time people keep version numbers in their built files, so they'd be different urls.

http://getcomposer.org/doc/05-repositories.md#repositories

@tomByrer

I hope people keep in mind this scenario; what to do if the dependency's repo is taken down. A few authors have removed their projects before.

@rogeriopradoj

Interesting hearing what @afeld is using at @github complies with what @sergiolopes from @caelum (and other guys too) said in Brazilian iMasters event, 7Masters Javascript.

When I questioned @lucasmazza (the speaker of Bower talk) @sergiolopes said that the only way (which is the way he is using) to ensure your dependencies are the same is include all the dependencies inside your project repository. The discussion take place for a while and because that I changed my mind a little (nevertheless I still prefer PHP @composer way πŸ˜„ ).

Again, thanks @afeld for sharing how GitHub is doing it!

@Shumkov
Shumkov commented Dec 18, 2013

+1

@wolfflow

+1

@chrismendis

I just ran in to a problem with a library changing their entire directory structure in a point release. So, major πŸ‘ for this feature, for me. :-)

@bbhoss
bbhoss commented Jan 3, 2014

It was really surprising to me that bower didn't do this from the first release, coming from Ruby (Bundler) land. Committing actual libraries is very foreign to people that traditionally do "server-side" development. Gems can still get yanked in Ruby, it's your responsibility to mirror them if that's important to you.

Hope this gets done soon!

@jenkoian
jenkoian commented Jan 3, 2014

Yeah @bbhoss summed it up perfectly. I think this is a must have feature.

@mohdhallal

+1

@AyhamAlzoubi

+1

@geshan
geshan commented Jan 14, 2014

+1

@puffin
puffin commented Jan 14, 2014

+1

@lolmaus
Contributor
lolmaus commented Jan 14, 2014

I think Bower maintainers already know this feature is much wanted.

There are over 700 people receiving a separate email for every +1, so... enough. :P

@sandermarechal

One thing that might be useful is to look at how composer allows specifying alternate repositories.

I think this is definitely the best way. Not only does this allow you to host your own versions of yanked dependencies. It also allows you to override a dependency with a patched version. We use this a lot. When we come across a bug we fork the library and patch the bug. Then we add a repository override and version alias to the composer.json so our patched version is used. When our bugfix has been accepted upstream by the package maintainers we revert our composer.json changes.

This means you don't have to wait for your bugfixes to be accepted upstream. This is a huge benefit of the Composer model. Please copy this for bower!

@robwierzbowski

I think you can already do this by using a git endpoint.

On Thursday, January 23, 2014, Sander Marechal notifications@github.com
wrote:

One thing that might be useful is to look at how composer allows
specifying alternate repositories.

I think this is definitely the best way. Not only does this allow you to
host your own versions of yanked dependencies. It also allows you to
override a dependency with a patched version. We use this a lot. When we
come across a bug we fork the library and patch the bug. Then we add a
repository override and version alias to the composer.json so our patched
version is used. When our bugfix has been accepted upstream by the package
maintainers we revert our composer.json changes.

This means you don't have to wait for your bugfixes to be accepted
upstream. This is a huge benefit of the Composer model. Please copy this
for bower!

β€”
Reply to this email directly or view it on GitHubhttps://github.com/bower/bower/issues/505#issuecomment-33104126
.

Rob Wierzbowski
@robwierzbowski http://twitter.com/#!/robwierzbowski
http://github.com/robwierzbowski
http://robwierzbowski.com

@sandermarechal

@robwierzbowski Unless I misread, it seems that the Composer override is more powerful. For example, if you have a package with a sub-dependency (e.g your application requires your/package which requires jquery/jquery 2.0.3) you can fork jquery/jquery to your own github account (e.g. your/jquery) and add it as a custom repository. It will install your custom version even though your/package says it requires jquery/jquery instead of your/jquery. And with version aliases you can even patch the master branch of your/jquery and say "use master as 2.0.3".

@paulozoom

+1

@eggers
eggers commented Jan 31, 2014

+1

@jimthedev

+1

@apfelbox

πŸ‘

@robwierzbowski

Friends. Please. Go easy on the +1s.

@natorojr
natorojr commented Mar 3, 2014

I use Composer for PHP development; it has profoundly simplified, enhanced, and stabilized our development and deployment processes. The lock file has proven itself useful more often than I would have expected, so I can't wait to see the equivalent feature rolled into bower.

Is there an ETA / release schedule for this?

@Glideh
Glideh commented Mar 11, 2014

I would also mention pip that allows to track versions with pip freeze & pip install -r as it has not been mentioned.

@sandermarechal

Another call for why this is badly needed: The exact same bower.json file gave me two completely different versions of jquery 2.1.0 on two different computers. My bower.json lists:

{
    "name": "my/project",
    "private": true,
    "dependencies": {
        "jquery": "2.1.0",
    }
}

One one computer this gave me https://github.com/component/jquery. On another it gave me https://github.com/jquery/jquery. A lockfile would prevent such issues.

@lolmaus
Contributor
lolmaus commented Mar 11, 2014

Can someone please explain how the concept of shrinkwrap is different from Bundler's approach?

Bundler uses two files to track version numbers.

In Gemfile you define desired dependencies. Optionally, you can mention their versions. Versions can be specified in ranges, e. g. "any version above 1.2". The most popular definition type is "any version in the 1.2.x branch above 1.2.5".

When installing dependencies, Bundler will look whether it is already storing matching versions of these gems. If a gem is not found locally, it will download the most recent version in the given range.

After that, Bundler will log exact version numbers of gems in Gemfile.lock. Gemfile.lock should be checked into version control. On subsequent installs, Bundler will use specific version numbers from Gemfile.log, guaranteeing a uniform environment for all developers.

You can overwrite Gemfile.lock with more recent dependency versions by telling Bundler to run the update dependencies command.

I find this approach to be very convenient. I wonder whether the shrinkwrap approach is similar or different.

@evillemez

It's the same thing - a lock file. For some reason npm calls its command for this "shrinkwrap", so we're just using the terminology from the node world. I think everyone else says "lock file".

@lolmaus
Contributor
lolmaus commented Mar 13, 2014

Hey @evillemez, my concern is not about the lockfile, i. e. the shinkwrap vs Gemfile.lock.

My concern is about the manifest file, i. e. bower.json vs Gemfile.

The problem is, that in the dependencies section of bower.json you can only define either a specific version or a link to a repo.

  • If you define a specific version, it gets locked even without the lockfile. You can't tell Bower to update the dependency because it will always download the same version.

  • If you define a link to a repo, it gets locked via the lockfile, and you supposedly could tell Bower to update the dependency (overwrite the lockfile with dependency's recent version).

    The problem here is that updating dependencies is not safe, because any update can result in backwards incompatibility (unless repo owner uses a separate branch for every middle version, e. g. 1.2.x, 1.3.x, which is a good yet rare practice).

Thus, introducing a lockfile is not enough. Bower should also improve how dependency versions can be defined. We should be able to define version ranges similar to Gemfile, e. g. ~>1.3.12 in Gemfile means "install the latest available version that starts with 1.3 and is not older than 1.3.12".

@artursvonda

@lolmaus You can already do that.

@natorojr

@lolmaus Can't you already do something "<=1.3.12" with bower? I'm fairly certain I'm using that feature in several projects.

@lolmaus
Contributor
lolmaus commented Mar 13, 2014

@artursvonda, can you please link to where this feature is documented?

All i could find was this piece of documentation:

  "dependencies": {
    "<name>": "<version>",
    "<name>": "<folder>",
    "<name>": "<package>"
  },

Not very informative :(

@donaldpipowitch

@lolmaus http://bower.io/#installing-packages-and-dependencies It's documented on the project page :)

You can specify a semver compatible version to fetch a specific release, and lock the package to that version. You can also use ranges to specify a range of versions.

@natorojr

@lolmaus

"These types of might have versions available. You can specify a semver compatible version to fetch a specific release, and lock the package to that version. You can also use ranges to specify a range of versions."

@wibblymat wibblymat changed the title from Implement 'shrinkwrap' feature [$45] to Implement 'shrinkwrap' feature [$80] Aug 5, 2014
@wibblymat wibblymat added the bounty label Aug 5, 2014
@munro
munro commented Aug 5, 2014

Just FYI, people have been talking about checksums in relation to NPM. NPM doesn't have checksums because they removed the ability to force push packages [1]. So as long as you trust NPM's registry, and SSL, this is a secure way of handling identical builds (except when a package is removed).

My assumption is that bower installs directly from git URLs, so we would need to implement checksumsβ€”but if we're only using git, commit hashes are exactly that. If we used commit hashes directly we would be able to verify if the package is correct quickly, as well. Instead of downloading based on a version tag, then computing the hashβ€”we could download based on the hash, and if doesn't exist in the git repo anymore, we would immediately know if it was incorrect.

[1] npm/npm-registry-couchapp#148

@schmurfy
schmurfy commented Aug 6, 2014

I already mentioned somewhere above that git revision are checksum for the commit and cannot be changed and would serve the purpose well, although it would only works if the package was loaded from a git repository.

@frank-weindel

πŸ‘

@thejase
thejase commented Nov 1, 2014

If only one could unsubscribe from emoji-only posts.

@deepsweet

no way :trollface:.

@rstacruz
rstacruz commented Nov 9, 2014

@sheerun, I really would love to see this feature and am open to helping out. how would you like to this to happen?

also:

Lock file contains concatenated .bower.json files generated by bower (some subset of their keys)

This looks like it's already available as bower list --json --offline --relative > bower.lock. is this what you had in mind?

@sheerun
Contributor
sheerun commented Nov 9, 2014

@rstacruz Sure. I created #1592 because it's little crowded here.

You can send any code against this branch (feature/shrinkwrap). Don't worry if it doesn't fully work.

After shrinkwrap feature stabilizes it will be squashed, cleaned up, and merged to master.

@sheerun
Contributor
sheerun commented Nov 9, 2014

I'm locking this thread. If anyone wants to contribute, please send code to / discuss in #1592.

@sheerun sheerun locked and limited conversation to collaborators Nov 9, 2014
@wibblymat wibblymat changed the title from Implement 'shrinkwrap' feature [$80] to Implement 'shrinkwrap' feature Jan 9, 2015
@wibblymat wibblymat removed the bounty label Feb 21, 2015
@theefer theefer referenced this issue in guardian/recommendations Mar 31, 2015
Merged

Add client-side recommendations for review #7

@sheerun sheerun unlocked this conversation Apr 8, 2015
@davidrapson davidrapson added a commit to guardian/membership-frontend that referenced this issue Apr 8, 2015
@davidrapson davidrapson Commit bower dependencies
This PR commits bower dependencies, while this is against typical
recommendations it is currently the preferred option for Bower:

- Bower does not have a lock file or shrinkwrap which means running
  bower install on deploy or build can cause different versions to be
pulled down.
- There is an open issue for a Bower lock file, but this has been open
  for ~2 years bower/bower#505
- Committing dependencies is the Bower teams recommendation

http://stackoverflow.com/a/17208939
dee38b4
@davidrapson davidrapson referenced this issue in guardian/membership-frontend Apr 8, 2015
Closed

Commit bower dependencies? #455

@elboletaire

πŸ‘

@gtrias
gtrias commented Jun 19, 2015

+1

@greg0ire

Stop "voting" for this feature please. It's already planned, no need to notify everyone on earth.

@robclancy

Haha how is this not implemented. Should be a feature planned from the very start for first release.

@jifeon jifeon referenced this issue in apsavin/bnsf-project-stub Aug 8, 2015
@jifeon jifeon Lock bower dependencies
Assumes that both bnsf-project-stub and bnsf itself uses bem-core, we should lock bem-components dependency to avoid situation when bower cannot find suitable dependency
e464288
@NgYueHong

@greg0ire Happy to see it's already planned. But this has been long talk for over 2 years. When will it be available?

@greg0ire

When the little green bar on the right is full I guess. I don't know I'm not part of the project.

@cwoolum cwoolum referenced this issue in Microsoft/vsts-tasks Sep 1, 2015
Closed

Provide bower support #357

@sheerun
Contributor
sheerun commented Sep 7, 2015

FYI this feature can be theoretically implemented using new Pluggable Resolvers feature of Bower.

Review for this PR is still pending, as I have very limited time resources. I hope to convince someone to sponsor work on this feature as it's quite important for Bower's community. Any help is appreciated.

@Turkish
Turkish commented Sep 8, 2015

so sad this feature is not yet implemented ...

@munro
munro commented Sep 8, 2015

Bower has been superseded by npm, just install your dependencies there and use npm shrinkwrap, and use webpack to bundle them.

@greg0ire
greg0ire commented Sep 8, 2015

Bower has been superseded by npm

Source ?

@Turkish
Turkish commented Sep 8, 2015

@munro yes i think you're right, it's useless to have both bower and npm in the same project. Since bower hasn't 'shrinkwrap' it get's superseded by npm naturally thanks for the advice :)

@robclancy

Both bower and npm are bad lol. Bower is understandable as it is just
simple assets and you can commit that crap to a repo for better than
shrinkwrap functionality. But NPM is slow, doesn't cache shit, has very
unclear error messages and yet another package manager without a good
naming scheme so when someone takes a certain name and never updates it can
easily be the proper name for something lost forever.

Our CI is incredibly slow because of npm. All our other package managers
actually cache things and run faster on subsequent builds. And I know there
are 3rd package things to help, they all suck. The only one I haven't done
yet which I am sure will work is the one where it proxys the server (can't
remember the name). Yes, I have to proxy the npm servers just to get basic
functionality all other package managers have so it isn't slow as balls.

Oh and shrinkwrap isn't even good enough. Should lock to exact commits to
get the exact code but it does versions which can easily be broken by
library developers and has twice in the last month for us. Would think the
"omg node is the best thing ever" programmers of the world who would die to
promote it and npm would have a bettter package manager.

On Tue, Sep 8, 2015 at 1:51 PM Turkish notifications@github.com wrote:

@munro https://github.com/munro yes i think you're right, it's useless
to have both bower and npm in the same project. Since bower hasn't
'shrinkwrap' it get's superseded by npm naturally thanks for the advice :)

β€”
Reply to this email directly or view it on GitHub
#505 (comment).

@braincelli

https://vid.me/Bratva/albums/sharetheinfo

2015-09-07 23:56 GMT+03:00 Adam Stankiewicz notifications@github.com:

FYI this feature can be theoretically implemented using new Pluggable
Resolvers http://bower.io/docs/pluggable-resolvers/ feature of Bower.

Review for this PR is still pending, as I have very limited time
resources. I hope to convince someone to sponsor work on this feature as
it's quite important for Bower's community. Any help is appreciated.

β€”
Reply to this email directly or view it on GitHub
#505 (comment).

@avindra
avindra commented Sep 8, 2015

​I've been thinking about switching full to npm as well. Just nice to have the front end vs. backend package separation. Does npm have any way to specify this in the package.json file?

@lolmaus
Contributor
lolmaus commented Sep 8, 2015

Bower has been superseded by npm

Source ?

With Browserify and Babel, I do simply:

import _ from 'npm:lodash';

_.range(2); // => [0, 1]

This works in browsers.

And if a package is not published to npmjs.org, it's possible to install directly from GitHub, optionally asking for a specific tag/release, branch or commit:

npm install --save-dev user-name/repo-name@v1.0.2

Tell my why I would still need Bower.

@greg0ire
greg0ire commented Sep 8, 2015

How is this a source ? What I'm waiting for is a link to a blog post from the author of Bower, saying Bower has been superseded by npm. Good luck.

@munro
munro commented Sep 8, 2015

robclancy But NPM is slow

Long off topic rant 😹, but you should post your issue on Stackoverflow, if there truly is an issue with NPM caching it will be a welcomed improvement. If all else fails use Docker, or NixOS.

robclancy Oh and shrinkwrap isn't even good enough

Shrinkwrap is good enough because developers cannot deploy the same version twice. [1]

[1] npm/npm-registry-couchapp#148

avindra Does npm have any way to specify this in the package.json file?

NPM at this point is just distribution for assets, webpack and browserify are the mechanisms for compiling on the front end. There's no standard for "front end" packages, since that's really vague. There are standards for JavaScript dependencies, so that will just work. Also webpack is very customizable so you can do all sort of crazy things like flat dependencies, or just flat for lodash & jQuery, or swap them with a CDN, or whatever you can think of! And then just wrap it in a webpack plugin.

greg0ire How is this a source ?

All the source is here: https://www.npmjs.com/ Give it a try!

@greg0ire
greg0ire commented Sep 8, 2015

All the source is here: https://www.npmjs.com/ Give it a try!

What the troll ?

What I'm waiting for is a link to a blog post from the author of Bower, saying Bower has been superseded by npm. Good luck.

@lolmaus
Contributor
lolmaus commented Sep 8, 2015

@greg0ire By any chance, do you move around town via a carriage? You know, because the carriage owner hasn't publish a confirmation that mules have been superseded by automobiles.

@greg0ire
greg0ire commented Sep 8, 2015

I don't think mules vs automobiles is the same debate as npm vs bower or emacs vs vim. My point is, don't come here stating your personal opinion as a fact. I don't see many carriage users in town, buty I know many bower users.

@munro
munro commented Sep 8, 2015

lolmaus

If only they did, good point that these sort of things happen organically. I added a small code bounty to this issue a year agoβ€”since then I've just found myself using npm more. Even in #Node.js they're actively recommending NPM and deterring Bower use, though an article would be nice to make it less confusing for new comers. Relevancy isn't decided by the project [1], but through hard work one can try to stay relevant [2].

[1] https://github.com/bower/bower/graphs/contributors?from=2014-10-04&to=2015-09-05&type=c
[2] https://github.com/npm/npm/graphs/contributors?from=2014-10-04&to=2015-09-05&type=c

@belka-ew

+1 for lockfile

@saschanos

+1 for lockfile

@PedroSena

+1 for lockfile

@albertinad

+1 for lockfile

@syzer
syzer commented Sep 29, 2015

+1 for resolution key in bower.json

@lsmith77

since it seems like this feature is taking a long time in the making, maybe an intermediate step would be a script living entirely outside of bower? just take a bower.json file, make a backup copy, traverse what is installed and update the bower.json file with pinned versions of what is currently installed.

@syzer
syzer commented Sep 29, 2015

@lsmith77 its possible to add custom resolver #1649 and http://bower.io/docs/pluggable-resolvers/

@lsmith77

see #1748

@dbellettini

πŸ‘ for lockfile

@marco-faustinelli

πŸ‘ for lock file

@twogee
twogee commented Oct 13, 2015

is #1748 ready for merge?

@avindra
avindra commented Oct 13, 2015

Switched to an npm / webpack setup and haven't looked back. Still working on migrating off bower, because bower is lacking good support for code modularity and dependency locking. I will say that I still like the bower logo though!

@jpike88
jpike88 commented Oct 15, 2015

+1 for this, come on guys I can't take bower seriously until it has a way of locking down dependency versions, how can you develop with peace of mind without this???

@Chekote
Chekote commented Oct 16, 2015

+1 for lockfile

@MelnikVasya

+1 for lockfile )

@avdd
avdd commented Nov 3, 2015

Why have a separate lock file when the dependencies can be pinned in bower.json?
But currently it's a pain updating pinned dependencies because bower update becomes a no-op, and bower install has only one option: --force-latest, otherwise you have to manually check and specify the target version.

@derrabus
derrabus commented Nov 3, 2015

@avdd: Like mentioned a couple of times before, you can only pin direct dependencies in bower.json. Indirect/transitive dependencies will always be pulled at their latest compatible version. So pinning in composer.json doesn't cut it.

@avdd
avdd commented Nov 3, 2015

@derrabus I'm not sure what you mean. Because bower installations are flat and not nested like node, indirect deps become direct deps. If my app uses Foo which requires Bar, I pin versions of both Foo and Bar in my bower.json, which I believe is the point of the feature under discussion?

@derrabus
derrabus commented Nov 3, 2015

@avdd Of course, you can pin the complete dependency graph in your bower.json. But the whole point of a dependency management tool is that you only define your direct dependencies and that it resolves all indirect dependencies for you. What you're describing is a valid workaround, but it's not the solution to the problem.

@greg0ire
greg0ire commented Nov 3, 2015

@derrabus @avdd says there is no dependency graph at all : "bower installations are flat". @avdd : what makes you say that ?

@avdd
avdd commented Nov 3, 2015

@greg0ire bower packages are all installed into one bower_components directory at one level, i.e. flat.

@derrabus it's not a workaround at all. Indirect dependencies are still very much dependencies that should be pinned for repeatable deployments. The dependency graph as such is only a temporary structure used for installing or updating packages. Why bother keeping it?

(I assume we can only be talking about top-level projects and not redistributable packages, which clearly have no use for a shrinkwrap feature.)

@greg0ire
greg0ire commented Nov 4, 2015

it's not a workaround at all. Indirect dependencies are still very much dependencies that should be pinned for repeatable deployments.

That's the point of the lock file. On the other hand, when you want to upgrade your top-level dependencies, you don't want to have to find yourself the right version of the indirect deps. That's bower's job, and that's why they should not be kept. Plus if the dependencies have many dependencies that also have dependencies, things could get cumbersome really quickly.

@davidchin

+1 for lockfile

@benallfree

Curious to know if anyone is working on this or what the status is?

@gitawego
gitawego commented Dec 8, 2015

+1 for lockfile

@VaclavDedik

+1

@morrislaptop

+1

@sheerun
Contributor
sheerun commented Dec 9, 2015

Please don't +1 this. We need someone skilled to declare to work at least 1-2 weeks on this feature and then maintain it for some time, so I can be sure it is solid and stable enough to release. We need to:

  1. Review my current requirements about this feature. Do they cover all the edge cases? What are the ramifications? How can we can achieve backward-compatibility and (even more importantly) forward compatibility of this feature (see: current state of shirnkwrap feature of npm...)
  2. Review all the initial code of #1748 and make sure everything is properly tested.
  3. Implement any changes we agreed on in 1.
  4. Test in alpha version with out @bower/contributors and deploy first fixes
  5. Release it in beta, wait for a feedback
  6. Maintain this feature for some time

If someone feels competent enough and is interested in this "gig" (JFrog is eager to pay for time spent on this feature), please post a motivational letter and your availability on my e-mail: sheerun@sher.pl

(e-mail me as well if you just want to volunteer without any liabilities)

@sheerun sheerun locked and limited conversation to collaborators Dec 9, 2015
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.