postinstall script (and others) #249

Closed
sindresorhus opened this Issue Feb 11, 2013 · 47 comments

Projects

None yet
@sindresorhus
Member

Like NPM.

From the roadmap.

@stevenbenner stevenbenner referenced this issue in stevenbenner/jquery-powertip Feb 28, 2013
Closed

bower: keep built version in vcs? #54

@necolas
Member
necolas commented Mar 9, 2013

npm generally considers install/postinstall scripts an anti-pattern. npm expects you to rely on a prepublish script to compile any assets you want to publish to the registry, and allows you to publish only the build assets. Bower has a different model where you don't publish any assets to the registry itself (you just register some details one time), so unless you want to start checking build assets into git (no), you can't rely on a prepublish script.

That leaves postinstall, which would have to be run any time you checkout a different commit-ish in the clone. The problem with postinstall is that you cannot know what dependencies are expected, and if the local (or global) environment doesn't have those dependencies, you can't run the script successfully. You can only be sure that node and npm are available. So postinstall scripts would have to be node-based. But even then, take Grunt as an example (which would require grunt-cli to be in the dev deps too, or globally installed on the target machine): you could have a node-based Sass task that will fail because Sass and/or Ruby isn't installed.

So, I'm not sure how to avoid these problems.

@stevenbenner

I agree with the points @necolas brought up. Those issues have been bothering me as well. I have been running into a number of problems trying to keep my projects compatible with Bower.

  1. Keeping built versions of projects in a git repo is just plain wrong. You wouldn't keep compiled binaries in any SCM scheme, so why would you keep built (let alone minified) versions of scripts, css, or html in the repo? Those files are never meant to be worked on. They are the result of the stuff you do actually work on.
  2. Bower actually requires that users hard-code the paths to their dependencies. So if a repo owner sees fit to move or rename an important file within their source code repository then they are essentially breaking the product for everyone who uses Bower to maintain their dependencies.
  3. Post install scripts are very sensitive to the user's environment. If you are planning to build your project via a Bower postinstall hook then you had better carefully consider every dependency required to do so. I can see this leading to some very long and complex postinstall scripts. And it's pretty safe to assume that those will be the first thing to break if the project maintainer goes awol.

Adding postinstall scripts will go a long way towards making this easier, and this is a feature that I support greatly, but to be honest I think all of these problems show why NPM works the way it does. You want to keep a stable snapshot of every version compiled and ready to go, in a place separate from the development repository.

In fact Bower is already doing this, but only for exceptionally popular projects (see: https://github.com/components). And unfortunately those snapshots are being maintained by hand.

I would suggest that Bower should do one of two things to fix these issues:

  1. Implement a full-blown asset registry that lets project maintainers publish built versions to the registry and keeps the published content in the registry itself (like NPM), or the far easier option…
  2. Let project maintainers register a URL to a zip/tar/rar/whatever of their built version to the registry.

Sorry for hijacking. I do want to see postinstall scripts added, but I am concerned that they are being added solely as a workaround to a more fundamental problem.

@necolas
Member
necolas commented Mar 10, 2013

Some random thoughts:

-1. Bower could switch to bower publish for registration, which will get all the relevant data from the directory's bower.json. Then it doesn't really matter how that publish happens beneath the surface. At the moment, it's error prone and redundant to have to repeat the package name and endpoint in the CLI when registering.

-2. You could include any path as a location in the bower.json. If it's a Git endpoint, then Bower will clone it and checkout the relevant version. If it's a not a Git endpoint, then you can mark where the version exists in the path (a bit like the shorthand_resolver template -- requires that we keep version). This could allow you to publish the location of a distribution package to the registry, in a departure from a pure-Git-endpoints approach. But you'd have to run bower publish each time a new release was available. Might be tricky to know for sure that the new release is actually available at that URL though.

{
  "name": "normalize-css",
  "version": "2.1.0",
  "main": "normalize.css",
  "location": "http://necolas.github.com/normalize.css/{{{version}}}/normalize.css"
}

This is how it might work if you were looking to depend on a published distribution that is not registered with Bower:

{
  "dependencies": {
    "jquery": "http://code.jquery.com/jquery-{{{version}}}.min.js#1.9.1",
  }
}

-3. Work out how to make it easy for people to use a "releases" branch, where they tag builds that Bower can then checkout. For example, you could tag your source, run the build, checkout the release branch, and then tag that build in a way that Bower could know about. This feels like it's probably not particularly intuitive, easy, or idiomatic.

-4. Switch to an npm model where you publish the assets themselves to the registry. Involves more work for package authors (have to publish each time), but avoids postinstalls and means we could do package integrity checks.

@sindresorhus
Member

I should have included an example. I totally agree that postinstall in modules is a bad thing, and it's been proven so in npm. My use-case is actually quite different. I want it for my own project component.json file to be able to trigger a script after installing a component with Bower. In my case I created grunt-bower-requirejs which is a grunt task that adds installed components to the users RJS config when triggered. Currently the user have to manually trigger this. I would like it triggered when a component is installed (postinstall). I could see this useful for lots of other tools that interfaces with Bower to do something useful with the components, since Bower is deliberately unopinionated.

@sindresorhus
Member

@necolas

  1. I absolutely agree. This must be fixed.
  2. Very OT. Can you open an new ticket about that?

I've always favored the npm approach of having a hosted registry you publish to, but it would require more resources and people agreeing with me.

@addyosmani
Member

I would just like to add a +1 to everything Sindre said about the use-case for a post-install module. It would be ideal for an improved RJS story but could also be used for simpler tasks like auto-wiring script tags or in Yeoman's case usemin blocks for dependencies that are installed.

@passy
Contributor
passy commented Apr 28, 2013

+1 on @sindresorhus' remarks. An ideal solution for me would be a post-install setting in the local .bowerrc of the application, e.g.

{
  "directory": "app/bower_components",
  "post_install": "app/bower_postinstall"
}

Which would execute the app/bower_postinstall module as a function after a successful bower install with a metadata object describing the installed component. That module could then trigger auto-wiring of usemin blocks or the RJS setup.

@necolas
Member
necolas commented Apr 28, 2013

Yeah, end-user configuration should be in the .bowerrc, so that could work.

@danheberden
Member

👍

@mlatief
mlatief commented May 5, 2013

+1 especially for RJS scenario

@ericf
Contributor
ericf commented May 11, 2013

This idea of the consumer defining the post install script seems interesting to me, but should you be able to define a script per dependency?

This way, for simpler projects, I could define post install scripts "grunt" for dependA, and "make" for dependB directly in my project's bower.json (or .bowerrc?). Having this ability would mean I don't have to create a new file in my project, and if I just update dependA, I don't have to wait for dependB's build to also run.

@joefiorini

👍 Another use case for this is necolas/normalize.css#9. The best solution to a problem with the way Sass includes CSS files is to symlink CSS files installed via bower to the same directory as Sass assets, but with a different extension. I want to write a task to do this for us, but currently I'd have to run it manually after every bower install. It'd be nice to have this run automatically.

@simeonwillbanks

👍 A postinstall hook could trigger an end-user processor which preps package CSS files for the Rails asset pipeline.

  1. Create a SASS file from a CSS file
    1. mv select2.css select2.css.scss
  2. Replace all url declarations with asset helpers
    1. :%s/url('/image-url('select2\//g
    2. image-url("rails.png") becomes url(/assets/rails.png)
@cgross
Contributor
cgross commented Jun 19, 2013

+1 on the option in .bowerrc.

I have a similar use-case @sindresorhus. I have an yeoman angular generator and I'd like to add the script tag automatically and add the angular module dependency as a postinstall step.

The script references can be parsed out of the main property. The angular module name of the package is not as easy. Has there been any thought into where custom/extra properties of a package could be put in bower.json? With the addition of a postinstall capability, I think there's a need for a section of extra properties in the bower.json that would be applicable to postinstall scripts.

@necolas
Member
necolas commented Jun 19, 2013

Sounds like a good amount of overlap with existing build tasks.

@cgross
Contributor
cgross commented Jun 19, 2013

@necolas Not sure I follow. My use-case is that my project A that uses package B needs to know what package B's angular module name is. Today I have to go to package B's github page or dig in the code, find the module name, and then manually update my angular init code in project A. I'd like that module name to be in package B's bower.json so I don't have to go digging for it. Heck even if there's no postinstall step I'd really like that in the bower.json. For any angular component its a required piece of metadata before it can be consumed by any application.

@trek
trek commented Jul 15, 2013

@addyosmani and I had a brief twitter conversation about this with regards to https://github.com/square/es6-module-transpiler and https://github.com/umdjs/umd. I was going to write something up, but @rpflorence did the work for me

As a library maintainer I'm suffering from wanting to make my code useable in whatever module/loader system the consumer wants but also cut the loader-ritual-noise when developing.

So, Ryan's first step

  1. getting library authors to author in es6.

I can start doing today. The downside is we still need to transpile into a consumption format that is either a) checked into the repo in a dist folder or b) published as separate repos.

For a) unless we want to force all committers to transpile first or are willing to have 2x commits (once for code change, once for transpile) this is only useful for release versions. This ruins the usefulness of "my-lib": "latest"

For b) we've run into some issues emberjs/ember.js#2823 (comment). Even if we did manage this repo, it would probably only be updated on version bumps rather than each commit (nullifying the value of "latest"). It also increases the confusion for consumer who need to know that the authoring repo or branch ≠ the published repo or branch.

and Ryan's second step

Step 2: getting package managers to support es6 and transpile for us.

would be handy to have, letting a library consumer determine what module format she prefers. I don't know if an arbitrary post_install script is a good idea (would I now need to list both library dependencies and npm-based build dependencies?) but one based on upcoming standards and existing practices seems prudent.

@jamesarosen

NPM and Bower (and older tools like Rubygems) have the idea of "types" of dependencies. A bower package might have the following dependencies:

  • a runtime dependency on jQuery
  • a dev dependency on Grunt
  • an install-time dependency on Sass

The question remains, though: are the devDependencies and installDependencies NPM dependencies or Bower ones? If the latter, that means that Grunt, Sass, etc. must declare both a package.json and a bower.json (or there are Bower-specific forks of those projects).

@necolas
Member
necolas commented Jul 15, 2013

@trek Publishing build assets directly to the Bower registry is planned.

@trek
trek commented Jul 17, 2013

@necolas how would people use "latest" version or point their dependency at a specific fork while waiting on PR acceptance?

@cgross
Contributor
cgross commented Jul 24, 2013

I created a project in anticipation of a these hooks being added:
https://github.com/cgross/bowinst

It will added script tags and more. Completely configurable and extensible.

Feedback would be appreciated!

@cgross
Contributor
cgross commented Jul 27, 2013

Is the bower team willing to look at PRs for this feature? If so, I'd be happy to work on this.

@satazor
Member
satazor commented Jul 27, 2013

@cgross absolutely, it still has use cases

@mishanga

+1 for the feature

@psyrendust

+1 for this feature

@quarterto quarterto added a commit to quarterto-archive/Semana that referenced this issue Nov 11, 2013
@quarterto quarterto ugh bower 83225cb
@gobengo
gobengo commented Nov 12, 2013

+1

@jiyinyiyong

+1

@joliss joliss added a commit to broccolijs/broccoli that referenced this issue Jan 5, 2014
@joliss joliss Eliminate support for Broccolifile.js
The idea was to preprocess dependencies' source files (like .coffee) on
the fly. But this is fundamentally overkill, as the correct solution
tends to be publishing precompiled files through the package manager.

Bower doesn't support pushing tarballs yet, but I don't think there's
much debate that this is a desirable feature.
bower/bower#249
b248ac4
@khilnani
khilnani commented Feb 3, 2014

+1
Something along the lines of @passy @@sindresorhus but with a script per dep since each one has its own internal structure. eg. "jquery/jquery.min.js" vs "bootstrap/dist/js/bootstrap.min.js"

I think this would be a workaround but easier solution to creating a standard for bower compatible dependencies and having each project conform.

.bowerrc

{
  "directory": "app/bower_components",
  "post_install": {
      "jquery": "bower/postinstall_jquery.js",
      "bootstrap": "bower/postinstall_bootstrap.js"
  }
}
@sindresorhus
Member

@khilnani no, that's not what it's supposed to be used as. Bower components should come prebuilt. If they're not, open up a ticket on the relevant repo.

@joefiorini

@sindresorhus are there still plans to support more delivery/registration mechanisms than just git? Or is that it?

@sindresorhus
Member

@joefiorini #1055 #979

There's also work being done on creating a new registry where you can publish packages to a central server npm style: https://github.com/bower/registry/tree/node_rewrite

@khilnani
khilnani commented Feb 3, 2014

@sindresorhus its not that the projects are not pre-built. They are, its just that the file/dir structure is not consistent across projects. Additionally, the downloaded projects also have files that might not be appropriate in a production environment (eg. examples dir). With the scripts post project install, only the needed files could be moved to a more app/deployment specific production location.

The registry would solve the mixing of source and distribution code, but would it make the project contents consistent?

@sindresorhus
Member

Additionally, the downloaded projects also have files that might not be appropriate in a production environment

Packages should use the ignore property in bower.json to ignore unneeded files.

They are, its just that the file/dir structure is not consistent across projects.

Stop fighting it. You shouldn't have move out any files. Just reference them from the bower_components folder

but would it make the project contents consistent?

?

@ninjabiscuit

What about projects like loash and modernizr that sometimes require
configurable builds?

On Monday, February 3, 2014, Sindre Sorhus notifications@github.com wrote:

Additionally, the downloaded projects also have files that might not be
appropriate in a production environment

Packages should use the ignore property in bower.json to ignore unneeded
files.

They are, its just that the file/dir structure is not consistent across
projects.

Stop fighting it. You shouldn't have move out any files. Just reference
them from the bower_components folder

but would it make the project contents consistent?

?

Reply to this email directly or view it on GitHubhttps://github.com/bower/bower/issues/249#issuecomment-34002523
.

@sindresorhus
Member

@ninjabiscuit that should rather be a npm installed CLI tool or something. but that's not the normal anyways.

@ninjabiscuit

@sindresorhus sure, I agree that it's not the norm but it would be quite useful to be able to fire a configuration script post install. Especially when the bower packages are not checked into source control. Unless there's a better way of doing this that I've missed, in which case I'd genuinely love to hear about it. Even jQuery has a configuration script these days.

@sindresorhus
Member

@ninjabiscuit it just executes a command so you can do whatever you can imagine. You even create a node script that downloads all the cat gifs on the internet and execute it on postinstall. Bower will give the package that was installed as an argument so you're free to do something clever with that.

@khilnani
khilnani commented Feb 3, 2014

@sindresorhus I suppose 'ignore' would help but that would need to be customized per project. Include May need a shorter list of files since it's more specific.

UI JS/CSS projects are inherently more complicated from node modules I think. I agree with @ninjabiscuit re. Customizing projects such as bootstrap. Eg. We use a custom config of bootstrap or any project that offers UI components (JS and CSS).

It's not a matter of working against the application's design. The post install hook simply provides flexibility with integration at the right point in the component/project installation life cycle.

@jayphelps

Maybe I'm missing something (I've read the discussion here and elsewhere) but this seems silly not to have a post install hook. I don't want to distribute builds in my repos and I don't want to have to maintain a separate repo with just the builds. Registering builds sounds good in theory but in practice, what about private repos?

+1 post install massively needed.

Edit: Nevermind https://github.com/bower/bower/blob/master/HOOKS.md Even if it says it isn't intended to build your project, as long as this functionality stays, Imma do it anyway. Also looks like that commit resolves this ticket.

@konsumer
konsumer commented Apr 2, 2014

Hmm, this doesn't seem to work. My .bowerrc looks like this:

{
    "directory": "app/bower_components",
    "scripts": {
        "postinstall": "grunt bower-install"
    }
}
@wibblymat wibblymat removed this from the 1.3.0 milestone Apr 3, 2014
@sheerun
Contributor
sheerun commented Apr 16, 2014

This is utterly wrong idea... Allowing postinstall raises serious security issues. With them anyone is able to run arbitrary code on your computer and on your production machines... That's why it's impossible in tools like git to commit any hooks to repository. Bower is git of web.

This is especially dangerous in case of bower as it doesn't use any checksums, or packaging. A lot of people are depending on branches which can change in any moment (as well as tags btw.).

As @necolas pointed out postinstall is also useless to post-process files as user environment is unknown and unpredictable. Bower is going to have publish command so pre-publish hook will be ok.

If hooks are implemented, they should be immediately reverted and deprecated.

ping @sindresorhus @wibblymat @paulirish @satazor @benschwarz @svnlto

@sheerun
Contributor
sheerun commented Apr 16, 2014

If anyone want to compare with npm:

With npm post-install is more acceptable (still bad idea) because you can't avoid executing javascript files on server. Bower is different story as packages are executed only in web browser. Also npm has checksums, packaged packages, projects like https://nodesecurity.io/.

Moreover bower is used not only by node projects. This "feature" makes any project using bower directly vulnerable (like https://github.com/42dev/bower-rails or https://github.com/d-i/half-pipe or bower CDNs).

@benschwarz
Member

@necolas convinced me last year that this was a bad idea… nothing said here has been able to swing that opinion.

If you wanted to install custom build of modernizr, I'd probably script something with grunt/gulp/make… with, or without bower.

@sheerun
Contributor
sheerun commented Apr 17, 2014

OK, I see postinstall was implemented only as .bowerrc which is not a security issue.

Am I right @sindresorhus ?

@sindresorhus
Member

postinstall is meant for users to be able to run a script around the install command as a way to manage bower components. Example: https://github.com/yeoman/bower-requirejs#using-bower-hooks

@sheerun
Contributor
sheerun commented Jun 8, 2014

Closing, since postinstall and preinstall hooks are already implemented.

@sheerun sheerun closed this Jun 8, 2014
@sheerun
Contributor
sheerun commented Jun 8, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment