New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v4: Move to npm scripts #20332

Closed
wants to merge 2 commits into
base: v4-dev
from

Conversation

Projects
None yet
@mdo
Member

mdo commented Jul 20, 2016

This PR is heavily WIP, but addresses a growing concern I've had with developing Bootstrap—the build process is way to convoluted. Currently we make use of Grunt, npm, Bundler, Jekyll, etc all just to run our docs and compile source assets. More personally, I find the current dev process incredibly fatiguing—as a maintainer, it's hard to be motivated to jump into this codebase right now.

My goal here is to overhaul the build process and make Bootstrap development more approachable to everyone (myself included). Originally I was thinking Gulp would be our answer, but it's still a task runner build on npm rather than npm itself.

So far I've started to pull apart a few bits of our current tooling and replacing them with npm scripts. Right now they all live under tasks/ and are split by dist/ and docs/. I'm probably doing a ton wrong, but maybe you can help with that :D.

Here's the list of tasks we need for developing and distributing Bootstrap:

  • CSS
    • Lint Sass before trying to compile (with stylelint over scss-lint)
    • Compile Sass files with node-sass (no more Ruby options)
    • Compile and minify Sass files
    • Autoprefixer compiled CSS files
  • JavaScript
    • Babel
    • Uglify
    • Qunit tests
    • Concatenate and minify
  • Docs/development
    • Update version
    • Shrinkwrap dependencies
    • Lint HTML files
    • Run Jekyll and watch Sass/JS files (stylelint, compile, autoprefixer)
    • Saucelabs?
    • Package dist files in zipfile for release
    • Deploy Jekyll _site directory to remote to update docs site

I'm probably missing some tasks/scripts we need, but I do believe we need the above. These tasks should be runnable by us as maintainers, as well as casual users who wish to extend Bootstrap's code and docs, so I want them to be clear, concise, and documented.

/cc @twbs/team

@mdo mdo added meta v4 labels Jul 20, 2016

@mdo mdo added this to the v4.0.0-alpha.3 milestone Jul 20, 2016

@ismnoiet

This comment has been minimized.

ismnoiet commented Jul 20, 2016

First time here 😄 and i think this would be a great move @mdo 👍

@cvrebert cvrebert added the build label Jul 20, 2016

@@ -0,0 +1 @@
node-sass --output-style expanded --source-map true --precision 6 docs/assets/scss/docs.scss docs/assets/css/docs.css

This comment has been minimized.

@fat

fat Jul 20, 2016

Member

probably better if you reference the local npm installs of these (in .bin ) so that people don't need global installs 👍

This comment has been minimized.

@XhmikosR

XhmikosR Jul 20, 2016

Member

Well this whole approach will have issues on Windows. But I'll try it later
or tomorrow to verify this.

On Jul 20, 2016 22:44, "fat" notifications@github.com wrote:

In tasks/dist/css
#20332 (comment):

@@ -0,0 +1 @@
+node-sass --output-style expanded --source-map true --precision 6 docs/assets/scss/docs.scss docs/assets/css/docs.css

probably better if you reference the local npm installs of these (in .bin
) so that people don't need global installs 👍


You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
https://github.com/twbs/bootstrap/pull/20332/files/795f6d924b1c1eb9b637917a41f2c83655408c97#r71592896,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAVVtVO5bBIj0o15D8lDel5ya1-05Smrks5qXnqfgaJpZM4JRHjn
.

This comment has been minimized.

@mdo

mdo Jul 20, 2016

Member

In https://docs.npmjs.com/cli/run-script, it sounded like if the devDependency is there, npm run will do that for us.

In addition to the shell's pre-existing PATH, npm run adds node_modules/.bin to the PATH provided to scripts. Any binaries provided by locally-installed dependencies can be used without the node_modules/.bin prefix. For example, if there is a devDependency on tap in your package, you should write:

Reading that, maybe I just need to add node-sass to devDependency? Again, I've no idea so I'm likely wrong 😁.

This comment has been minimized.

@XhmikosR

XhmikosR Jul 20, 2016

Member

This is true for the case where you call the bin via package.json. Not sure
if it will work here since you call it in a js file. I'll try it when im
home and let you know.

On Jul 20, 2016 23:19, "Mark Otto" notifications@github.com wrote:

In tasks/dist/css
#20332 (comment):

@@ -0,0 +1 @@
+node-sass --output-style expanded --source-map true --precision 6 docs/assets/scss/docs.scss docs/assets/css/docs.css

In https://docs.npmjs.com/cli/run-script, it sounded like if the
devDependency is there, npm run will do that for us.

In addition to the shell's pre-existing PATH, npm run adds
node_modules/.bin to the PATH provided to scripts. Any binaries provided by
locally-installed dependencies can be used without the node_modules/.bin
prefix. For example, if there is a devDependency on tap in your package,
you should write:

Reading that, maybe I just need to add node-sass to devDependency? Again,
I've no idea so I'm likely wrong 😁.


You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
https://github.com/twbs/bootstrap/pull/20332/files/795f6d924b1c1eb9b637917a41f2c83655408c97#r71598457,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAVVtdykpzNATYk8LkXMda5OCPTJJQIlks5qXoLrgaJpZM4JRHjn
.

This comment has been minimized.

@rogeruiz

rogeruiz Jul 21, 2016

Running these task files using npm run * will run them in the npm environment. as you mentioned above @mdo

https://docs.npmjs.com/misc/scripts#environment

Dependency versus DevDependency just has to do with what gets installed when someone using the npm package run npm install you-package. Only dependencies needed at run-time will be installed on the user's system. DevDependencies will only be installed locally when npm install is called within the repository. It's useful / helpful to only include run-time dependencies in dependencies so devs know what's needed to be installed.

FWIW, bootstrap's run-time dependencies are all bundled in the project's files right? So it shouldn't have really have any dependencies.

More info on that here: https://docs.npmjs.com/files/package.json#devdependencies

@thatguyyoulove

This comment has been minimized.

thatguyyoulove commented Jul 20, 2016

As much as I love Gulp, this does seem like the right way to go about things. 👍

@HugoGiraudel

This comment has been minimized.

HugoGiraudel commented Jul 20, 2016

Hey,

Allow me to share my thoughts on this. At Edenspiekermann, we had projects running on Grunt, and we were more and more unhappy about it every day. Then we started a couple of projects with nothing but npm scripts.

It seemed fine at first, and it made things much simpler to actually run. npm run watch, npm run deploy, npm run translations, npm run build, etc. The API looked easier.

And then, we needed more commands. More options. More scripts. To do more things. We kept adding more and more scripts to the package.json, without the ability to properly document them because y’know, JSON. Clever developers did clever things with clever commands, and while it worked fine, it got harder and harder to figure out what was actually going on. Let alone update it.

From there, we started moving our projects to Gulp because this is just what we needed. Simpler than Grunt, but more explanatory, obvious and flexible than npm scripts. Since then, most of our projects run on Gulp and this is a decision we did not regret. We usually have a Gulpfile.js acting as some task manifest, and do have our individual tasks in separate files.


That’s it for the story time. Now regarding Bootstrap, I can think of a few reasons why moving to npm scripts might not be the best idea of all.

First, some commands are not cross-platform, like cat, awk, cp, mv, (I might be wrong for some of them, pardon me). In the current state of the PR, I cannot see any potentially problematic command being used, but it will come eventually I suppose. At some point, we’ll need another task that does something that is easily solved with a bash command which is not supported in Windows environment for instance. I am just afraid of possible future problems around this.

Secondly, command line skills are not a given. I do like CLI much, but I know a lot of developers — especially beginners — who are quite afraid of it, or simply can’t really make their way around it. I wonder whether moving to npm scripts could be some kind of deal breaker for some potential contributors. In a way, Gulp or Grunt appear simpler (even if they come with their share of issues).

Third, this tweet is just full of truth.

Just my 2 cents. :)

@XhmikosR

This comment has been minimized.

Member

XhmikosR commented Jul 20, 2016

I agree too with the gist of this post. Npm scripts will be just as hard to
maintain.

Not sure Gulp is the solution either. It's nice to follow the trends and
all but that doesn't mean they are fit everyone's needs.

I for one don't like Gulp at all for my personal reasons.

To summarize this pull request does not bring any improvement in my
opinion. We have something that works. The dependencies will be
dependencies either way. Maintenance will be maintenance.

On Jul 20, 2016 23:12, "Hugo Giraudel" notifications@github.com wrote:

Hey,

Allow me to share my thoughts on this. At Edenspiekermann, we had projects
running on Grunt, and we were more and more unhappy about it every day.
Then we started a couple of projects with nothing but npm scripts.

It seemed fine at first, and it made things much simpler to actually run. npm
run watch, npm run deploy, npm run translations, npm run build, etc. The
API looked easier.

And then, we needed more commands. More options. More scripts. To do more
things. We kept adding more and more scripts to the package.json, without
the ability to properly document them because y’know, JSON. Clever
developers did clever things with clever commands, and while it worked
fine, it got harder and harder to figure out what was actually going on.
Let alone update it.

From there, we started moving our projects to Gulp because this is just
what we needed. Simpler than Grunt, but more explanatory, obvious and
flexible than npm scripts. Since then, most of our projects run on Gulp and
this is a decision we did not regret. We usually have a Gulpfile.js acting
as some task manifest
https://gist.github.com/HugoGiraudel/a8424fd11ec5bec255bd, and do have

our individual tasks in separate files.

That’s it for the story time. Now regarding Bootstrap, I can think of a
few reasons why moving to npm scripts might not be the best idea of all.

First, some commands are not cross-platform, like cat, awk, cp, mv, (I
might be wrong for some of them, pardon me). In the current state of the
PR, I cannot see any potentially problematic command being used, but it
will come eventually I suppose. At some point, we’ll need another task that
does something that is easily solved with a bash command which is not
supported in Windows environment for instance. I am just afraid of possible
future problems around this.

Secondly, command line skills are not a given. I do like CLI much, but I
know a lot of developers — especially beginners — who are quite afraid of
it, or simply can’t really make their way around it. I wonder whether
moving to npm scripts could be some kind of deal breaker for some potential
contributors. In a way, Gulp or Grunt appear simpler (even if they come
with their share of issues).

Third, this tweet
https://twitter.com/jaffathecake/status/700320306053935104 is just full
of truth.

Just my 2 cents. :)


You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
#20332 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAVVtV5Iij5zIdr5U810f2VWIptqfhNwks5qXoFBgaJpZM4JRHjn
.

@lorvent

This comment has been minimized.

lorvent commented Jul 20, 2016

Great idea,
but i also lean towards gulp.

Also, since alpha 3 is getting late, can i ask you to move this to alpha 4 phase please.

its been 7 months you released an alpha and this is a breaking change which requires lot of time to fix everything...which makes alpha3 delay again.

2days back, no. of open issues in alpha 3 are 25 and i am happy that its getting closer but today again its 27 and this issue (big change) is one of those new ones.

so please consider moving it to alpha 4, so you will have lot of time to work on it.

thanks.

@mdo mdo modified the milestones: v4.0.0-alpha.3, v4.0.0-alpha.4 Jul 20, 2016

@artemgurzhii

This comment has been minimized.

artemgurzhii commented Jul 20, 2016

What are the reasons not to use Gulp? As now Gulp4 hasn't been released yet but it's already in 4.0.0-alpha.2 stable version which let you run tasks asynchronously using gulp.parallel, it also let you use ES6 in gulpfile.babel.js and other and gulp will also add you more functionality, unlike npm scripts.

@ahtik

This comment has been minimized.

ahtik commented Jul 20, 2016

Has there been webpack on the table? It's been a great replacement for our projects replacing gulp& grunt for CSS and JavaScript pipeline.

If the decision is already set for pure npm scripts then of course my note is just a noise and let's move on with the current plan :)

@kaelig

This comment has been minimized.

kaelig commented Jul 21, 2016

When I started working on the Salesforce Design System, we had a lot of bespoke scripts that were sort of using gulp in places, but not in a standard way. We were mostly using npm for our task orchestration. It was very challenging to troubleshoot performance issues and other bugs because of the lack of documentation, testing, and the inability to find community help.

We started porting as much of our development environment to gulp as we could.
A few interesting side effects happened:

  • we're able to implement the basics at no cost (there's plugins for basically everything)
  • a new joiner is able to apprehend our codebase (especially since documentation on Gulp is easy to find!), no need to be an expert engineer to understand what's going on and propose changes
  • performance: fast! Transitioning to gulp made our development environment so much faster than what we had before. Gulp seems very well optimized for speed
  • we write our own plugins when needed. Gulp provides standard ways to build plugins, meaning anyone in the team can understand/maintain them
  • Windows support: Gulp plugins typically work in Windows, so we have a lot less testing to do. We got that for free too.

I hope this helps making a decision. I am not advocating Gulp in particular, but definitely advising against npm scripts for everything as I don't think it's a maintainable solution.

Now we have a mix where we use gulp for what it's good at (running a front-end development environment) and npm for our testing, installation and production build process orchestration. We're pretty happy with it!

If you're curious about the way we do things, here's our package.json: https://github.com/salesforce-ux/design-system/blob/master/package.json#L115
and our gulpfile:
https://github.com/salesforce-ux/design-system/blob/master/gulpfile.babel.js

@cvrebert

This comment has been minimized.

Member

cvrebert commented Jul 21, 2016

So, having worked a lot on the build process, I'm heavily in favor of npm scripts, though perhaps not this exact implementation. (I favor doing it piece-by-piece in several small PRs.)

Firstly, there are currently N different JS build tools (Grunt, Gulp, Brunch, Broccoli, etc.) battling it out for dominance. It's not obvious which is going to win, or how long the winner will last until a hip new rival appears, or even how long the "winner" will stay maintained. (Remember how Grunt got really popular but then its maintenance stagnated for a long while?) So, I'd rather stay out of the brawl and not pick any of them.

Secondly, these build tools typically involve relying on a layer of build-tool-specific plugins that are often not as actively maintained as their upstream projects, which can delay updates and fixes. By using the upstream CLIs directly, we avoid this problem.

With regard to Windows compatibility, I'm optimistic about https://github.com/shelljs/shx (or similar) as a solution to that problem.

[...] command line skills are not a given.

Newbies at that level are unlikely to be modifying the build in the first place, and I would feel nervous about accepting build-related PRs from such folks.
Most folks merely need to know how to invoke the build, which ought to be trivial. I also agree with @mdo that the shell is more approachable (certainly it's more fundamental) than any of the build tools.

The dependencies will be dependencies either way. Maintenance will be maintenance.

At the least, we'd have one less layer of dependencies (no "plugins") to deal with, and we'd depend on shx (or similar) instead of the build tool. And the build tools implement complicated things, whereas shx implements simple things like cp and mv, so less chance of bugs and less chance of difficult upgrades.

@ahmedelgabri

This comment has been minimized.

ahmedelgabri commented Jul 21, 2016

I also share the same opinions like @HugoGiraudel & @kaelig. npm scripts tends to get really complex at some point and hard to maintain. especially when you need add more flags, configs, etc...

I just finished migrating our app to use gulp/webpack & in fact our package.json scripts section looks very similar to Salesforce one, even with the "gulp": "gulp" task.

What I think is pretty good setup is this:

  • Use gulp/Make/whatever build tool for the project, but...
  • Use npm scripts to run these commands, this will give you the flexibility to change the underlying tool at anytime without changing the commands. Most people won't care about the underlying implementation details anyway.
  • This is often missed and not really used enough by people but you can pass flags/options to your npm scripts like so npm run <task> -- --flag [--flag2] which can be really useful and powerful too.

This tweet can show you also how things can get out of hand at some point https://twitter.com/sindresorhus/status/724259780676575232

@jbokkers

This comment has been minimized.

jbokkers commented Jul 21, 2016

First of all, I'm all for refactoring and simplifying the tooling and their dependencies but I'm with @HugoGiraudel on this.

You'll be placing all your hope for twbs4 on a single build dependency which currently has 88% coverage and 6 contributors by enforcing sfx for the entire community.

and if you don't want to enforce but expect it to work magically across all OS'es

You'll be putting all your hopes on IDE vendors that they'll update their integrated tooling to support this or expect developers to manually work around commands if we run into command-line issues like @HugoGiraudel pointed out...

We'll all follow you where ever you take us with v4, but by not picking one out of the brawl, you're also making a choice and its not necesarrily one that makes our live easier because for us, Bootstrap's always part of a bigger solution and not the final end-result...

@diegobfernandez

This comment has been minimized.

diegobfernandez commented Jul 27, 2016

Totally agree with @ahmedelgabri. npm should be the interface to go and abstract the underlying tooling. That way no one will get confused if the tooling change.
Although I use npm scripts a lot to manage projects tasks I think Webpack is a better choice for browser code.
This is a production-grade project I work on.

...
"scripts": {
  "clean": "rimraf ./dist",
  "start": "NODE_ENV=development webpack-dev-server --hot --host 0.0.0.0",
  "build": "NODE_ENV=production webpack -p",
  "build:debug": "NODE_ENV=debug webpack",
  "lint": "eslint ./",
  "test": "karma start",
  "test:watch": "npm test -- --auto-watch --no-single-run",
  "publish": "npm run clean && node scripts/publish.js"
}
...

I myself never executed these scripts on Windows but others on the team use Windows on a daily basis and no one ever complained. Running tasks sequentially with && works nice cross-platform but for parallel execution there are options available.

I know Bootstrap is an exponentially larger project and there will be many more dificulties but the upsides seems to outweight the downsides.

@jbokkers shx is just a wrapper for ShellJS which has 52 contributors, 4k stars and 6M downloads last month on npm. Seems like a robust project used by many other respected projects.

@mdo mdo modified the milestones: v4.0.0-alpha.4, v4.0.0-alpha.5 Jul 30, 2016

@HallofFamer

This comment has been minimized.

HallofFamer commented Aug 1, 2016

Well why is there still alpha 5 after alpha 4? I thought alpha 4 is the last Bootstrap 4 alpha release, and then it will go Beta.

@cvrebert

This comment has been minimized.

Member

cvrebert commented Aug 1, 2016

Well why is there still alpha 5 after alpha 4? I thought alpha 4 is the last Bootstrap 4 alpha release, and then it will go Beta.

That was more of a hopeful prediction as opposed to some sort of hard-and-fast promise.
Also, alpha 3 had a bad packaging bug (#20385), which might necessitate another release just to get that fix out quickly.

@sleewoo

This comment has been minimized.

sleewoo commented Aug 2, 2016

so dist/umd ditched from alpha.3
was truly nice to load only needed modules.

would you consider add them back?
or at least distribute js source via NPM so people can load modules without fetching bootstrap source from github.

thanks for great and tremendous work.
very appreciable efforts.

@cvrebert

This comment has been minimized.

Member

cvrebert commented Aug 2, 2016

@sleewoo You're veering off-topic. See #20072 for the context of that decision. Feel free to comment over on that PR.

@sleewoo

This comment has been minimized.

sleewoo commented Aug 2, 2016

uh-oh, did not check deep enough :)
thanks for pointing

@JacobLett

This comment has been minimized.

JacobLett commented Oct 24, 2016

Would it be possible to have a sans-docs package.json with the documentation scripts and dependencies removed? That way a beginner can see the bare minimum they need to start working with the source files. Web designers that just want a custom build of Bootstrap could get overwhelmed easily.

@pvdlg

This comment has been minimized.

Contributor

pvdlg commented Jan 17, 2017

I've spent some time trying to continue the effort to migrate the build process to npm scripts only or at least to improve it.
I'd really like to propose a full PR for the first beta but I ran in several issues and I have several solutions to propose.
A full PR would takes time, so before moving forward I'd like to expose the issues and solutions and have some guidance from the Bootstrap team regarding the desired approach.

Problems encountered with a full npm script solution

Some tasks would require a lot of custom code to be migrated

  • Babel: babel-cli does not support the attribute --extends. A npm solution would have to move the .babelrc at the root of the compilation, then delete it.
  • stamp: the npm solution would involve creating JS scripts to generate the banner and the footer. In addition, concatenating the banner, the file and the footer would require extra work and JS scripts as ShellJS cat command does not support concatenating files and stdin shelljs/shelljs#637
  • QUnit: qunit-cli does not support the inject option
  • htmllint: htmllint does not support the ignore option
  • saucelabs: no cli tool I could find
  • buildcontrol: no cli tool I could

Simplicity for the user / developer

Even if we externalize the npm scripts, each one of them has to be referenced in the package.json and are exposed to the user/developer. That would be several tasks and subtask and it would be difficult to know which one is a subtask only and which one is supposed to be run.

Configuration

Ideally we would want to have a config file that define all the paths in the projects (dist, dist/js, dist/css scss, js/src etc... and global parameters like sass precision, babel plugins etc...). It's possible in package.json to define a config attribute and its values are exposed as environment variables, accessible by the scripts.
The problem is reading environment variables in not cross-platform. The only workaround is npm-cross-conf-env but it prevent to use pipe, defeating an important feature of npm scripts.
So all the paths and other config global to the project have to be hard-coded in the command in multiple places, making maintenance harder.

Dependencies

One of the objective of the full npm solution was to limit the number of dependencies. In practice we would just replace a grunt plugin dependency by a command line one.
For example:

I found that in many cases the cli package is less maintained and offer less configuration than the Grunt or Gulp plugin. In several cases, as mentioned above, there is no cli tool while there is a good Grunt or Gulp plugin. Other than sass, cleancss, eslint and a few others the npm cli tools is quite limited.
A lot of the npm package do not include a cli tool. The either require a cli wrapper (either inexistent or less maintained than the Grunt/Gulp equivalent) or require us to write a custom JS script to call them.

Conclusion

Many tasks would require to write custom node scripts (in javascript) and add more complexity and more difficulty to maintain, therefore defeating the purpose.
The possible solution would be to migrate the task to NPM that are simple enough and to keep the other in Grunt. But that would mean an ever more complicated build process as we have to maintain a bridge between both tools like here and there.

Objectives

Despite the issues mentioned above I think the build process can be greatly improved and simplified. Here is what I think we can achieve.

Simple to use

We should expose a limited number of command:

  • Install (1 command to install the whole environment and dependencies include jekyll)
  • Build
  • Watch
  • Release

All these command should be started from the same tool and should be the only one mentioned in the documentation.

A lot faster

We should run the tasks we can in parallel to make the process as fast as possible.
With current tool it takes:

  • 25s for a full build
  • 12 second to build the css
  • 4 seconds to build the JS

With a reorganization of the tasks and parallelisation I managed to achieve ~15s for a full build, a ~2s to build the js and the css.
That would make the development experience a lot better as watch would be done in a couple seconds.

More modular

Ideally we should avoid big files and try to have smaller, more manageable tasks in their own files.

Maintainable

The less custom JS scripts the better. The less config and path duplicated the better (i.e. --precision 6 for node should be set only once).

Propositions

1/ Grunt as the engine, NPM as the user interface

  • All tasks implemented in Grunt for consistency. Grunt does not call NPM scripts.
  • Each Grunt task configured in its own file.
  • Small JS scripts for generating the banner and determining the type of test to run if running on Travis (same as currently but in it's own file) .
  • Tasks available to the user/developer:
    • npm install (install everything required including jekyll, grunt etc..)
    • npm run build (dist + lint)
    • npm run watch (watch js + watch css + start local doc + live-reload on change)
    • npm run release (build + change-version + github site + create downloadable zips)
  • Parallelisation handled with grunt-concurrent

2/ Gulp as the engine, NPM as the user interface

Same as 1/ but with Gulp as the engine.
Has the advantage to handle concurrency natively but I'm not sure there is a point in rewriting the Grunt tasks we already have.

3/ Gulp + NPM as the engine, NPM as the user interface

Migrate to npm for the task that are simple simple enough, but keep Grunt for the others.

  • Will require to keep a bridge between Grunt and NPM (so Grunt can call NPM tasks and vice-versa)
  • Less consistency and would complicate the maintenance
  • Less clear for the user / developer what task he should run as there would be a bunch of them in the package.json
  • Keep the few task that has already been migrated to NPM

Next steps

After all that, I think /1 is the best solution to achieve the objective stated in the first comment by @mdo (even though the solution is the opposite of the one proposed).
It's IMO and for the reasons exposed above, the cleanest possible solution and the one that takes the less rework.

I understand the desire to move to npm in order to simplify the build process. I think npm scripts are the most simple solution as long as you build is limited to sass, concat, minify and lint. Everything beyond that (babel, qunit, file headers, linting other than stylestat and eslint, saucelab, conditional build for Travis) would require so much custom JS scripts that it defeat the purpose.

@twbs and community, I'd love to have your feedback. Hopefully, if a decision is made I think I can implement it in the following couple weeks.

@cr101

This comment has been minimized.

cr101 commented Jan 17, 2017

Migrating to Webpack should also be considered in my humble opinion.

@bardiharborow

This comment has been minimized.

Member

bardiharborow commented Jan 18, 2017

@vanduynslagerp you present some valid points.

Babel: babel-cli does not support the attribute --extends. A npm solution would have to move the .babelrc at the root of the compilation, then delete it.

We could use --presets=./js/.babelrc.js or I can bug the Babel team to support --extends or we can move .babelrc to the root.

stamp: the npm solution would involve creating JS scripts to generate the banner and the footer. In addition, concatenating the banner, the file and the footer would require extra work and JS scripts as ShellJS cat command does not support concatenating files and stdin shelljs/shelljs#637

I still need to work out how we are going to do this, but as you'll notice, shelljs/shelljs#637 is my issue, and I'm working on a PR for it.

QUnit: qunit-cli does not support the inject option

As per #21003, the inject file can be removed if needed.

htmllint: htmllint does not support the ignore option

I need to review our choice of HTML linter anyway. We are using two, one of which requires Java, and it all seems a bit weird.

saucelabs: no cli tool I could find

We might be dropping SauceLabs soon and moving to BrowserStack (#18754), but we can switch to this in the meantime.

buildcontrol: no cli tool I could

Normal git commands will work in this instance.

In conclusion, there are going to be one or two rough edges until a few upstream issues get fixed, but I think that we should be able to drop grunt without too many issues. You might be interested in looking at bardiharborow/bootstrap/build, my heavily work-in-progress build branch.

@pvdlg

This comment has been minimized.

Contributor

pvdlg commented Jan 19, 2017

@bardiharborow I see we reached more or less the same situation :-)

I think the initial objective was to make the build tool more simple, more maintainable and more pleasant to work with.
After a couple hours we were both writing custom Node script, opening bug and finding workarounds...
Not mentioning that if we solve these rough edges, other issues remains:

  • A bunch of scripts in a JSON file that are so long they wrap on 3 or 4 lines, with hardcoded parameters is difficult to maintain
  • Even if writing shell script make it accessible to more people, the necessity to write and maintain some node script make it harder to many (dev who develop mostly css for example)
  • The build is relatively slow, especially the watch that takes 10s when you edit a scss file (which is due to the way the task are organized more than the tool we use)
  • The issues we've encountered (bugs, missing parameters in cli tools etc...) to migrate the existing might be encountered in the future for new tasks

So I wondered "what if we keep what we have that works and rearrange the part that are not that great?"
So I started to investigate what can be achieve with Grunt and the existing. It turns out we can do a lot, and it was quite fun to do. So much that after a couple hours I had a decent build process. As it was almost complete I decided to spend a bit more time to polish it, update the doc and I opened this PR #21777.

I understand there is an effort to move to npm, and frankly my opinion is "why not?". This is why I spend time at the beginning to help in that direction. But now that I have something that works pretty well and I think achieved at least part of the objectives mentioned in the very first post, I thought it might worth to share :-)

@TomFoyster

This comment has been minimized.

TomFoyster commented Mar 3, 2017

I'm trying to convert the Grunt build process to Gulp - but I'm coming unstuck at the JS processes as I'm not able to figure out where some of the JS comes from.

I see that the Official Bootstrap Theme's state they use Gulp for their build tasks though?

@mdo

This comment has been minimized.

Member

mdo commented Mar 26, 2017

Closing for #22045.

@mdo mdo closed this Mar 26, 2017

@mdo mdo removed this from the v4.0.0-beta milestone Mar 26, 2017

@mdo mdo deleted the v4-npm branch Jun 15, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment