Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

build script: v.Next #165

paulirish opened this Issue Oct 29, 2010 · 34 comments


None yet

paulirish commented Oct 29, 2010

A few issues and ideas I've noticed when working with the build script:

  • Build script should gracefully handle additional CSS and HTML files: issue #143
    • fixed with new build script.
  • minifier will overwrite an existing one... so if you have a regular jquery and a min'd jquery.. the min'd one will get replaced with your own compressed version. I'd rather see the script honor any already min'd files you have.
    • update. It doesn't overwrite them now, because libs files always get minned with the build#. In jquery's case the original one is used. That's sort of a bug but that's okay.
  • Right nowscripts are concated alphabetically. This probably won't work in some script dependency scenarios. Getting them in the script order of the markup would be ideal.
    • This could be dealt with by changing the hardcoded *.min.js in build.xml to a .property much like the file.stylesheets and file.pages
    • Relatedly, looking at which scripts are in the markup let us ignore extra js files that are in the folder but aren't actively used.
    • html5lib ? yup, i went there
  • If we're handling file revving in the build script, we can confidently upgrade the expires timeouts in .htaccess/etc to 1yr
  • Leverage zakas's cssembed to get datauri'd images in the css when sensible.
  • investigate plovr
    • uglify seems to be better than closure these days.. doubting this one now.
  • consider css-compressor instead of YUI for css. it inlines @imports and minifies better (so sez stoyan).. php based though..

roblarsen commented Oct 29, 2010

I've been playing around with the command line smushit. It's actually pretty brute force (you just point it at a directory and it just goes to town) so at this point I'm not sure it's a replacement for the more granular approach of the two binaries. I'll see how it goes.

I love the idea of the data uris. At the Boston Performance Meetup we went into them pretty heavily and it was clearly something that could/should be automated.


paulirish commented Nov 15, 2010

Dave Kirk just added a bunch of stuff but he's not too sharp on Git so i have it in a zip here....

You can download the build directory from http://dl.dropbox.com/u/9273778/build.zip

You can now add multiple stylesheets to the project (defined in the project.properties file) and they will be concatenated into 1 file, with any default stylesheets defined in the default.properties file being added at the end.

Adding new 'pages' (html, php, etc) is handled in the same way and the script should iterate through any files listed here and update the css / javascript references as per the original script

I have also added the ability to define 'environments' for the build process as this fits in with the way I work. The original targets still exist, and run the same functions, however each target now has a prod, test and dev environment too. This has come at a as it requires using the ant-contrib library and due to the way Ant works you can no longer use the depends model, and the sub targets have to be called with antcall.

Here is how the environments work:

  • dev (default) - Increases build number, cleans and copies the build and optimises any images if the target originally did
  • test - Runs everything that the original target did, however it does not strip the console.log or profiling parts
  • prod - Runs everything the original target did

To run it you simply use ant <target> -Denv=<environment>

ant build -Denv=prod

If you don't provide the env variable it will use the development environment, but that would be pretty easy to change to run another environment as default.

Like I say, it could probably still be improved no end but it is getting to the limits of Ant without writing a java library with extra tasks, which I may look at in the future....


paulirish commented Nov 15, 2010

Rob, if you wanna dig into this and bring it in, that'd be great. I'll be tied up for a few days.


roblarsen commented Nov 15, 2010

I'll take a look this week. I've got a bit of a break between writing assignments, so I can dive into this, this week.

Many developers know how to right and read php incredibly well.
It really sounds like you are trying to grow this build script idea, and php provides familiarity and easy maintainability.
Just curious, have you guys thought about writing the build scripts in php?


paulirish commented Dec 20, 2010

@limeblack, we want to stay backend agnostic as much as we can.

however..... css compressor is appearing to be one of the finest css minifiers.. http://bit.ly/uGaKG iterating quickly and it looks more robust than others. however it's PHP. we should consider trying to use it in the build script if php is available and using a fallback otherwise.
is that even possible? hope so. :)


paulirish commented Jan 23, 2011

Just a heads up. I've merged in the build script work from Dave Kirk. We have a lot more flexibility and extensibility now. (and hopefully isn't any harder to learn)

I'm gonna try to update the docs as this stuff is fresh in the mind.


andrewle commented Jan 24, 2011

Great work by Dave Kirk on making build much more flexible. I was just playing with the build scripts, however, and noticed that running vanilla ant build will end up doing parts of the build twice and does not create the minified and concatted js files.

It seems that ant is executing dependent targets more than once, which it shouldn't. Can you verify whether this is an issue you as well and not specific to me?


paulirish commented Jan 24, 2011

@andrewle, you're using the latest version that's in the repo?
I had some issues with no min/concat (eventually leading to a 404 on scripts-xxx.min.js on the page).. but fixed them after landing the big patch.

Here is my build just now... https://gist.github.com/458e710208567d2aa670

yeah wtf....
this is definitely not where i had it last night...


paulirish commented Jan 24, 2011

@andrewle, fixed in 15b3282 i was being lazy with some last minute fixes yesterday.. turns out @Depends works different than I thought. Thanks mucho for the heads up.

We still have a bunch of extra load-build-info but as far as I can tell, they aren't hurting anything. To fix I'll have to include the new targets -serverconfig and jslibsminify into the repeated chunks at some point.


andrewle commented Jan 24, 2011

Thanks for the quick fix. The extra load-build-info calls shouldn't be a problem, but it would be nice to remove them so we can have clean builds. :)

I'm new to ant builds, but it looks likes one of the reasons this became an issue is that antcall executes all of its dependent targets regardless of whether they've been called before. If we pull out the tasks called with antcall into a containing target as dependents and just call the container targets instead, that'll give us more straight-forward dependency resolution.

I've committed an example you can check out on my fork: b168c23

This gets us a cleaner with just one load-build-info: https://gist.github.com/792962


roblarsen commented Feb 13, 2011

I'm going to go through the process of tweaking the new build script for my personal-use project branch. Feedback/bug fixes should follow.

Additionally, is a "cookbook" section of the wiki useful? People are going to be tweaking the build script all day for their own needs, so it might be nice to share some ways the build script can be tweaked for different project goals.


paulirish commented Feb 13, 2011

Rob, yeah a cookbook section for build script configuration would be hugely useful. Maybe augmenting the existing build script docs would work well.

I'm also digging back into the build script today (as is shi chuan) so hopefully we all emerge with added polish.


paulirish commented Feb 22, 2011

We've landed a few fixes and improvements to the build script in the past 3 weeks. I'm feeling really good about it.

I just updated some stuff at the top of this ticket around html5lib, which still interests me greatly... but we can currently work around parsing the HTML ourselves, so I'm not gonna block 1.0 on it.

I think you can consider the build script in RC status. Let me know if anything should change.

Rob, if you can add those Build Script cookbook docs, that'd be awesome.. Check out how things look now.. http://html5boilerplate.com/docs/#Build-script.md :)


roblarsen commented Feb 23, 2011

I'll see what I can do over the next few days. I've got about 9 hours of airplane time coming up, so if sleep doesn't take me I'll knock at this a little bit.

Some additional ideas for build script.
Make CSS sprites automatically with something like https://github.com/hagenburger/lemonade . Additionally something like SASS or Less can be used (they support @import and compatible with simple CSS).
Jruby can be used for ruby scripts

scq commented Feb 25, 2011

Google's closure compiler generates files 10-15% smaller in size than YUI Compressor on the "Simple optimizations" mode.


roblarsen commented Feb 28, 2011

I started in on the cookbook section on my flight yesterday. I've got a few more examples I plan to add as soon as I get time. We'll see how the week goes.


shichuan commented Mar 7, 2011

although this shouldn't be a showstopper for v1.0, but i think we can consider using some of the following features for v1.1+ roadmap

gifsicle: to optimize GIF animations by stripping repeating pixels in different frames.
ImageMagick: to identify the image type and to convert GIF files to PNG files.
also to output the total bytes saved in console


roblarsen commented Mar 7, 2011

Also... I just got a couple of sample ant tasks that use cssembed to embed Data: uris into CSS files. Beyond just getting it working for my own devices, I'm going to see if I can make it configurable enough to work here.


paulirish commented Mar 7, 2011

An additional idea from davidmurdoch today was generating the 5ish apple-touch-icon's neccessary these days. Trivial if we're using ImageMagick.


nimbupani commented Mar 15, 2011

Great idea, but I think we should provide all 5 as default anyways.


roblarsen commented Mar 16, 2011

hey all. the cssembed task, as of right now, is an all or nothing proposition. Anyone have any opinion on how it might be included? Should it be? As it stands it seems to me like a single serving site enhancement. Add another target? it almost seems like it would start to turn into https://github.com/remy/inliner


paulirish commented Mar 17, 2011

I think it'd be fun as an additional task, maybe one that operates on the published files, so it's just run totally separately?


roblarsen commented Mar 17, 2011

Ah, good idea. That'd be pretty easy to pull together, too.

On the subject of potential switch away from YUI compiler:

I am porting H5B to waf build system (for selfish reasons and general hatred towards writing IF statements in XML) and found an interesting, stable replacement for YUI for css compression:

slimmer (http://pypi.python.org/pypi/slimmer/) for HTML, CSS.

This sorta clears my way to switch from YUI compiler to Closure (cause Node.js is a pita to get right on Windows).

On the subject of managing H5B CSS file:

The file asks the user to put custom CSS in between prescribed blocks. This means CSS literally must be part of the CSS file. Diffing customized version vs new mainline version becomes painful. @include would not work, theoretically, because it must be on top, not in the middle.

Of course one can do style.css file with:
@include "H5B_prefix.css"
@include "mine.css"
@include "H5B_suffix.css"

But that's 4 downloads and style flicker with every download.

Thoughts on sucking http://sass-lang.com/ and http://compass-style.org/ into H5B?


JoeMorgan commented Apr 5, 2011

Any interest in JSLint integration? Made a pull request just in case.


roblarsen commented Apr 5, 2011

Hey Joe,


How about JSHint too? I don't like having my feeling hurt. You could make it configurable in the properties file.


JoeMorgan commented Apr 5, 2011

Done. Added JSHint as a separate target and passed in options from project.properties (overrides in default.properties).

ivoba commented Aug 31, 2011

What about deployment tasks like f.e. FTP?
Ant has a FTP task, its not standard however and needs some additional jars.
I use it to upload the build folder with an seperate task: ant ftpupload prod

if thats nothing for the core build.xml i would also request something like here: https://github.com/paulirish/html5-boilerplate/issues/704
to avoid editing build.xml


nimbupani commented Sep 13, 2011

ivoba, #704 is now in the build.


rigelglen commented Oct 8, 2011

@ivoba I'd double that, maybe we can go crazy with git and upload only the files that need uploading. http://goo.gl/VRWnG


necolas commented Feb 3, 2012

The build script isn't part of this repo anymore, so please harvest whatever you want to keep track of and open a corresponding issue (with link back here) in the build script repo you're working on. We've got ant, rake, and node repos to choose from.

I'll close this soon.

cc @mklabs @KushalP

@necolas you gave 3 identical links. Here are right node, rake

@necolas necolas closed this Feb 7, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment