Automated analysis of what frameworks give you 'out of the box'

This is a follow-up to JavaScript Framework Battle: ‘Hello World’ in each CLI

How does it work?

For each CLI tool, the following happens (automated)

1. First, the 'scaffold' command is run to create a default application
2. Then the 'build' is run to produce the production ready assets
3. A lightweight server is then spun up pointing at the newly created production app
4. Then we run Lighthouse against the app to gather metrics.

Currently I'm just looking at JS bundle size as a POC (and because it's of particular interest to me - wanting to begin projects with as little vendor code as possible to reduce size).

I will add more metrics soon. (The following table is generated by the automated script)

Framework	JS Bundle Size (compressed, as served in the browser)
Preact (Create Preact App)	9.29 kB
Inferno (Create Inferno App)	17.5 kB
Glimmer (Ember CLI)	34.7 kB
Vue (Vue CLI)	45.9 kB
React (Create React App)	47.7 kB
Angular 2 (Angular CLI)	94.6 kB
Ember (Ember CLI)	205 kB

Up next

Once we have the setup to run all of these tools in a uniform way, it will make it easier for more automated benchmarks in the future. For example, we could compare hacker-news clones etc using the massive amount of detail you can get out of Lighthouse.

Crossbow tasks

The following tasks have been defined by this project's Crossbow configuration. Run any of them in the following way

$ crossbow run <taskname>

Task name	Description
glimmer-run	Create production Glimmer app & test in lighthouse
ember-run	Create production Ember app & test in lighthouse
cra-run	Create production React app & test in lighthouse
cpa-run	Create production *Preact app & test in lighthouse
ng-run	Create a production NG2 app & test in lighthouse
vue-run	Create a production Vue app & test in lighthouse
install:global	Alias for: - @sh yarn global add create-react-app create-preact-app ember-cli/ember-cli vue-cli @angular/cli perf-battle
glimmer:clean	Alias for: - @sh rm -rf $GLIMMER_DIR
glimmer:init	Alias for: - @npm ember new $GLIMMER_DIR -b @glimmer/blueprint
glimmer:build	Alias for: - @sh cd $GLIMMER_DIR && ember build --environment="production"
glimmer:serve	Alias for: - @sh pushstate-server $GLIMMER_DIR/dist $PORT
glimmer:perf-battle	Alias for: - @npm perf-battle http://localhost:$PORT --reporter size --output json --outFile results/glimmer.json
glimmer:lighthouse	Alias for: - glimmer:perf-battle - @cb exit
ember:clean	Alias for: - @sh rm -rf $EMBER_DIR
ember:init	Alias for: - @npm ember new $EMBER_DIR
ember:build	Alias for: - @sh cd $EMBER_DIR && ember build --environment="production"
ember:serve	Alias for: - @sh pushstate-server $EMBER_DIR/dist $PORT
ember:perf-battle	Alias for: - @npm perf-battle http://localhost:$PORT --reporter size --output json --outFile results/ember.json
ember:lighthouse	Alias for: - ember:perf-battle - @cb exit
cra:clean	Alias for: - @sh rm -rf $CRA_DIR
cra:init	Alias for: - @npm create-react-app $CRA_DIR
cra:build	Alias for: - @sh cd $CRA_DIR && yarn run build
cra:serve	Alias for: - @sh pushstate-server $CRA_DIR/build $PORT
cra:perf-battle	Alias for: - @npm perf-battle http://localhost:$PORT --reporter size --output json --outFile results/cra.json
cra:lighthouse	Alias for: - cra:perf-battle - @cb exit
cpa:clean	Alias for: - @sh rm -rf $CPA_DIR
cpa:init	Alias for: - @npm create-preact-app $CPA_DIR
cpa:build	Alias for: - @sh cd $CPA_DIR && yarn run build
cpa:serve	Alias for: - @sh pushstate-server $CPA_DIR/build $PORT
cpa:perf-battle	Alias for: - @npm perf-battle http://localhost:$PORT --reporter size --output json --outFile results/cpa.json
cpa:lighthouse	Alias for: - cpa:perf-battle - @cb exit
ng:clean	Alias for: - @sh rm -rf $NG_DIR
ng:init	Alias for: - @npm ng new $NG_DIR
ng:build	Alias for: - @npm cd $NG_DIR && ng build --aot --target=production
ng:serve	Alias for: - @sh pushstate-server $NG_DIR/dist $PORT
ng:perf-battle	Alias for: - @npm perf-battle http://localhost:$PORT --reporter size --output json --outFile results/ng.json
ng:lighthouse	Alias for: - ng:perf-battle - @cb exit
vue:clean	Alias for: - @sh rm -rf $VUE_DIR
vue:init	Alias for: - @npm vue init webpack $VUE_DIR
vue:build	Alias for: - @npm cd $VUE_DIR && yarn && yarn run build
vue:serve	Alias for: - @sh pushstate-server $VUE_DIR/dist $PORT
vue:perf-battle	Alias for: - @npm perf-battle http://localhost:$PORT --reporter size --output json --outFile results/vue.json
vue:lighthouse	Alias for: - vue:perf-battle - @cb exit