ci: Travis build speed improvements #504

lychyi · 2020-03-10T05:53:33Z

Things we did to speed up Travis (hopefully for real this time):

Exit Chromatic after upload so it doesn't hold up the build
Remove caching for yarn and npm (node_modules). Loading and building the cache is longer than the savings incurred by having a cache.
Deduplicate instances of Webpack in our yarn.lock
Replace concurrently with run-p (part of npm-run-all, a dep we already use). Concurrently was a 40mb hit to our cache and didn't add any value from what we already have.

We went from around 20 to 25 min to 15min. Not too shabby.

…ravis-part-deux

cypress · 2020-03-10T06:09:57Z

Test summary

185 • 0 • 0 • 0

Run details


Project	canvas-kit
Status	Passed
Commit	`f410af1` ℹ️
Started	Mar 11, 2020 4:26 AM
Ended	Mar 11, 2020 4:29 AM
Duration	02:46 💡
OS	Linux Ubuntu Linux - 16.04
Browser	Electron 78

View run in Cypress Dashboard ➡️

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

Reduce that cache size and node_modules folder size

lychyi · 2020-03-10T22:47:49Z

.travis.yml

-        "yarn cypress run --record" &&
+        yarn http-server docs -p 9001 & npx wait-on http://localhost:9001 &&
+        yarn run-p -l
+        "chromatic -- --quiet --exit-zero-on-changes --exit-once-uploaded --storybook-build-dir docs"


Most of the work is done on Chromatic's side so this allows us to make our Chromatic build command exit after we've uploaded the Storybook source. This also ensures that we don't incur CI build time increases in Travis as we add more visual testing stories.

--exit-once-uploaded

Genius. I didn't matter much when we only had Chrome and it took like 15s for 300 stories. IE11 is much slower. We already have a status check that is reported by Chromatic. We don't need to wait for it here. 👍

lychyi · 2020-03-10T22:48:41Z

.travis.yml

-        "yarn lint"
-        "yarn depcheck"
-        "yarn test"
+        yarn run-p -l


For some reason, concurrently is about 40mb and npm-run-all/run-p pretty much do the same thing at a fraction of the size.

lychyi · 2020-03-10T22:49:32Z

package.json

@@ -137,6 +135,7 @@
    "create-module": "./utils/create-component/createComponent.js",
    "create-component": "./utils/create-component/createComponent.js",
    "precommit": "lint-staged",
+    "chromatic": "chromatic",


Need this to run chromatic via run-p in Travis

NicholasBoll · 2020-03-11T14:55:03Z

.travis.yml

@@ -74,7 +73,7 @@ jobs:
    script:
      - >- # Chromatic relies on a built Storybook, so exit early if build-storybook fails
        yarn build-storybook &&
-        yarn chromatic --quiet --auto-accept-changes --storybook-build-dir docs
+        yarn chromatic --quiet --auto-accept-changes --exit-once-uploaded --storybook-build-dir docs


NicholasBoll · 2020-03-11T14:58:03Z

yarn.lock

-webpack@^4.33.0:
-  version "4.34.0"
-  resolved "https://registry.yarnpkg.com/webpack/-/webpack-4.34.0.tgz#a4c30129482f7b4ece4c0842002dedf2b56fab58"
-  integrity sha512-ry2IQy1wJjOefLe1uJLzn5tG/DdIKzQqNlIAd2L84kcaADqNvQDTBlo8UcCNyDaT5FiaB+16jhAkb63YeG3H8Q==
-  dependencies:
-    "@webassemblyjs/ast" "1.8.5"
-    "@webassemblyjs/helper-module-context" "1.8.5"
-    "@webassemblyjs/wasm-edit" "1.8.5"
-    "@webassemblyjs/wasm-parser" "1.8.5"
-    acorn "^6.0.5"
-    acorn-dynamic-import "^4.0.0"
-    ajv "^6.1.0"
-    ajv-keywords "^3.1.0"
-    chrome-trace-event "^1.0.0"
-    enhanced-resolve "^4.1.0"
-    eslint-scope "^4.0.0"
-    json-parse-better-errors "^1.0.2"
-    loader-runner "^2.3.0"
-    loader-utils "^1.1.0"
-    memory-fs "~0.4.1"
-    micromatch "^3.1.8"
-    mkdirp "~0.5.0"
-    neo-async "^2.5.0"
-    node-libs-browser "^2.0.0"
-    schema-utils "^1.0.0"
-    tapable "^1.1.0"
-    terser-webpack-plugin "^1.1.0"
-    watchpack "^1.5.0"
-    webpack-sources "^1.3.0"
-
-webpack@^4.38.0:
-  version "4.41.2"
-  resolved "https://registry.yarnpkg.com/webpack/-/webpack-4.41.2.tgz#c34ec76daa3a8468c9b61a50336d8e3303dce74e"
-  integrity sha512-Zhw69edTGfbz9/8JJoyRQ/pq8FYUoY0diOXqW0T6yhgdhCv6wr0hra5DwwWexNRns2Z2+gsnrNcbe9hbGBgk/A==
-  dependencies:
-    "@webassemblyjs/ast" "1.8.5"
-    "@webassemblyjs/helper-module-context" "1.8.5"
-    "@webassemblyjs/wasm-edit" "1.8.5"
-    "@webassemblyjs/wasm-parser" "1.8.5"
-    acorn "^6.2.1"
-    ajv "^6.10.2"
-    ajv-keywords "^3.4.1"
-    chrome-trace-event "^1.0.2"
-    enhanced-resolve "^4.1.0"
-    eslint-scope "^4.0.3"
-    json-parse-better-errors "^1.0.2"
-    loader-runner "^2.4.0"
-    loader-utils "^1.2.3"
-    memory-fs "^0.4.1"
-    micromatch "^3.1.10"
-    mkdirp "^0.5.1"
-    neo-async "^2.6.1"
-    node-libs-browser "^2.2.1"
-    schema-utils "^1.0.0"
-    tapable "^1.1.3"
-    terser-webpack-plugin "^1.4.1"
-    watchpack "^1.6.0"
-    webpack-sources "^1.4.1"
-


Nice. Less webpacks. It looks like those other webpack versions had the same dependencies, so it wasn't too bad, but nice to get it cleaned up

lychyi added 3 commits March 9, 2020 19:59

chore: Remove stale comments address typos

b634934

ci: Use dev env when building PRs

d812656

Merge branch 'master' of https://github.com/Workday/canvas-kit into t…

1974a3b

…ravis-part-deux

ci: Empty commit

586c956

lychyi changed the title ~~ci: Run build-storybook in dev env to avoid caching~~ WIP: ci: Run build-storybook in dev env to avoid caching Mar 10, 2020

ci: Explicitly add node_modules to cache

913bfc1

lychyi changed the title ~~WIP: ci: Run build-storybook in dev env to avoid caching~~ WIP: ci: Travis improvements Mar 10, 2020

lychyi added 11 commits March 10, 2020 00:25

ci: Omit node_modules/.cache from build cache

279558d

ci: Empty commit

341f6a0

chore: Dedupe Webpack

0e2c456

ci: Change to run-p for parallel cmds

40a4102

Reduce that cache size and node_modules folder size

ci: Remove unnecessary node_modules entry in cache

026480c

ci: Fix run-p cmds

dcc4f03

ci: Exit chromatic early to speed up build

f7610df

ci: Cleanup yarn cmd

3f23e3b

ci: Disable node_modules cache

c90bb60

ci: Clean up Travis yml

8e1df02

ci: Forget you cache

6416cb5

lychyi changed the title ~~WIP: ci: Travis improvements~~ ci: Travis build speed improvements Mar 10, 2020

lychyi commented Mar 10, 2020

View reviewed changes

lychyi requested review from anicholls and NicholasBoll and removed request for anicholls March 10, 2020 22:57

lychyi added 2 commits March 10, 2020 20:27

ci: Try caching only npm

138fd3f

ci: Remove caching for node_modules (again)

3217ca8

NicholasBoll reviewed Mar 11, 2020

View reviewed changes

NicholasBoll approved these changes Mar 11, 2020

View reviewed changes

NicholasBoll reviewed Mar 11, 2020

View reviewed changes

NicholasBoll merged commit 3601ebf into Workday:master Mar 11, 2020

lychyi deleted the travis-part-deux branch March 11, 2020 20:20

anicholls mentioned this pull request Mar 11, 2020

chore: Release v3.5.0 #510

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: Travis build speed improvements #504

ci: Travis build speed improvements #504

lychyi commented Mar 10, 2020 •

edited

Loading

cypress bot commented Mar 10, 2020 •

edited

Loading

lychyi Mar 10, 2020

NicholasBoll Mar 11, 2020

lychyi Mar 10, 2020

lychyi Mar 10, 2020

NicholasBoll Mar 11, 2020

NicholasBoll Mar 11, 2020

ci: Travis build speed improvements #504

ci: Travis build speed improvements #504

Conversation

lychyi commented Mar 10, 2020 • edited Loading

cypress bot commented Mar 10, 2020 • edited Loading

Test summary

Run details

lychyi Mar 10, 2020

Choose a reason for hiding this comment

NicholasBoll Mar 11, 2020

Choose a reason for hiding this comment

lychyi Mar 10, 2020

Choose a reason for hiding this comment

lychyi Mar 10, 2020

Choose a reason for hiding this comment

NicholasBoll Mar 11, 2020

Choose a reason for hiding this comment

NicholasBoll Mar 11, 2020

Choose a reason for hiding this comment

lychyi commented Mar 10, 2020 •

edited

Loading

cypress bot commented Mar 10, 2020 •

edited

Loading