add production build setup and aws CICD config files #1087

sharkySharks · 2019-03-29T17:09:14Z

Description

Adding production build scripts, AWS cicd config files, and pr check scripts for standing up CICD system in AWS.

Related Issue

Verification Steps

See CircleCI configuration removed and only test checks performed by AWS in PR #1132

vercel · 2019-03-29T17:09:19Z

This pull request is automatically deployed with Now.
To access deployments, click Details below or on the icon next to each push.

coveralls · 2019-03-29T19:07:09Z

Coverage remained the same at 76.822% when pulling 7acdae1 on aws-cicd into f9947f2 on develop.

sharkySharks

Video review with Ron Oribio from the DevOps Platform team

sharkySharks · 2019-04-02T20:11:58Z

.codebuild/buildspec.deploy.demo.yml

+    commands:
+      - echo build Docker image on `date`
+      - docker build -f Dockerfile.prod -t pwa-demo:latest .
+      - docker tag pwa-demo:latest 276375911640.dkr.ecr.us-east-1.amazonaws.com/pwa-demo


PR review Ron: add second tag with a version - deploy latest still - use semantic versioning (version file) to keep track of which version is deployed and which was the last deployed.

sharkySharks · 2019-04-02T20:13:06Z

.codebuild/buildspec.pr.build.yml

@@ -0,0 +1,26 @@
+version: 0.2


add lifecycle policy for each ECR repo

sharkySharks · 2019-04-02T20:14:46Z

Dockerrun.aws.json

+{
+  "AWSEBDockerrunVersion": "1",
+  "Image": {
+    "Name": "276375911640.dkr.ecr.us-east-1.amazonaws.com/pwa-demo:latest",


this tag will need to update based on semantic version tagging

this is modified in each build script to be either the pr id tag or the commit SHA ✅

sharkySharks · 2019-04-02T20:34:05Z

dangerfile.js

-            if (typeof stdout !== 'string') {
-                // execa didn't require
+            if (err.code === 'MODULE_NOT_FOUND') {
+                // execa din't require


sharkySharks · 2019-04-02T20:36:56Z

docker/.env.docker.prod

+PORT=8080
+PWA_STUDIO_HOST=localhost
+# magento enterprise edition - in production mode
+MAGENTO_BACKEND_URL=https://m231-pwa-ent-1.testsonfire.com/


pull in from environment variables rather than plain text

sharkySharks · 2019-04-02T20:37:11Z

docker/.env.docker.prod

+UPWARD_JS_UPWARD_PATH=venia-upward.yml
+UPWARD_JS_BIND_LOCAL=1
+UPWARD_JS_LOG_URL=1
+BRAINTREE_TOKEN=sandbox_8yrzsvtm_s2bg8fs563crhqzk


token should be pulled in from env vars rather than plain text and other files as well.

packages/venia-concept/package.json

@@ -13,7 +13,7 @@
    "venia": "./bin/venia.js"
  },
  "scripts": {
-    "build": "yarn run clean && yarn run build:esm && yarn run validate-queries && yarn run build:prod",


ericerway · 2019-04-08T20:54:58Z

Assigning @jimbo for review.

…emo images

jimbo

Looks good to me, so far. 👍

I've left a few questions, but nothing that needs to be addressed.

jimbo · 2019-04-16T22:38:50Z

package.json

@@ -19,6 +19,7 @@
  "scripts": {
    "build": "yarn workspaces run build",
    "build:dev": "yarn workspaces run build:dev",
+    "ci:pr-checks": "./.bin/pr-checks",


Would yarn run pr-checks work here?

I believe it would need to be part of node_modules for it to be accessible directly vs putting it under scripts.

The only ones I see for execution come up when I run yarn run:

@sharky.corp.adobe.com ➜ pwa-studio git:(aws-cicd) yarn run yarn run v1.15.2 info Commands available from binary scripts: JSONStream, acorn, ansi-html, apollo-codegen, atob, autoprefixer, babel, babel-external-helpers, babylon, browserslist, build-storybook, bundlesize, bundlesize-init, bundlesize-pipe, concurrently, conventional-changelog-writer, conventional-commits-parser, conventional-recommended-bump, coveralls, cssesc, danger, danger-ci, danger-init, danger-js, danger-local, danger-pr, danger-process, danger-reset-status, danger-runner, detect, detect-libc, detect-port, disparity, errno, escodegen, esgenerate, eslint, esparse, esvalidate, get-pkg-repo, git-raw-commits, git-semver-tags, gql, graphql, graphql-schema-linter, handlebars, he, html-minifier, hulk, husky-upgrade, import-local-fixture, is-ci, jest, jest-runtime, js-yaml, jsesc, jsome, json5, lerna, loose-envify, miller-rabin, mime, multicast-dns, nearley-railroad, nearley-test, nearley-unparse, nearleyc, needle, node-gyp, node-pre-gyp, nopt, npm-path, npm-run, npm-tree, npm-which, opener, parse-github-url, parse-nodejs-lockfile, parser, pbjs, pbts, pjv, prebuild-install, prettier, prettier-check, rc, react-docgen, regexp-tree, regjsparser, rimraf, run-node, sane, semver, sha.js, shjs, sl-log-transformer, snyk, sshpk-conv, sshpk-sign, sshpk-verify, start-storybook, storybook-server, strip-indent, svgo, swagger-to-graphql, tap-diff, tap-parser, tap-xunit, tape, terser, tree-kill, ts-node, txunit, uglifyjs, upward-js-server, upward-spec, uuid, watch, webpack, webpack-bundle-analyzer, webpack-cli, webpack-dev-server, which, window-size, z-schema, zopfli, zopflipng

jimbo · 2019-04-16T23:31:23Z

dangerfile.js

+        }
+    }
+    return errors;
+};


It's not like performance in CI is super important, but could these tasks be run in parallel?

I think that if Danger is going to be used more and expanded then this file should be refactored in a number of ways. I think that should be a separate endeavor outside of this pr though. Mostly what I am doing here is capturing all the errors at once and reporting all the errors, rather than only reporting the first error hit.

jimbo · 2019-04-16T23:52:53Z

Dockerfile.prod

+RUN yarn install --frozen-lockfile
+
+# copy over the rest of the package files
+COPY packages ./packages


Can you elaborate on why we can't just copy packages and run yarn install?

Copying over just the files needed to install takes advantage of the docker cache, which allows for faster builds when there isn't a change to an image layer, each command in the dockerfile creates a new image layer as it builds. So if no changes are made to the dependencies then those layers will build from cache on subsequent builds. after hte install we copy over the rest of the files and if there are changes to those files then it will rebuild otherwise it will build from cache. More deets on the docs

* add production build setup and aws CICD config files

sharkySharks force-pushed the aws-cicd branch from 49d1216 to 2d07319 Compare March 29, 2019 18:36

vercel bot deployed to staging March 29, 2019 18:36 View deployment

sharkySharks changed the title ~~add initial buildspec.yml file for codebuild~~ add production build setup and aws CICD config files Mar 29, 2019

This was referenced Mar 29, 2019

Create Dockerfile for production build of PWA #940

Closed

Production deploy instance of Venia for demo purposes #1034

Closed

Deploy pipeline for demo environment #1011

Closed

Connect Magento backend urls to demo deploy pipeline #1012

Closed

sharkySharks force-pushed the aws-cicd branch from 2d07319 to 3853363 Compare April 1, 2019 14:33

vercel bot deployed to staging April 1, 2019 14:33 View deployment

sharkySharks force-pushed the aws-cicd branch from 3853363 to b8bf496 Compare April 1, 2019 16:54

vercel bot deployed to staging April 1, 2019 16:54 View deployment

sharkySharks commented Apr 2, 2019

View reviewed changes

add production build setup and aws CICD config files

c3da586

sharkySharks force-pushed the aws-cicd branch from b8bf496 to 4c19fdf Compare April 3, 2019 22:02

vercel bot deployed to staging April 3, 2019 22:03 View deployment

sharkySharks mentioned this pull request Apr 4, 2019

Need deployment environment with shared credential manager and AWS access control group for team #587

Closed

sharkySharks force-pushed the aws-cicd branch from 4c19fdf to 86b3fee Compare April 5, 2019 16:08

vercel bot deployed to staging April 5, 2019 16:08 View deployment

sharkySharks force-pushed the aws-cicd branch from 86b3fee to 72c295a Compare April 5, 2019 20:42

vercel bot deployed to staging April 5, 2019 20:42 View deployment

ericerway assigned jimbo Apr 8, 2019

sharkySharks force-pushed the aws-cicd branch from 72c295a to 062681d Compare April 9, 2019 19:00

vercel bot deployed to staging April 9, 2019 19:00 View deployment

vercel bot temporarily deployed to staging April 9, 2019 20:36 Inactive

sharkySharks force-pushed the aws-cicd branch from c36d58e to c424abb Compare April 9, 2019 20:41

vercel bot requested a deployment to staging April 9, 2019 20:41 Abandoned

sharkySharks force-pushed the aws-cicd branch from c424abb to 68d3929 Compare April 9, 2019 21:04

sharkySharks force-pushed the aws-cicd branch from 89f592f to ceb6021 Compare April 10, 2019 18:09

vercel bot deployed to staging April 10, 2019 18:09 View deployment

sharkySharks force-pushed the aws-cicd branch from ceb6021 to 1bde146 Compare April 10, 2019 18:21

vercel bot deployed to staging April 10, 2019 18:21 View deployment

sharkySharks force-pushed the aws-cicd branch from 1bde146 to 5ae01f1 Compare April 10, 2019 18:30

vercel bot deployed to staging April 10, 2019 18:30 View deployment

sharkySharks mentioned this pull request Apr 10, 2019

Migrate off circleci #1132

Merged

5 tasks

sharkySharks force-pushed the aws-cicd branch from 5ae01f1 to 07e51ba Compare April 10, 2019 19:05

vercel bot deployed to staging April 10, 2019 19:05 View deployment

sharkySharks force-pushed the aws-cicd branch from 07e51ba to c526fe2 Compare April 12, 2019 16:30

vercel bot deployed to staging April 12, 2019 16:30 View deployment

sharkySharks force-pushed the aws-cicd branch from c526fe2 to bc00e05 Compare April 12, 2019 16:49

vercel bot deployed to staging April 12, 2019 16:50 View deployment

set up parameter store for sensitive env vars and add versioning to d…

07e7217

…emo images

sharkySharks force-pushed the aws-cicd branch from bc00e05 to 07e7217 Compare April 12, 2019 16:57

vercel bot deployed to staging April 12, 2019 16:57 View deployment

Merge branch 'develop' into aws-cicd

3f39225

vercel bot deployed to staging April 16, 2019 20:36 View deployment

jimbo reviewed Apr 16, 2019

View reviewed changes

dpatil-magento assigned dpatil-magento and unassigned jimbo Apr 17, 2019

jimbo approved these changes Apr 17, 2019

View reviewed changes

Merge branch 'develop' into aws-cicd

7acdae1

vercel bot deployed to staging April 17, 2019 21:07 View deployment

dpatil-magento merged commit 29317f2 into develop Apr 17, 2019

sharkySharks deleted the aws-cicd branch April 18, 2019 15:46

dpatil-magento pushed a commit that referenced this pull request Apr 18, 2019

add production build setup and aws CICD config files (#1087)

9181482

* add production build setup and aws CICD config files

dpatil-magento pushed a commit that referenced this pull request Apr 18, 2019

add production build setup and aws CICD config files (#1087)

44c0bee

* add production build setup and aws CICD config files

cherdman added this to the Next PWA Release (Core 2.3.2) milestone Apr 29, 2019

sirugh added the tooling Related to the developer tooling, including buildpack, webpack plugins, and debug UIs. label May 24, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add production build setup and aws CICD config files #1087

add production build setup and aws CICD config files #1087

sharkySharks commented Mar 29, 2019 •

edited

vercel bot commented Mar 29, 2019 •

edited

coveralls commented Mar 29, 2019 •

edited

sharkySharks left a comment

sharkySharks Apr 2, 2019

sharkySharks Apr 2, 2019

sharkySharks Apr 3, 2019

sharkySharks Apr 2, 2019

sharkySharks Apr 3, 2019 •

edited

sharkySharks Apr 2, 2019

sharkySharks Apr 2, 2019

sharkySharks Apr 2, 2019

This comment was marked as resolved.

ericerway commented Apr 8, 2019

jimbo left a comment

jimbo Apr 16, 2019

sharkySharks Apr 17, 2019

jimbo Apr 16, 2019

sharkySharks Apr 17, 2019

jimbo Apr 16, 2019

sharkySharks Apr 17, 2019

add production build setup and aws CICD config files #1087

add production build setup and aws CICD config files #1087

Conversation

sharkySharks commented Mar 29, 2019 • edited

Description

Related Issue

Verification Steps

vercel bot commented Mar 29, 2019 • edited

coveralls commented Mar 29, 2019 • edited

sharkySharks left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sharkySharks Apr 3, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

ericerway commented Apr 8, 2019

jimbo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sharkySharks commented Mar 29, 2019 •

edited

vercel bot commented Mar 29, 2019 •

edited

coveralls commented Mar 29, 2019 •

edited

sharkySharks Apr 3, 2019 •

edited