Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: combine "define_env_vars" and "download_yarn" anchor #28788

Closed
wants to merge 1 commit into from
Closed

ci: combine "define_env_vars" and "download_yarn" anchor #28788

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
84 changes: 29 additions & 55 deletions .circleci/config.yml
View file
Open in desktop
Expand Up @@ -28,11 +28,14 @@ var_2: &browsers_docker_image circleci/node:10.12-browsers
# See https://circleci.com/docs/2.0/caching/#restoring-cache for how prefixes work in CircleCI.
var_3: &cache_key v2-angular-node-10.12-{{ checksum "yarn.lock" }}-{{ checksum "WORKSPACE" }}-{{ checksum "packages/bazel/package.bzl" }}

# Define common ENV vars
var_4: &define_env_vars
# Initializes the CI environment by setting up common environment variables and
# downloading Yarn. Yarn is used by most jobs and therefore is part of this anchor.
var_4: &init_environment
run:
name: Define environment variables
command: ./.circleci/env.sh
name: Initializing environment (setting up variables, downloading Yarn)
command: |
source ./.circleci/env.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need to source it now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because we need the yarn version variable available in the current bash shell. The command after the env.sh script depends on it.

Not sure if we want to add a comment for this. Should be quite clear IMO 😁

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that PROFILE=$BASH_ENV was sufficient. Fair enough then 😏

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😄 Yeah that PROFILE variable is just used by the Yarn install script to update the $PATH.

Copy link
Contributor

@alexeagle alexeagle Feb 18, 2019 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've gone back and forth between installing tooling during the run (slower) or building the right environment into the docker image (more maintenance)
I feel like the latter is the more principled thing but maybe not practical, up to you

With more work, we ought to be able to have both:

  • change the Dockerfile, the CI picks that up and builds a new docker image for this build
  • don't need to do any environment setup on each execution

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'm personally a bigger fan of preparing a proper docker image. We should have a discussion about this in a follow-up as this should be something we need to figure out long-term.

curl -o- -L https://yarnpkg.com/install.sh | PROFILE=$BASH_ENV bash -s -- --version "$CI_YARN_VERSION"

var_5: &setup_bazel_remote_execution
run:
Expand Down Expand Up @@ -70,16 +73,7 @@ var_9: &setup_circleci_bazel_config
name: Setting up CircleCI bazel configuration
command: sudo cp .circleci/bazel.rc /etc/bazel.bazelrc

# Sets up Yarn by downloading it and installing it globally. We don't use Yarn from the
# docker image because this means that we can only use the Yarn version that comes with
# a specific version of NodeJS. We want to be able to update Yarn without having to
# update the docker image that comes with NodeJS.
var_10: &download_yarn
run:
name: Downloading Yarn
command: curl -o- -L https://yarnpkg.com/install.sh | PROFILE=$BASH_ENV bash -s -- --version "$CI_YARN_VERSION"

var_11: &restore_cache
var_10: &restore_cache
restore_cache:
keys:
- *cache_key
Expand All @@ -94,8 +88,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- *yarn_install

- run: 'yarn bazel:format -mode=check ||
Expand All @@ -113,8 +106,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- *yarn_install
- *setup_circleci_bazel_config

Expand All @@ -133,8 +125,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- *yarn_install
- *setup_circleci_bazel_config
- *setup_bazel_remote_execution
Expand Down Expand Up @@ -172,8 +163,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
# Build aio
- run: yarn --cwd aio build --progress=false
# Lint the code
Expand All @@ -200,8 +190,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
# Deploy angular.io to production (if necessary)
- run: setPublicVar CI_STABLE_BRANCH "$(npm info @angular/core dist-tags.latest | sed -r 's/^\s*([0-9]+\.[0-9]+)\.[0-9]+.*$/\1.x/')"
- run: yarn --cwd aio deploy-production
Expand All @@ -217,8 +206,7 @@ jobs:
- *restore_cache
- attach_workspace:
at: dist
- *define_env_vars
- *download_yarn
- *init_environment
# Build aio (with local Angular packages)
- run: yarn --cwd aio build-local --progress=false
# Run PWA-score tests
Expand All @@ -237,8 +225,7 @@ jobs:
- *restore_cache
- attach_workspace:
at: dist
- *define_env_vars
- *download_yarn
- *init_environment
# Build aio with Ivy (using local Angular packages)
- run: yarn --cwd aio build-with-ivy --progress=false

Expand All @@ -250,8 +237,7 @@ jobs:
- *restore_cache
- attach_workspace:
at: dist
- *define_env_vars
- *download_yarn
- *init_environment
# Install
- run: yarn --cwd aio install --frozen-lockfile --non-interactive
- run: yarn --cwd aio extract-cli-command-docs
Expand All @@ -271,8 +257,7 @@ jobs:
- *restore_cache
- attach_workspace:
at: dist
- *define_env_vars
- *download_yarn
- *init_environment
# Install aio
- run: yarn --cwd aio install --frozen-lockfile --non-interactive
# Run examples tests. The "CIRCLE_NODE_INDEX" will be set if "parallelism" is enabled.
Expand All @@ -292,8 +277,7 @@ jobs:
- *restore_cache
- attach_workspace:
at: dist
- *define_env_vars
- *download_yarn
- *init_environment
# Install aio
- run: yarn --cwd aio install --frozen-lockfile --non-interactive
# Run examples tests with ivy. The "CIRCLE_NODE_INDEX" will be set if "parallelism" is enabled.
Expand All @@ -310,8 +294,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- run: ./aio/scripts/build-artifacts.sh $AIO_SNAPSHOT_ARTIFACT_PATH $CI_PULL_REQUEST $CI_COMMIT
- store_artifacts:
path: *aio_preview_artifact_path
Expand All @@ -330,8 +313,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- run: yarn --cwd aio install --frozen-lockfile --non-interactive
- run:
name: Wait for preview and run tests
Expand All @@ -353,8 +335,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- *yarn_install
- *setup_circleci_bazel_config
- *setup_bazel_remote_execution
Expand Down Expand Up @@ -383,8 +364,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- *yarn_install
- *setup_circleci_bazel_config
- *setup_bazel_remote_execution
Expand Down Expand Up @@ -420,8 +400,7 @@ jobs:
- *restore_cache
- attach_workspace:
at: dist
- *define_env_vars
- *download_yarn
- *init_environment
# Some integration tests get their dependencies from the root `node_modules/`.
- *yarn_install
# Runs the integration tests in parallel across multiple CircleCI container instances. The
Expand All @@ -435,7 +414,7 @@ jobs:
steps:
- checkout:
<<: *post_checkout
- *define_env_vars
- *init_environment
# See below - ideally this job should not trigger for non-upstream builds.
# But since it does, we have to check this condition.
- run:
Expand Down Expand Up @@ -467,8 +446,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- run:
name: Run tests against the deployed apps
command: ./aio/scripts/test-production.sh $CI_AIO_MIN_PWA_SCORE
Expand All @@ -487,8 +465,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- *yarn_install
- run: yarn tsc -p packages
- run: yarn tsc -p modules
Expand All @@ -504,8 +481,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- *yarn_install
- run:
name: Preparing environment for running tests on Saucelabs.
Expand All @@ -530,8 +506,7 @@ jobs:
- checkout:
<<: *post_checkout
- *restore_cache
- *define_env_vars
- *download_yarn
- *init_environment
- *yarn_install
- attach_workspace:
at: dist
Expand All @@ -550,8 +525,7 @@ jobs:
steps:
- checkout:
<<: *post_checkout
- *define_env_vars
- *download_yarn
- *init_environment
- attach_workspace:
at: dist
- run: ./scripts/ci/run_angular_material_unit_tests.sh
Expand Down
7 changes: 4 additions & 3 deletions .circleci/env.sh
View file
Open in desktop
@@ -1,8 +1,9 @@
#!/usr/bin/env bash

# Variables
readonly envHelpersPath="`dirname $0`/env-helpers.inc.sh";
readonly getCommitRangePath="`dirname $0`/get-commit-range.js";
readonly projectDir=$(realpath "$(dirname ${BASH_SOURCE[0]})/..")
readonly envHelpersPath="$projectDir/.circleci/env-helpers.inc.sh";
readonly getCommitRangePath="$projectDir/.circleci/get-commit-range.js";

# Load helpers and make them available everywhere (through `$BASH_ENV`).
source $envHelpersPath;
Expand All @@ -14,7 +15,7 @@ echo "source $envHelpersPath;" >> $BASH_ENV;
####################################################################################################
# See https://circleci.com/docs/2.0/env-vars/#built-in-environment-variables for more info.
####################################################################################################
setPublicVar PROJECT_ROOT "$(pwd)";
setPublicVar PROJECT_ROOT "$projectDir";
setPublicVar CI_AIO_MIN_PWA_SCORE "95";
# This is the branch being built; e.g. `pull/12345` for PR builds.
setPublicVar CI_BRANCH "$CIRCLE_BRANCH";
Expand Down