[rush] Specify environment variables for build cache

[chengcyber]

Summary

This is a feature request for build cache: support specifying a list of environment variable name, and take them into consideration when calculating cache id.

Details

This proposal wants to add a configuration to specify a list of environment variable name, and use them when calculating cache id.

Use case

Running CDN_PREFIX=/a/b rush build --to <app>, it writes cache archive with cache id <cache_id_a>(for example)

Then, running CDN_PREFIX=/c/d rush build --to <app>, it calculates cache id to <cache_id_a> which is same as calculated in the first command, and restores the cache archive, which is wrong.

With specifying "CDN_PREFIX" as one of the environment variables, the cache id of second command will be <cache_id_b>, and the second command can get its own cache archive.

Possible solutions

1. Specify it in "common/config/rush/build-cache.json"

{
  "cachingEnvVariables": [
    "CDN_PREFIX"   <-- here
  ]
}

2. Specify it in "/config/rush-project.json"

{
  "operationSettings": [
    {
      "operationName": "build",
      "outputFolderNames": [
        "dist"
      ],
      "cachingEnvVariables": [
        "CDN_PREFIX"   <-- here
      ]
    }
  ]
}

3. Specify it in "common/config/rush/command-line.json"

{
  "commands": [
    {
       "name": "build",
       // ...
       "cachingEnvVariables": [
          "CDN_PREFIX"  <-- here
       ]
    }
  ]
}

Standard questions

Please answer these questions to help us investigate your issue more quickly:

Question	Answer
@microsoft/rush globally installed version?	latest
rushVersion from rush.json?	latest
useWorkspaces from rush.json?	true
Operating system?	Mac
Would you consider contributing a PR?	Yes
Node.js version (node -v)?	14

@elliot-nelson @dmichon-msft

[octogonz]

May be related to this PR: #3696

[octogonz]

My vote is for option 2, since the behavior of the environment variable is really determined by a toolchain and/or rig. For example, if we relocate a Rush project into a different monorepo, with different custom-commands.json, the cachingEnvVariables state should come along with it.

[dmichon-msft]

Option 2 is the only option that scales as we expand to more phases in repositories. As much as possible we should be focusing anything to do with per-operation configuration into rush-project.json. Ideally I would like to move the phase dependency relationships over to there as well, so that they can be customized depending on the needs of the project.

My PR #3696 retools the way cache keys are generated so that they are based on the structure of the operation graph, in that each operation's cache key is a function of its local state (files, environment variables, CLI parameters, and anything else the operation runner wants to include), and the cache keys of its operation dependencies. This ensures that if a dependency is impacted by an environment variable or other state value, it will impact the cache keys of its consumers, as well.

In the existing computation of cache keys, CLI parameters are only included in the cache keys of the phases to which they directly apply. For example, if you have phases defined like in the rushstack repository, where:
rush test runs _phase:build and _phase:test
_phase:build depends on upstream _phase:build
_phase:test depends on self _phase:build

Define a command line parameter --build-param that only applies to _phase:build.

Then the cache keys for _phase:test on rush test --build-param and rush test will match, despite having potentially different inputs via the _phase:build execution.

[elliot-nelson]

Also agree the settings go in rush-project.json; if this is part of a larger change to cache key calculation that sounds good to me.

[elliot-nelson]

I wanted to also mention in this ticket, for people that need this functionality right now, what work-arounds exist.

If your only concern is CI builds, and you're less concerned about the accuracy of local developer builds, then I think there's two reasonable options:

---

1. Hack cacheEntryNamePattern

In a simple node script before you run the rush build, open up common/config/rush/build-cache.json and find the cacheEntryNamePattern line, and override it to include more stuff.

For example:

    lines[i] = `"cacheEntryNamePattern": "${process.env.FOO}-${process.env.BAR}-[projectName:normalize]-[phaseName:normalize]-[hash]"`;

With this method, project hashes don't change, but for the entire run you'll be selecting a different "build cache namespace" from the cloud build cache.

2. Hack project contents

In a simple node or bash script before you run the rush build, write out "token" files to specific projects.

For example:

echo "${FOO}-${BAR}" > apps/my-app/TOKEN
echo "${FOO}-${BAR}" > tools/special-tool/TOKEN

With this method, you're selecting specific projects and modifying their project hashes with data inserted into a new (untracked) file.

---

Neither of those approaches are very useful for local developers, unfortunately, because they rely on temporarily changing current git state which would leave artifacts everywhere for them to deal with.