Enabling Plugin Cache Directory

[bflad]

Description

We have a use case that requires running Terraform in a GitHub Action across multiple, separate configurations in the same repository. Since Terraform prefers to maintain its own plugin directory per directory, this leads to re-downloading (mostly the same) plugins for each separate directory. It would be great if there was the option to enable the plugin cache directory, which can be handled in the Terraform CLI configuration file via the plugin_cache_dir option, e.g. from the documentation:

plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"

Seems like it could be either a boolean where the GitHub Action maintains the directory location itself (enable_plugin_cache_dir: true) or allow configurations to provide it directly (plugin_cache_dir: /wherever/whenever), which might be useful for managed runners at the expense of being more complex. It seems like in either case, the GitHub Action would should try to create the directory, skipping errors for existing ones, as its prior existence is required.

Potential Workarounds

env:
  TF_PLUGIN_CACHE_DIR: ${{ github.workspace }}/.terraform.d/plugin-cache

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
    # alternatively: run: mkdir -p $TF_PLUGIN_CACHE_DIR
    - uses: actions/cache@v1
      with:
        key: ${{ runner.os }}-terraform-plugin-cache
        path: ${{ env.TF_PLUGIN_CACHE_DIR }}
    - uses: actions/checkout@v2
    - uses: hashicorp/setup-terraform@v1
    - name: terraform
      run: |
        for DIR in $(find ./examples -type d); do
          pushd $DIR
          terraform init
          terraform fmt -check
          terraform validate
          popd
        done

References

• Validate Examples Terraform Configurations via CI Testing terraform-provider-aws#12861

[nantiferov]

Thank you for workaround example, just want to mention that order of cache and checkout should be opposite. In your example checkout will clean cache.

So it should look like:

    - uses: actions/checkout@v2
    - uses: actions/cache@v1
      with:
        key: ${{ runner.os }}-terraform-plugin-cache
        path: ${{ env.TF_PLUGIN_CACHE_DIR }}

[jwenz723]

It is mentioned here that the plugin cache directory will not be created by the terraform cli. So you must ensure the directory exists before trying to cache anything. Here is an example of a workaround which is functioning for me:

env:
  TF_PLUGIN_CACHE_DIR: ${{ github.workspace }}/.terraform.d/plugin-cache

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1.2.1
        with:
          terraform_wrapper: false
          terraform_version: '1.0.0'

      - name: Create Terraform Plugin Cache Dir
        run: mkdir --parents $TF_PLUGIN_CACHE_DIR

      - name: Cache Terraform
        uses: actions/cache@v2
        with:
          path: ${{ env.TF_PLUGIN_CACHE_DIR }}
          key: ${{ runner.os }}-terraform-${{ hashFiles('**/.terraform.lock.hcl') }}

[ctrlok]

I tried to set up caching, but I'm not sure it's working. At least I see no difference in terraform init time with and without caching.

Did you see the difference?

[danopia]

I tried to set up caching, but I'm not sure it's working.

To confirm if caching worked, check the init's output text, it should include "from the shared cache directory".

Snip from uncached provider download:

Initializing provider plugins...
- Reusing previous version of cloudflare/cloudflare from the dependency lock file
- Installing cloudflare/cloudflare v3.7.0...
- Installed cloudflare/cloudflare v3.7.0 (signed by a HashiCorp partner, key ID DE413CEC881C3283)

Snip from successful provider reuse:

Initializing provider plugins...
- Reusing previous version of cloudflare/cloudflare from the dependency lock file
- Using cloudflare/cloudflare v3.7.0 from the shared cache directory

Also, I believe the more-recent Github Actions YAML above will only restore from cache if the lockfile has an exact match

[audunsolemdal]

I've been trying to get this working on a single .lock.hcl file without any luck. From the docs it states that hashFiles() needs the path to be under the github_workspace folder

So
key: ${{ runner.os }}-terraform-${{ hashFiles('**/.terraform.lock.hcl') }}

works

but would it be possible to get something like
key: ${{ runner.os }}-terraform-${{ hashFiles('${{ inputs.workingDir }}/.terraform.lock.hcl') }} working, where workingDir can be "platform/dev" or "platform/test"?

[trallnag]

Hi @audunsolemdal, yes. The following works for me:

- name: Configure Terraform plugin cache
  run: |
    echo 'plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"' > ~/.terraformrc
    mkdir -p ~/.terraform.d/plugin-cache
- name: Cache Terraform
  uses: actions/cache@v3
  with:
    path: ~/.terraform.d/plugin-cache
    key: ${{ runner.os }}-terraform-${{ hashFiles(format('{0}/.terraform.lock.hcl', env.TF_DIR)) }}
    restore-keys: ${{ runner.os }}-terraform-

TF_DIR in my case is envs/dev or envs/prod.

[leonardovillela]

If I understood correctly this issue is about adding the caching capability to the setup-terraform action instead of using the actions/cache. Is that right?

[allanlewis]

setup-python and setup-node have built-in caching functionality, it would be great if setup-terraform could do the same.

[adrianmeraz]

It is mentioned here that the plugin cache directory will not be created by the terraform cli. So you must ensure the directory exists before trying to cache anything. Here is an example of a workaround which is functioning for me:

env:
  TF_PLUGIN_CACHE_DIR: ${{ github.workspace }}/.terraform.d/plugin-cache

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1.2.1
        with:
          terraform_wrapper: false
          terraform_version: '1.0.0'

      - name: Create Terraform Plugin Cache Dir
        run: mkdir --parents $TF_PLUGIN_CACHE_DIR

      - name: Cache Terraform
        uses: actions/cache@v2
        with:
          path: ${{ env.TF_PLUGIN_CACHE_DIR }}
          key: ${{ runner.os }}-terraform-${{ hashFiles('**/.terraform.lock.hcl') }}

I did try this solution, but I got an error on the cleanup for the caching step:

Warning: Path Validation Error: Path(s) specified in the action for caching do(es) not exist, hence no cache is being saved.

I am using a "multiple apps separated by environment" folder organization with s3 remote states.

For example:

tf-repo\apps\app1\environments\dev
tf-repo\apps\app2\environments\prod
tf-repo\apps\app5\environments\dev

Also verified that the TF_PLUGIN_CACHE_DIR var was set to the correct path:

##[debug]..=> 'TF_PLUGIN_CACHE_DIR'
##[debug]=> '/home/runner/work/***/***/.terraform.d/plugin-cache'

[jwadolowski]

Hi @audunsolemdal, yes. The following works for me:

- name: Configure Terraform plugin cache
  run: |
    echo 'plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"' > ~/.terraformrc
    mkdir -p ~/.terraform.d/plugin-cache
- name: Cache Terraform
  uses: actions/cache@v3
  with:
    path: ~/.terraform.d/plugin-cache
    key: ${{ runner.os }}-terraform-${{ hashFiles(format('{0}/.terraform.lock.hcl', env.TF_DIR)) }}
    restore-keys: ${{ runner.os }}-terraform-

TF_DIR in my case is envs/dev or envs/prod.

One small yet important detail regarding above - in order to leverage plugin cache please make sure your .terraform.lock.hcl file includes all the relevant checksums if you happen to have mixed architecture situation (i.e. development on darwin_arm64 Macs + linux_amd64 GH runners).

.terraform.lock.hcl generated on your M1/2/3 machine will include only darwin_arm64 entries which are useless from linux_amd64 runner perspective (architecture mismatch).

Here's what did the trick for me:

• terraform init started on my MacBook produced .terraform.lock.hcl file which got pushed to git
• GH runner (Linux-based) has been re-downloading all the providers despite of the fact ~/.terraform.d/plugin-cache was successfully restored prior to terraform init (triple checked that)
• terraform providers lock -platform=darwin_arm64 -platform=linux_amd64 got executed on my machine and new entries appeared in .terraform.lock.hcl file (as per this comment)
• from now on terraform init works as intended

$ terraform init -lockfile=readonly
...
Initializing provider plugins...
- Reusing previous version of fastly/fastly from the dependency lock file
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of ngrok/ngrok from the dependency lock file
- Using fastly/fastly v5.13.0 from the shared cache directory
- Using hashicorp/aws v5.69.0 from the shared cache directory
- Using ngrok/ngrok v0.3.0 from the shared cache directory

Sample workflow:

jobs:
  plan:
    # ...

    - name: Configure Terraform plugin cache
      run: |
        mkdir -p ~/.terraform.d/plugin-cache

    - name: Cache Terraform plugins
      uses: actions/cache@v4
      with:
        path: ~/.terraform.d/plugin-cache
        key: ${{ runner.os }}-terraform-${{ hashFiles('.terraform.lock.hcl') }}
        restore-keys: ${{ runner.os }}-terraform-

    - name: Terraform Init
      id: init
      run: |
        # Alternative to ~/.terraformrc file
        export TF_PLUGIN_CACHE_DIR="${HOME}/.terraform.d/plugin-cache"
        terraform init -lockfile=readonly