New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI/CD Best Practice for plan-all output? #720
Comments
Given that, when you give an absolute path for the plan output, each This use case that you are describing (having a single plan output from There are a few reasons for this: plan-all output is broken for certain use casesIf you have interdependencies between the modules, then the
In this scenario, when you run Now suppose that nothing is deployed. In this case, the latter What about if we had already deployed resources? In this case the However, if the So in practice, We have discussed in the past of trying to address this concern. You can see #262 for the relevant thread. If you are always running
|
I really appreciate the extremely thorough writeup! You've made some awesome points. You're right that most CI/CD doesn't have manual triggering as a first class citizen. We have the capability, but it seems that there are plenty of other reasons why we should avoid this approach for now. Again, thanks for the top to bottom explanation! |
Here’s a Terraform module that detects all changes within Terragrunt dependency chains and orchestrates a continuous deployment flow that respects the dependeny chain order. The module offers AWS account-level dependencies and approval requirements. In addition, it includes handling for rolling back newly deployed Terraform provider resources. See the README.md for the full description of how this all works here. Side note: I’m the owner of this repository. If you do check it out, I'm open to any questions and brutally honest feedback! |
One option to get around this in CI, at least Github Actions, is to not use jobs:
build_matrix:
name: Build Matix
runs-on: [self-hosted, linux]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Get all directories with terragrunt.hcl
uses: sergeysova/jq-action@v2
id: find
with:
cmd: for file in $(find * -mindepth 1 -name terragrunt.hcl); do dirname $file; done | jq -R -s -c 'split("\n")[:-1]' | sed 's/"/\\"/g'
- id: set-matrix
run: echo ::set-output name=matrix::${{ steps.find.outputs.value }}
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
run:
name: Run Terragrunt
runs-on: [self-hosted, linux]
needs: build_matrix
strategy:
fail-fast: false
matrix:
dir: ${{ fromJson(needs.build_matrix.outputs.matrix) }}
steps:
- name: Checkout
uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '14'
- uses: hashicorp/setup-terraform@v1
with:
terraform_wrapper: false # needed for terragrunt to be able to parse TF output
- name: Setup Terragrunt
uses: autero1/action-terragrunt@v1.1.1
with:
terragrunt_version: "latest"
- name: Run terragrunt
run: |
cd ${{ matrix.dir }}
terragrunt plan --terragrunt-ignore-external-dependencies -detailed-exitcode -lock=false -parallelism=30 Hope this is useful to someone else. |
@norman-zon, won't this lead to issues if one stack depends on another one? |
I'm starting to place our infrastructure into a CI/CD pipeline with steps for
validate -> plan (with planfile output) -> apply
.At first I thought I could simply run
terragrunt plan-all -out "planfile"
in one step, save some plan file, and then runterragrunt apply-all -input=false "planfile"
in another step. However, it seems that terragrunt actually places these plan files into the individual module.terragrunt-cache
directories.Next I provided an absolute path like
terragrunt plan-all -out /my/plan/file
which seems to generate the single file I'm after.However, on the next step of trying to apply, since it's a single plan I expect I should use
terragrunt apply /my/plan/file
as opposed to theapply-all
syntax. Doing this actually fails:I'm not sure if I should actually be letting the individual modules deal with plan files or trying to consolidate into this single file.
Is there a best practice for how to handle this situation?
The text was updated successfully, but these errors were encountered: