-
-
Notifications
You must be signed in to change notification settings - Fork 961
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cache terraform configurations used to fetch dependency outputs #2202
Comments
We are experiencing a similar "lag" when running plans on modules with 4-5 dependency blocks, and we're using AWS s3. To mitigate this I had in mind a similar suggestion to @JohannesRudolph More specifically, it would be great if there was an optional TG CLI flag for caching dependency outputs so they're available for subsequent plans. |
I have the exact same issue, terragrunt is taking a minute to run and iterating on it is frustrating 😢 |
@fenos maybe add a 👍 on this issue descr |
@parmouraly have you tried the new CLI flag? https://terragrunt.gruntwork.io/docs/reference/cli-options/#terragrunt-fetch-dependency-output-from-state Since this one was added, we are using it to speed up all of our commands without any issues. And, BTW, as originally discussed in the issue #2119 Terragrunt has cache for dependencies included in the same @JohannesRudolph I think that it might be better to expand this feature to GCS instead. |
@Ido-DY Could you please clarify whether you're suggesting you suspect this new |
@stevenpitts If I got that right, the issue is all about the slowness of Terragrunt mostly when handling dependencies. I actually investigated it myself before opening #2119, and I found out that it actually
The 1st component is generated super fast and there is no point to my mind to cache it. And the last one is cached by default only within the memory of the same run. It takes a few seconds for Terragrunt to run on a single component and most of the time is spent on the Terraform commands themselves, however when resolving dependencies throughout Terraform commands this time usually multiplied. Is this answer your question? @parmouraly have you find this flag useful for your case? |
I think this makes sense, thank you! But doesn't it make sense that generating output of all dependent modules would take longer than fetching the output from state, since it involves many provider API calls, rather than a single call to the authoritative state file (local or remote)? Thanks for the clarification! |
Thanks everyone who joined in on the discussion so far. I'll summarize my understanding of the issue and will try to clarify my original enhancement suggestion. The key problem is that when Since this is slow, various forms of caching have been proposed here. Any approach needs to maintain correctness though. The solutions and their respective limitations are
@Ido-DY suggested extending 1) to GCS. While that's possible, my next feature request would be to extend that to Azure as well as we have some state backends there. I expect sooner or later there will be further requests to add first-party support in terragrunt for more state backends. Another variant is extending 4) to persist the cache between different terragrunt run-all configurations and somehow maintain correctness. My personal favorite would be 3) as this builds on terragrunt behavior that is already there i.e. maintaining the initialized terraform configuration inside a |
I noticed one of my configurations being very slow to actually start running
terraform apply
. After inspecting terragrunt's debug output, I traced it back to a call toterraform init -get=false
of a dependency configuration taking up most of the init time. I noticed on my system activity monitor that this command was pulling down quite a few MiBs from the internet (assumingly terraform backend plugins?) and this was slowing down everything.I then looked for why terragrunt was doing this and discovered this recent discussion -
#2119 introduced new behavior to optimize fetching the state of dependency blocks by bypassing terraform alltogether and fetching directly from an S3 bucket.
However, I'm using GCS backend and that optimization isn't implemented for GCS yet. The issue also discusses the challenges of that optimization and the robust fallback behavior.
Now, based on this comment:
Originally posted by @yorinasub17 in #2119 (comment)
I wonder if not a "naive" but more robust implementation could (persistently) cache the terraform configuration that's used to get the
terraform output
of the dependency configuration? This way terragrunt would still useterraform output
to fetch the dependency but can avoid expensiveterraform init
calls on that dependency configuration.This optimization could easily shave of 10s and up to 60s of "init" time from some of my more complex terragrunt configurations and yield a huge boost for interactive development.
The text was updated successfully, but these errors were encountered: