-
Notifications
You must be signed in to change notification settings - Fork 9.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce redundant walks when resolving module depends_on
#35157
Conversation
A walk can start from multiple roots, so making the Ancestors and Descendents methods variadic is the easiest way to add access to that via the API with no changes in existing callers. We can use this from terraform to avoid repeatedly walking overlapping sets of nodes.
The existing implementation of parentModuleDependsOn would walk each reference separately, but if there were many dependencies which were all interconnected, that would end up walking over the same nodes multiple times, and storing many duplicate results. Since a graph walk can start at multiple nodes, we use the extended version of Ancestors to start at all dependencies simultaneously, reducing the time spend traversing the graph as well as automatically removing duplicates.
Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch. |
Amazing! now Terraform faster than ever on large graph!. Thanks |
I had this issue without understand really why If I did a plan or an apply locally, the plan|apply took ~1m30 on a large graph (>2'000 resources) And then I removed by coincidence the more So I was suspecting an issue on Azure Devops runner because everything came back to normal. Anyway, good news because now my plan run in ~20 sec for ~2'500 resources... |
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. |
The use of
depends_on
in modules can cause large numbers of nodes to become connected to all of the same dependencies. When usingAncestors
to walk the graph and find all of these dependencies, there can be a lot of redundant ancestors shared between each of the original nodes, causing an exponential increase in processing time and memory use.Instead of traversing the graph from each dependency individually, which can frequently end up duplicating results, we can start a walk from all nodes at once, reducing the redundant graph traversals and automatically removing duplicates.
Fixes #35154
Target Release
v1.8.4