-
-
Notifications
You must be signed in to change notification settings - Fork 995
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Terraform 0.12 map merging #744
Comments
Hm, I don't think I'd want to merge maps by default. I think that would be unexpected and is one of the reasons Terraform itself stopped doing it. However, if there was an explicit way to opt into that sort of behavior, it would make sense. For example, in a few issues, the idea of adding a inputs = {
foo = "bar"
baz = get_input("../common.hcl", "baz")
} That allows you to explicitly reuse a single input from another inputs = {
foo = "bar"
baz = merge(get_input("../common.hcl", "baz"), { "foo": "bar" })
} This is verbose, but clear and explicit... It would work if you only have a small number of such variables, but might not be effective if you have dozens. |
As of v0.19.4, |
I've been trying to figure out the best way to do this, and I'm just not clear where terraform built-in functions can be run. The end goal in my scenario is specifically merge tags where I could have account level tags (or any arbitrary directory level) that would be applied to all sub-directories and merged into a single tag map to be applied to all resources. The documentation under locals (https://github.com/gruntwork-io/terragrunt#locals) seems to indicate the only way to do this is by adding something like the below which works. The only issue is that now we have to use another configuration yaml configuration file throughout the directory structure, and the yaml files only need to be used for this type of map merge scenario which adds a bit of overhead/confusion.
Is there a simpler/more explicit method to merge maps for this type of scenario that I am just missing? Thanks |
It seems like you are looking for a better way to reference common variables. It would help to know what would be the ideal setup for you, since there are a few different ideas that we've floated around: Implementing Implementing This provides an alternative to Updating In this setup, you can load the common vars from a base, reference terragrunt config in your setup. This is an alternative to
|
@yorinasub17 - Appreciate the quick response. I think the cleanest solution would at least be able to have all the variable data within a single file type (so maybe just the hcl files), and being able to pull/merge from parent hcl files. Having the ability to pull in tfvars/yaml/json files does allow for a lot of flexibility, but makes it a bit more complicated to track variables through a directory structure. Ignore the below example, and see the edit at the bottom. For example: root level account.hcl
mid-level app.hcl file
leaf level resource terragrunt.hcl
I think the get_input/hcldecode helper functions might accomplish this. I think using the dependency functionality might make some things more complicated though I'd have to do some testing around it to see if it would work. Thanks Edit: Actually in thinking about it some more, I don't think my above solution would be ideal. I think it would be preferable to explicitly pull in a specific parent directory file. I think the following would be better.
Would the hcldecode be able to parse any block from the account.hcl file. It seems like get_input would only be able to look at the inputs block, so I'm not sure what additional functionality would be needed from the hcldecode. |
The idea of
I know you realized this wasn't ideal, but FWIW, we discussed a similar construct called |
Thanks for the link. That is definitely similar to what I was thinking. In the end any solution that allows for eplicit merging of variables/maps, and only using hcl files would be ideal. |
Very interested in this as well as; it looks like #858 will likely address this sort of functionality? |
As a matter of providing a use case where something like this is required, here is my example. My Terraform deploys an Azure Function which takes a For my simple example, I have the following directory structure.
The The ability to opt-in/opt-out to a recursive merge would be much appreciated. This has broken my currently workflow as it has for @justicel as it was quite elegant to have it recursively merge before; it just worked. It also appears there is a related ticket opened here hashicorp/terraform#16517 If anything, a If anyone needs a temporary workaround, this is how I solved it.
|
I wrote up an RFC that introduces |
@yorinasub17 I skimmed through your proposal, I will go through it more in-depth when I have time, but I don't see it solving this use case. Maybe I just missed it, but I actually think this is more cumbersome and verbose rather than DRY. I don't believe I would use this proposal over my current configuration and still would like a way to merge maps. It still appears as you have to merge the inputs manually and would be required to pick out the maps as I do in my example above and in this post. Below I've posted a real root .\tf\terragrunt.hcl
.\tf\apps\function-apps\qa\order-processing\terragrunt.hcl
|
Thanks for sharing your use case! There is a lot going on there and I didn't have a whole lot of time to deep dive into your config, but I think what I pieced out in terms of feature list was:
The first one makes sense, and I had actually planned The second one only makes sense if we support the third bullet point, although I am not sure I want to go that route. It certainly does make the child config more terse, but at the expense of making it harder to parse. And I think you can still go pretty far in terms of terseness with just deep merges and imports. Of course, this is edging towards flame war territory as you mention (convention vs configuration). Given that, if you were to implement your original simpler use case with the imports feature, it would be as follows:
app/terragrunt.hcl
app/qa/terragrunt.hcl
app/qa/services/terragrunt.hcl
This pushes some of the complexity around merges and imports to the middle configuration. So yes, there is going to be some repetition in the middle layers around the import logic and I can understand if you disagree with the design principles here. FWIW, I think the missing feature here to get what you want is |
@yorinasub17 not trying to start a flame war over convention vs configuration! Trying to provide as much feedback as possible ;).
As far as my use case, there would be too much repetition in my opinion to spread out all of the importing logic which I have consolidated in the root file once for the entire directory. Although, if I didn't have multiple applications in one repository, I do believe I would prefer proposal because I wouldn't have so many leaf nodes, but would rather have a pretty straight tree instead of a wide bottom. I bundle my Terraform with my application code in the same repository. Most of my repositories only have one application deployment and one logical Terraform deployment (multiple environments/regions), and my approach is overly verbose for this and would be simplified by your proposal. 👍 |
Yup agreed that this should be straightforward once the deep merge functionality is implemented for imports.
Yup I figured that from the full use case. Do you think this would change if we implemented
From there, the child config just needs to import the root config to get all the imports. Not promising that we would implement this (still an open question: would like to see some more use cases that would suggest this need), but certainly would be a data point in favor of it.
Sorry I meant that in the most light hearted way possible. Apologies if that came off negative! I really appreciate this feedback 👍 |
@yorinasub17 |
We now have multiple If the original use case still can't be addressed with the new functionality, please open a new issue. Thanks! |
It appears that Terraform 0.12 no longer merges maps, but only selects the 'newest' one when using a chained set of tfvars files. It is documented at the bottom of this page:
https://www.terraform.io/docs/configuration/variables.html
This definitely breaks a good part of my workflow with 0.12. I'm wondering if perhaps this could be something that is supported in the terragrunt.hcl inputs = {} sections so that you can chain map variables and merge them like with 0.11.x?
The text was updated successfully, but these errors were encountered: