-
-
Notifications
You must be signed in to change notification settings - Fork 987
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature Request: Shared and overridable variables (globals?) #814
Comments
Since I suggested this, I am on board with this idea! Although, I am curious to hear others' opinions about separating I am a bit confused by what you mean by |
By Edit: Yes, access would be explicit (e.g. |
If you wanted to make it even more complicated, in reality |
After some more thought, I have an alternate proposal:
Both This strategy would accomplish a few things:
As far as implementation goes, looking through the terraform code it doesn't seem incredibly complicated to copy the variable discovery, graph creation, graph verification, and evaluation. Some code from the terraform project could even be imported and re-used. Thoughts? |
One minor issue I have with the current implementation of locals is that I cannot use the As for the original proposal, a lot of my locals are being passed down to the underlying child modules using inputs, so yeah I would say this seems like a good idea in general. |
Ah that is a good point and an oversight on my part. The original proposal for
While I understand where you are going with this, I think this could overcomplicate the implementation. Right now the path functions are implemented as simple go functions, which makes them fairly easy to maintain. Having to fold them into the include block requires adding additional custom parsing logic while the include is being created, that then needs to be passed down throughout the code to each execution contexts. I can see this being unintuitive and a pain to maintain. I'd say that for now, start by implementing the basic version of |
And it might turn out that we don't need |
Thanks for going through this thought exercise by the way! Very helpful in understanding why you need it, and your thought process on approaching the implementation! |
What's the difference between |
You can have a child config that is:
|
Would that be different than adding support for inputs referencing other inputs? E.g., Or how does this compare with a |
I think The benefit of |
Oh, I was thinking BTW, I'm not 100% set on any approach here and am just thinking out loud. My main goal is to find the minimal set of tools we could expose for defining, sharing, reusing, and passing variables around. So the question is whether |
Sorry, I was away for a few days and didn't get a chance to respond. Re: replacing the path functions with static variables: On the contrary, I think using functions for this is actually the unintuitive way to do it. In terraform, functions are available and idempotent no matter where they're used. If We can still forego complicated dependency logic for now by just erroring if the Re: It would definitely be possible to replace globals with locals, but I see two issues with that (assuming simple dependency logic):
Re: What's the use-case for this? It seems to me all of those cases would be covered by good |
I'm definitely willing to whip up a PoC even if it needs significant changes before being merged. |
I wasn't talking about the end user UX. I was talking about the code that actually implements it. So it isn't about whether or not it is intuitive to users of This is why I mentioned this should be addressed independent of the
The main use case for this is if you want to fragment the |
Ah, sorry, I completely misunderstood that. I think it should be just as easy to implement - just remove those functions and add a variable group to the evaluation context when evaluating everything but
Cool, thanks for the explanation. Is multi-include being considered, and if so would |
I think you are underestimating the complexity of interjecting something to the parsing pipeline to add this. But who knows! I might be missing some clever way to do this.
Yes let's keep that separate.
You can see #303 for past thoughts on this, as well as some current thoughts. In general, multi-include is not something we want to add and if possible would like to use other primitives to achieve the same effect. |
By the way, you might want to hold off on implementation until I am done with #828. |
I don't think so, unless I'm missing something. It should be just as easy as adding if locals != nil {
ctx.Variables = map[string]cty.Value{"local": *locals}
} In fact this function definition already has the value of the include block available to it: func CreateTerragruntEvalContext(
filename string,
terragruntOptions *options.TerragruntOptions,
include *IncludeConfig,
locals *cty.Value,
) *hcl.EvalContext {
Thanks for the heads up, it definitely looks that way. I'll do some initial exploration but I won't spend too much time on it till that's complete. |
Ok #828 is now merged. Also, I've refactored the eval context parameters into a struct So @apottere , feel free to start working on this! Be sure to think through where |
Sounds good, thanks! |
@yorinasub17 I got really busy, but I finally had some time to put together a POC of this in #858. Let me know what you think. I decided to go for the more complicated approach because I was pretty sure I could make it work and it was more fun to try and figure out. I can definitely implement a simpler approach if you want. |
Any news on this one? |
Wanted to ask for any update on this, thanks! |
This would be nice to have |
I do something a little like this, but with recursion hell. Not sure how interested people would be to use it. terragrunt.hcl ( I don't use inputs, in terragrunt.hcl I generate a vars file since I hit the system limit for commandline length )
root.hcl - I read the top most level here
In instance.hcl I read and merge all lower levels
Note: This is just snippets so it doesn't tell the whole story or describe the entire solution, but hopefully you get the gist of it. And I do the same for service, region, sub, env. While also in each lower namespace I subtract those files that are higher up. I do this because I want to gain access to all the variables and use them in a higher namespace. It takes like 40 seconds to parse everything. An alternative could have been to merge it all the end instead of this in every single level. Then it goes much much faster, but I as I want to have all the vars from below.. its a bit of a tradeoff. But honestly, is speed such a big deal for infra if I'm going to wait for an azure virtual gateway to come up in 25 minutes.. 💤 I use the same approach for modules, so that I can override terraform module versions per instance, service, region, sub, env.. etc.. I've done this for quite a few of my projects.. if you know a better way or if you think I'm crazy for doing it this way, I'm always open to learn. edit: I guess its a bit like phatcher gave an example on in the mentioned issue here: #1942 |
I came up with something very similar, but instead of using Example of root module with this and a few more ideas:
|
@gw0 - is the above possible using yaml files instead of hcl files? We've already got the necessary configuration in yaml files which is already being used by other things? Making a hcl specific config file just for terraform would be a pain |
Sure, a similar approach could be used with YAML. For YAML there is |
The reason I don't use input is because of this issue: #2132 |
Instead of cramming everything into inputs (meaning env vars), have you considered:
|
I don't cram everything into inputs :) hence I said The ticket I referered to has a solution proposed by denis which I use. |
Background
For most of our terraform projects, we prefer to keep the remote state in the same region as the resources that are being created. This allows us to manage resources in other regions when S3/DynamoDB is down in a particular region.
We also have a pretty standard directory structure for services:
With the addition of terraform functions, it was simple to get the region from the directory name and use that in the backend configuration.
However, we also have a separate structure containing all of the infrastructure that doesn't belong to a single application - and this structure is much less predictable. Some resources have multiple directories before the region, and some don't have a region directory at all. Since many of these resources don't (and shouldn't) conform to the structure above, it would be very convenient to specify a
region
variable in the parentterragrunt.hcl
that can be overridden in the childterragrunt.hcl
.There are certainly ways to work around this right now, but what I would love to be able to do is:
Proposed Solution
I would like to implement something similar to
locals
, except keys in the block would be merged with the parent config (with child definitions overwriting parent definitions). Thus, the config resolution order would be:locals
include
globals
This would accomplish three things:
locals
but need the relative path to the parent configQuestions
globals
the correct name, or is there something better?This is a result of #802 (comment) and possibly related to #132.
The text was updated successfully, but these errors were encountered: