Feature Request: Explode object properties into module options

[marceloboeira]

Terraform Version

Any terraform version

Use Cases

As companies grow/environments grow, it is relevant to create generic modules to orchestrate end-to-end provisioning of multiple parts of the infrastructure. However, those modules tend to evolve to have many parameters and, most importantly, NOT to make assumptions about the underlying environment, being flexible in accepting different cloud-platforms accounts, environments, network configuration, security, monitoring, and so on...

Once you create a local setup folder/module for a specific environment, most infra pieces under that folder will likely share the same region/environment/network setup, making it easier for non-platform contributors to have that "default" setup available simply.

Attempted Solutions

For instance, we've created our internal setup with something like the following:

locals.tf

# Defines most of the common defaults for the infra in this terraform state
locals {
  defaults = {
    region      = "us-east-1"
    environment = "staging" 
    network     = {
      ...
     }
    monitoring   = {
      ...
    }
    authentication = {
      ...
    }
  }
}

Then, we have hundreds of modules using those defaults, but they need to do that line by line:

module "redis" {
  source         = "../../../modules/redis/cluster"
  region         = local.default.region
  environment    = local.default.environment
  network        = local.default.network
  authentication = local.default.authentication
  monitoring     = local.default.monitoring
  name           = "cache-foo"
...
}

Proposal

Allow exploding object properties from a shared object:

module "redis" {
  source = "../../../modules/redis/cluster"
  # Inject all defaults
  local.defaults&

  # Custom Attributes
  name = "foo"
}

the local.defaults& would have the same effect as:

  region         = local.default.region
  environment    = local.default.environment
  network        = local.default.network
  authentication = local.default.authentication
  monitoring     = local.default.monitoring

References

No response

[apparentlymart]

Hi @marceloboeira! Thanks for sharing this feature request.

In today's Terraform we see folks solve this problem by passing all of the "common" items together in a single input variable:

variable "common" {
  type = object({
    region      = string
    environment = string
    network = object({
      # ...
    })
    authentication = object({
      # ...
    })
    monitoring = object({
      # ...
    })
  })
}

module "redis" {
  source = "../../../modules/redis/cluster"

  common = local.defaults
  name   = "foo"
}

This structure has the advantage of still allowing Terraform to statically validate the contents of module "redis" and detect early mistakes such as mistyped variable names, whereas with a dynamic expression that could potentially define any argument inside that block Terraform would in the worst case need to wait until the apply step to even know what's defined inside that block. The Terraform language design generally errs on the side of explicit rather than implicit so that it has the best chance of giving early feedback during either the validation phase or the planning phase.

Would passing some sort of "common object" to your modules work to address your problem, or is there an additional detail that makes that design not viable for your situation?

Thanks!

[BastienGenestal]

Hi ! Glad I'm not the only one with such a use case.

I've faced your exact situation before.
Right now, I am in the following scenario and I think it would require a similar feature;

I just created all the code for a python lambda. I would like to use this lambda through a custom tf module that creates the lambda function, some of its environment variables, the role it is using etc...

My custom module uses the official Terraform lambda module behind the scene.
My custom module would only:

• Force some parameters into the official AWS lambda module (such as the description, handler or runtime)
• Force the usage of my python code for the lambda function

Now I still want my custom module to be flexible and users to be able to pass any argument to my custom module that is not overwritten and forward it to the official lambda module.

The only way I can reach my goal right now would be to manually list all the existing parameters in the official module in my custom module and forward them one by one.

For now, my custom module is something around the lines of:

module "lambda_function" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "~> 6.0.0"

  parameter1 = "my_forced_value"
  ...
  parameter2 = var.parameter2
  parameter3 = var.parameter3 
}
...

I wish I could tell terraform: here are my forced values, and I would accept and forward any other parameter here to be used in this object.
That would mean the module object has a dynamic_parameter block,
Such as:

module "lambda_function" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "~> 6.0.0"

  parameter1 = "my_forced_value"
  ...
  dynamic_parameters {} 
}
...

Then anytime someone using my module passes through a parameter with undeclared key name, it would be interpreted as a parameter for this resource.

I am clueless about the layers under this and maybe what I am requesting is way out of Terraform capabilities but I feel like my use case might not be that unusual.