Support use cases with conditional logic

[phinze]

It's been important from the beginning that Terraform's configuration language is declarative, which has meant that the core team has intentionally avoided adding flow-control statements like conditionals and loops to the language.

But in the real world, there are still plenty of perfectly reasonable scenarios that are difficult to express in the current version of Terraform without copious amounts of duplication because of the lack of conditionals. We'd like Terraform to support these use cases one way or another.

I'm opening this issue to collect some real-world example where, as a config author, it seems like an if statement would really make things easier.

Using these examples, we'll play around with different ideas to improve the tools Terraform provides to the config author in these scenarios.

So please feel free to chime in with some specific examples - ideally with with blocks of Terraform configuration included. If you've got ideas for syntax or config language features that could form a solution, those are welcome here too.

(No need to respond with just "+1" / :+1: on this thread, since it's an issue we're already aware is important.)

[chrisferry]

Here are 2 examples: https://gist.github.com/chrisferry/780140d709bfad51038c
The RDS and ELB modules have minor differences. SSL cert or no. IOPs or no.

[ketzacoatl]

OMG, I could rant on about this issue for a long while. One litany of clear and concise use cases are found in implementing a concept as a terraform module. There are even community modules which exemplify this.. two modules for essentially the same thing, one provides an ELB, one does not.

I tend to want to write terraform source as I do with Saltstack: as a giant jinja template. This affords me a whole lot of flexibility while ensuring the application (Salt) ends up with a machine-readable format. Terraform sort of has this type of pre-processing (with interpolation), but Saltstack's implementation leverages the concept of a pluggable renderer system.. this pre-processor renders the template to give to salt for processing. The renderer can be jinja, mako, or any one of a few different systems (made available as modules). The user/developer experience has been exceptional, and I am thankful for the power it lends, while still providing a declarative system. In contrast, using terraform has felt cumbersome and restrictive in the expression of one's needs (especially when I have gone to encapsulate a working POC into a module).

Thank you for opening this dicussion!

[apparentlymart]

This is a bit of a stretch on the topic of this issue, but a couple of times I've found myself wishing for an iteration construct to allow me to create a set of resources that each map one-to-one to an item in a list.

I've found and then promptly forgotten a number of examples (having dismissed them as impossible), but one that stayed in my mind was giving EC2 instances more memorable local hostnames and then creating Route53 records for each of them.

resource "aws_instance" "app_server" {
    # (...)

    count = 5

    provisioner "remote-exec" {
        inline = [
            "set-hostname-somehow appname-${join(\"-\", split(\".\", self.private_ip))}"
        ]
    }
}

foreach "${aws_instance.app_server.*}" {
    resource "aws_route53_record" "app_server-${item.private_ip}" {
        zone_id = "${something_defined_elsewhere}"
        name = "appname-${join(\"-\", split(\".\", self.private_ip))}.mydomain.com"
        type = "A"
        records = ["${item.private_ip}"]
    }
}

In my imagination, this creates a set of resources named things like aws_route53_record.app_server-10.0.0.1 which then get created/destroyed as you'd expect when the corresponding instances come and go. In case it wasn't obvious, item here is imagined as the iteration variable.

While I was sketching this out I also came up with an alternative formulation that might end up leading to a simpler mental model:

resource "aws_instance" "app_server" {
    # (...)

    count = 5

    provisioner "remote-exec" {
        inline = [
            "set-hostname-somehow appname-${join(\"-\", split(\".\", self.private_ip))}"
        ]
    }

    child_resource "aws_route53_record" "hostname" {
        zone_id = "${something_defined_elsewhere}"
        name = "appname-${join(\"-\", split(\".\", parent.private_ip))}.mydomain.com"
        type = "A"
        records = ["${parent.private_ip}"]
    }
}

In this formulation, rather than generically supporting iteration over all lists we can just create a family of child resources for each "parent" resource. This feels conceptually similar to how per-instance provisioners work. I'm imagining that the child resource would interpolate like ${aws_instance.app_server.0.children.hostname.records}, or indeed ${aws_instance.app_server.*.children.hostname.name} to get all of the created hostnames.

[bgeesaman]

When your environment always looks the same (e.g. for a long standing/running app), the declarative language of terraform is expressive enough for most needs. However, my use case (frequent spin ups/tear downs of AWS VPCs of a similar general structure but with plenty of instance/subnet variation) means that terraform is not the "start" of my pipeline. I need to combine some configuration, logic, and templates on the fly each time to define my desired environment before terraform can ingest it.

Right now, I'm planning on writing something custom (rake and erb?) to generate the needed terraform json and go from there because I don't see template-esque logic as a job for terraform's declarative config language, It already can use json as an incoming interchange format, so wouldn't it be more versatile to just leverage any of the existing template renderers out there as @ketzacoatl described? It would keep logic out of the configuration and keep the terraform language simple/clean.

Appreciate this discussion and I'm open to learning about a better way to solve these kinds of problems.

[bgeesaman]

And I just realized why my suggested approach is flawed in some cases. The power of terraform is using derived data at runtime as variables elsewhere. If you render separately ahead of time in some cases, you lose that. The count=5 is an example of something you can't pre-render and then reference easily. Gah, sorry.

[ketzacoatl]

The power in terraform, IMHO, is that we have the flexibility to choose how much we do before , in TF, and after TF runs. In most cases, you need to start working with some wrapper to create the JSON you want, when the existing interpolation syntax won't get you what you want. I personally, have avoided this as I would rather keep the before limited to a CI / admin who defines details in the terraform.tfvars file to pass to TF when it runs. At the same time, this is also where the conditional logic and other interpolation/syntacitic sugar comes in (but is in some cases missing).

[pmoust]

Last December in AMS Dockercon I had asked @mitchellh if there would be any plans to add such logic control in Terraform DSL, he explained his view on keeping Terraform as simple as possible maybe adding a little algebraic functionality (which has already been merged) and standard string operations.

I am glad there are second thoughts on this, but it is a decision that needs a lot of input and real world justification, so thatnks @phinze for bringing this up.

I have two real world scenarios I 've faced where an if statement would have come in handy in Terraform:

1. IaC of MongoDB replica sets using AWS ASG terraform module.
   Currently, if I am not wildly mistaken, 10gen does not provide a simple enough recipe to automate this through Cloudformation, the solutions are just nasty and involve adding shell script in the Cloudformation map.
   Using Terraform I would like to call a provisioner on a single node to initialise the replicaSet and obtain master status from just the first node in the ASG, while a remote file is called via cloud-init on each ASG member to rs.add() itself by connectiong to the master node.
   Right now I 've resorted in a nasty script to get the job done along with a dummy resource to invoke a provisioner.
   I am sure there must be a more elegant way, irregardless if would greatly mitigate the nastiness in my setup
2. Legacy app EC2/OpenStack webserver fleet that requires just one node to be running crons. While this setup is fundementaly flawed for obvious reason, it is still very common use-case scenario and I 've come across this. An if would be nice to instruct a remote-exec provisioner to work its magic on just one node. Right now I just create a dummy resource that depends on .0.id so that it gets provisioned from there. Ugly. There are ways around it, but still...

[pmoust]

Other scenario,

1. CoreOS fleet in an AWS ASG, should a terraform run detects an even number behind ASG, deregister one etcd server from the etcd cluster to enable sensible quorum. This touches the surface of current issue raised by @phinze as such functionality (run provisioners on ASG cluster nodes) is not implemented in Terraform, and it more-or-less defy the purpose of exact clones in ASG. Still it is something I have personally come across and I manage externally.

[tayzlor]

A simple scenario -

Optionally use Atlas artifacts to deploy infrastructure - and fallback to just AMI strings (if not using atlas).

Something like -

if ${var.atlas_enabled} {
  resource "atlas_artifact" "machine" {
    name = "${var.atlas_artifact.name}"
    type = "aws.ami"
  }
}

then in a resource 

resource "aws_instance" "machine" {
  if ${var.atlas_enabled} {
     ami = "${atlas_artifact.machine.id}"
  } else {
     ami = "${var.ami_string}"
  }
}

[mzupan]

👍

I like the idea of even a simple

if ${var.atlas_enabled} {

}

There are lots of times where I'm building the same infrastructure for dev/stage/prod but don't need things in dev/stage as are needed in prod.

[franklinwise]

Even simpler, doesn't break the current syntax and prevents complexity (Inspired by Ansible):

resource "aws_instance" "machine" {
  when = "${var.atlas_enabled}"
  ami = "${atlas_artifact.machine.id}"
}

resource "aws_instance" "machine" {
  when = "not ${var.atlas_enabled}"
  ami = "${var.ami_string}"
}

[phinze]

@franklinwise that's pretty nice!

I like that it preserves the declarative syntax.