Support use cases with conditional logic #1604

Closed
phinze opened this Issue Apr 20, 2015 · 165 comments

Comments

Projects
None yet
@phinze
Member

phinze commented Apr 20, 2015

It's been important from the beginning that Terraform's configuration language is declarative, which has meant that the core team has intentionally avoided adding flow-control statements like conditionals and loops to the language.

But in the real world, there are still plenty of perfectly reasonable scenarios that are difficult to express in the current version of Terraform without copious amounts of duplication because of the lack of conditionals. We'd like Terraform to support these use cases one way or another.

I'm opening this issue to collect some real-world example where, as a config author, it seems like an if statement would really make things easier.

Using these examples, we'll play around with different ideas to improve the tools Terraform provides to the config author in these scenarios.

So please feel free to chime in with some specific examples - ideally with with blocks of Terraform configuration included. If you've got ideas for syntax or config language features that could form a solution, those are welcome here too.

(No need to respond with just "+1" / 👍 on this thread, since it's an issue we're already aware is important.)

@chrisferry

This comment has been minimized.

Show comment
Hide comment
@chrisferry

chrisferry Apr 20, 2015

Here are 2 examples: https://gist.github.com/chrisferry/780140d709bfad51038c
The RDS and ELB modules have minor differences. SSL cert or no. IOPs or no.

Here are 2 examples: https://gist.github.com/chrisferry/780140d709bfad51038c
The RDS and ELB modules have minor differences. SSL cert or no. IOPs or no.

@ketzacoatl

This comment has been minimized.

Show comment
Hide comment
@ketzacoatl

ketzacoatl Apr 23, 2015

Contributor

OMG, I could rant on about this issue for a long while. One litany of clear and concise use cases are found in implementing a concept as a terraform module. There are even community modules which exemplify this.. two modules for essentially the same thing, one provides an ELB, one does not.

I tend to want to write terraform source as I do with Saltstack: as a giant jinja template. This affords me a whole lot of flexibility while ensuring the application (Salt) ends up with a machine-readable format. Terraform sort of has this type of pre-processing (with interpolation), but Saltstack's implementation leverages the concept of a pluggable renderer system.. this pre-processor renders the template to give to salt for processing. The renderer can be jinja, mako, or any one of a few different systems (made available as modules). The user/developer experience has been exceptional, and I am thankful for the power it lends, while still providing a declarative system. In contrast, using terraform has felt cumbersome and restrictive in the expression of one's needs (especially when I have gone to encapsulate a working POC into a module).

Thank you for opening this dicussion!

Contributor

ketzacoatl commented Apr 23, 2015

OMG, I could rant on about this issue for a long while. One litany of clear and concise use cases are found in implementing a concept as a terraform module. There are even community modules which exemplify this.. two modules for essentially the same thing, one provides an ELB, one does not.

I tend to want to write terraform source as I do with Saltstack: as a giant jinja template. This affords me a whole lot of flexibility while ensuring the application (Salt) ends up with a machine-readable format. Terraform sort of has this type of pre-processing (with interpolation), but Saltstack's implementation leverages the concept of a pluggable renderer system.. this pre-processor renders the template to give to salt for processing. The renderer can be jinja, mako, or any one of a few different systems (made available as modules). The user/developer experience has been exceptional, and I am thankful for the power it lends, while still providing a declarative system. In contrast, using terraform has felt cumbersome and restrictive in the expression of one's needs (especially when I have gone to encapsulate a working POC into a module).

Thank you for opening this dicussion!

@apparentlymart

This comment has been minimized.

Show comment
Hide comment
@apparentlymart

apparentlymart Apr 23, 2015

Contributor

This is a bit of a stretch on the topic of this issue, but a couple of times I've found myself wishing for an iteration construct to allow me to create a set of resources that each map one-to-one to an item in a list.

I've found and then promptly forgotten a number of examples (having dismissed them as impossible), but one that stayed in my mind was giving EC2 instances more memorable local hostnames and then creating Route53 records for each of them.

resource "aws_instance" "app_server" {
    # (...)

    count = 5

    provisioner "remote-exec" {
        inline = [
            "set-hostname-somehow appname-${join(\"-\", split(\".\", self.private_ip))}"
        ]
    }
}

foreach "${aws_instance.app_server.*}" {
    resource "aws_route53_record" "app_server-${item.private_ip}" {
        zone_id = "${something_defined_elsewhere}"
        name = "appname-${join(\"-\", split(\".\", self.private_ip))}.mydomain.com"
        type = "A"
        records = ["${item.private_ip}"]
    }
}

In my imagination, this creates a set of resources named things like aws_route53_record.app_server-10.0.0.1 which then get created/destroyed as you'd expect when the corresponding instances come and go. In case it wasn't obvious, item here is imagined as the iteration variable.

While I was sketching this out I also came up with an alternative formulation that might end up leading to a simpler mental model:

resource "aws_instance" "app_server" {
    # (...)

    count = 5

    provisioner "remote-exec" {
        inline = [
            "set-hostname-somehow appname-${join(\"-\", split(\".\", self.private_ip))}"
        ]
    }

    child_resource "aws_route53_record" "hostname" {
        zone_id = "${something_defined_elsewhere}"
        name = "appname-${join(\"-\", split(\".\", parent.private_ip))}.mydomain.com"
        type = "A"
        records = ["${parent.private_ip}"]
    }
}

In this formulation, rather than generically supporting iteration over all lists we can just create a family of child resources for each "parent" resource. This feels conceptually similar to how per-instance provisioners work. I'm imagining that the child resource would interpolate like ${aws_instance.app_server.0.children.hostname.records}, or indeed ${aws_instance.app_server.*.children.hostname.name} to get all of the created hostnames.

Contributor

apparentlymart commented Apr 23, 2015

This is a bit of a stretch on the topic of this issue, but a couple of times I've found myself wishing for an iteration construct to allow me to create a set of resources that each map one-to-one to an item in a list.

I've found and then promptly forgotten a number of examples (having dismissed them as impossible), but one that stayed in my mind was giving EC2 instances more memorable local hostnames and then creating Route53 records for each of them.

resource "aws_instance" "app_server" {
    # (...)

    count = 5

    provisioner "remote-exec" {
        inline = [
            "set-hostname-somehow appname-${join(\"-\", split(\".\", self.private_ip))}"
        ]
    }
}

foreach "${aws_instance.app_server.*}" {
    resource "aws_route53_record" "app_server-${item.private_ip}" {
        zone_id = "${something_defined_elsewhere}"
        name = "appname-${join(\"-\", split(\".\", self.private_ip))}.mydomain.com"
        type = "A"
        records = ["${item.private_ip}"]
    }
}

In my imagination, this creates a set of resources named things like aws_route53_record.app_server-10.0.0.1 which then get created/destroyed as you'd expect when the corresponding instances come and go. In case it wasn't obvious, item here is imagined as the iteration variable.

While I was sketching this out I also came up with an alternative formulation that might end up leading to a simpler mental model:

resource "aws_instance" "app_server" {
    # (...)

    count = 5

    provisioner "remote-exec" {
        inline = [
            "set-hostname-somehow appname-${join(\"-\", split(\".\", self.private_ip))}"
        ]
    }

    child_resource "aws_route53_record" "hostname" {
        zone_id = "${something_defined_elsewhere}"
        name = "appname-${join(\"-\", split(\".\", parent.private_ip))}.mydomain.com"
        type = "A"
        records = ["${parent.private_ip}"]
    }
}

In this formulation, rather than generically supporting iteration over all lists we can just create a family of child resources for each "parent" resource. This feels conceptually similar to how per-instance provisioners work. I'm imagining that the child resource would interpolate like ${aws_instance.app_server.0.children.hostname.records}, or indeed ${aws_instance.app_server.*.children.hostname.name} to get all of the created hostnames.

@bgeesaman

This comment has been minimized.

Show comment
Hide comment
@bgeesaman

bgeesaman Apr 25, 2015

When your environment always looks the same (e.g. for a long standing/running app), the declarative language of terraform is expressive enough for most needs. However, my use case (frequent spin ups/tear downs of AWS VPCs of a similar general structure but with plenty of instance/subnet variation) means that terraform is not the "start" of my pipeline. I need to combine some configuration, logic, and templates on the fly each time to define my desired environment before terraform can ingest it.

Right now, I'm planning on writing something custom (rake and erb?) to generate the needed terraform json and go from there because I don't see template-esque logic as a job for terraform's declarative config language, It already can use json as an incoming interchange format, so wouldn't it be more versatile to just leverage any of the existing template renderers out there as @ketzacoatl described? It would keep logic out of the configuration and keep the terraform language simple/clean.

Appreciate this discussion and I'm open to learning about a better way to solve these kinds of problems.

When your environment always looks the same (e.g. for a long standing/running app), the declarative language of terraform is expressive enough for most needs. However, my use case (frequent spin ups/tear downs of AWS VPCs of a similar general structure but with plenty of instance/subnet variation) means that terraform is not the "start" of my pipeline. I need to combine some configuration, logic, and templates on the fly each time to define my desired environment before terraform can ingest it.

Right now, I'm planning on writing something custom (rake and erb?) to generate the needed terraform json and go from there because I don't see template-esque logic as a job for terraform's declarative config language, It already can use json as an incoming interchange format, so wouldn't it be more versatile to just leverage any of the existing template renderers out there as @ketzacoatl described? It would keep logic out of the configuration and keep the terraform language simple/clean.

Appreciate this discussion and I'm open to learning about a better way to solve these kinds of problems.

@bgeesaman

This comment has been minimized.

Show comment
Hide comment
@bgeesaman

bgeesaman Apr 25, 2015

And I just realized why my suggested approach is flawed in some cases. The power of terraform is using derived data at runtime as variables elsewhere. If you render separately ahead of time in some cases, you lose that. The count=5 is an example of something you can't pre-render and then reference easily. Gah, sorry.

And I just realized why my suggested approach is flawed in some cases. The power of terraform is using derived data at runtime as variables elsewhere. If you render separately ahead of time in some cases, you lose that. The count=5 is an example of something you can't pre-render and then reference easily. Gah, sorry.

@ketzacoatl

This comment has been minimized.

Show comment
Hide comment
@ketzacoatl

ketzacoatl Apr 25, 2015

Contributor

The power in terraform, IMHO, is that we have the flexibility to choose how much we do before , in TF, and after TF runs. In most cases, you need to start working with some wrapper to create the JSON you want, when the existing interpolation syntax won't get you what you want. I personally, have avoided this as I would rather keep the before limited to a CI / admin who defines details in the terraform.tfvars file to pass to TF when it runs. At the same time, this is also where the conditional logic and other interpolation/syntacitic sugar comes in (but is in some cases missing).

Contributor

ketzacoatl commented Apr 25, 2015

The power in terraform, IMHO, is that we have the flexibility to choose how much we do before , in TF, and after TF runs. In most cases, you need to start working with some wrapper to create the JSON you want, when the existing interpolation syntax won't get you what you want. I personally, have avoided this as I would rather keep the before limited to a CI / admin who defines details in the terraform.tfvars file to pass to TF when it runs. At the same time, this is also where the conditional logic and other interpolation/syntacitic sugar comes in (but is in some cases missing).

@pmoust

This comment has been minimized.

Show comment
Hide comment
@pmoust

pmoust Apr 28, 2015

Contributor

Last December in AMS Dockercon I had asked @mitchellh if there would be any plans to add such logic control in Terraform DSL, he explained his view on keeping Terraform as simple as possible maybe adding a little algebraic functionality (which has already been merged) and standard string operations.

I am glad there are second thoughts on this, but it is a decision that needs a lot of input and real world justification, so thatnks @phinze for bringing this up.

I have two real world scenarios I 've faced where an if statement would have come in handy in Terraform:

  1. IaC of MongoDB replica sets using AWS ASG terraform module.
    Currently, if I am not wildly mistaken, 10gen does not provide a simple enough recipe to automate this through Cloudformation, the solutions are just nasty and involve adding shell script in the Cloudformation map.
    Using Terraform I would like to call a provisioner on a single node to initialise the replicaSet and obtain master status from just the first node in the ASG, while a remote file is called via cloud-init on each ASG member to rs.add() itself by connectiong to the master node.
    Right now I 've resorted in a nasty script to get the job done along with a dummy resource to invoke a provisioner.
    I am sure there must be a more elegant way, irregardless if would greatly mitigate the nastiness in my setup
  2. Legacy app EC2/OpenStack webserver fleet that requires just one node to be running crons. While this setup is fundementaly flawed for obvious reason, it is still very common use-case scenario and I 've come across this. An if would be nice to instruct a remote-exec provisioner to work its magic on just one node. Right now I just create a dummy resource that depends on .0.id so that it gets provisioned from there. Ugly. There are ways around it, but still...
Contributor

pmoust commented Apr 28, 2015

Last December in AMS Dockercon I had asked @mitchellh if there would be any plans to add such logic control in Terraform DSL, he explained his view on keeping Terraform as simple as possible maybe adding a little algebraic functionality (which has already been merged) and standard string operations.

I am glad there are second thoughts on this, but it is a decision that needs a lot of input and real world justification, so thatnks @phinze for bringing this up.

I have two real world scenarios I 've faced where an if statement would have come in handy in Terraform:

  1. IaC of MongoDB replica sets using AWS ASG terraform module.
    Currently, if I am not wildly mistaken, 10gen does not provide a simple enough recipe to automate this through Cloudformation, the solutions are just nasty and involve adding shell script in the Cloudformation map.
    Using Terraform I would like to call a provisioner on a single node to initialise the replicaSet and obtain master status from just the first node in the ASG, while a remote file is called via cloud-init on each ASG member to rs.add() itself by connectiong to the master node.
    Right now I 've resorted in a nasty script to get the job done along with a dummy resource to invoke a provisioner.
    I am sure there must be a more elegant way, irregardless if would greatly mitigate the nastiness in my setup
  2. Legacy app EC2/OpenStack webserver fleet that requires just one node to be running crons. While this setup is fundementaly flawed for obvious reason, it is still very common use-case scenario and I 've come across this. An if would be nice to instruct a remote-exec provisioner to work its magic on just one node. Right now I just create a dummy resource that depends on .0.id so that it gets provisioned from there. Ugly. There are ways around it, but still...
@pmoust

This comment has been minimized.

Show comment
Hide comment
@pmoust

pmoust Apr 28, 2015

Contributor

Other scenario,

  1. CoreOS fleet in an AWS ASG, should a terraform run detects an even number behind ASG, deregister one etcd server from the etcd cluster to enable sensible quorum. This touches the surface of current issue raised by @phinze as such functionality (run provisioners on ASG cluster nodes) is not implemented in Terraform, and it more-or-less defy the purpose of exact clones in ASG. Still it is something I have personally come across and I manage externally.
Contributor

pmoust commented Apr 28, 2015

Other scenario,

  1. CoreOS fleet in an AWS ASG, should a terraform run detects an even number behind ASG, deregister one etcd server from the etcd cluster to enable sensible quorum. This touches the surface of current issue raised by @phinze as such functionality (run provisioners on ASG cluster nodes) is not implemented in Terraform, and it more-or-less defy the purpose of exact clones in ASG. Still it is something I have personally come across and I manage externally.
@tayzlor

This comment has been minimized.

Show comment
Hide comment
@tayzlor

tayzlor May 7, 2015

A simple scenario -

Optionally use Atlas artifacts to deploy infrastructure - and fallback to just AMI strings (if not using atlas).

Something like -

if ${var.atlas_enabled} {
  resource "atlas_artifact" "machine" {
    name = "${var.atlas_artifact.name}"
    type = "aws.ami"
  }
}

then in a resource 

resource "aws_instance" "machine" {
  if ${var.atlas_enabled} {
     ami = "${atlas_artifact.machine.id}"
  } else {
     ami = "${var.ami_string}"
  }
}

tayzlor commented May 7, 2015

A simple scenario -

Optionally use Atlas artifacts to deploy infrastructure - and fallback to just AMI strings (if not using atlas).

Something like -

if ${var.atlas_enabled} {
  resource "atlas_artifact" "machine" {
    name = "${var.atlas_artifact.name}"
    type = "aws.ami"
  }
}

then in a resource 

resource "aws_instance" "machine" {
  if ${var.atlas_enabled} {
     ami = "${atlas_artifact.machine.id}"
  } else {
     ami = "${var.ami_string}"
  }
}

@tayzlor tayzlor referenced this issue in Capgemini/Apollo May 7, 2015

Closed

Why is Atlas a requirement ?? #140

@mzupan

This comment has been minimized.

Show comment
Hide comment
@mzupan

mzupan May 17, 2015

Contributor

👍

I like the idea of even a simple

if ${var.atlas_enabled} {

}

There are lots of times where I'm building the same infrastructure for dev/stage/prod but don't need things in dev/stage as are needed in prod.

Contributor

mzupan commented May 17, 2015

👍

I like the idea of even a simple

if ${var.atlas_enabled} {

}

There are lots of times where I'm building the same infrastructure for dev/stage/prod but don't need things in dev/stage as are needed in prod.

@franklinwise

This comment has been minimized.

Show comment
Hide comment
@franklinwise

franklinwise May 22, 2015

Even simpler, doesn't break the current syntax and prevents complexity (Inspired by Ansible):

resource "aws_instance" "machine" {
  when = "${var.atlas_enabled}"
  ami = "${atlas_artifact.machine.id}"
}

resource "aws_instance" "machine" {
  when = "not ${var.atlas_enabled}"
  ami = "${var.ami_string}"
}

Even simpler, doesn't break the current syntax and prevents complexity (Inspired by Ansible):

resource "aws_instance" "machine" {
  when = "${var.atlas_enabled}"
  ami = "${atlas_artifact.machine.id}"
}

resource "aws_instance" "machine" {
  when = "not ${var.atlas_enabled}"
  ami = "${var.ami_string}"
}
@phinze

This comment has been minimized.

Show comment
Hide comment
@phinze

phinze May 27, 2015

Member

@franklinwise that's pretty nice!

I like that it preserves the declarative syntax.

Member

phinze commented May 27, 2015

@franklinwise that's pretty nice!

I like that it preserves the declarative syntax.

@knuckolls

This comment has been minimized.

Show comment
Hide comment
@knuckolls

knuckolls May 27, 2015

Contributor

Preserving the declarative syntax seems like priority number one in my opinion. I like the 'when' solution. 👍

Contributor

knuckolls commented May 27, 2015

Preserving the declarative syntax seems like priority number one in my opinion. I like the 'when' solution. 👍

@ketzacoatl

This comment has been minimized.

Show comment
Hide comment
@ketzacoatl

ketzacoatl May 28, 2015

Contributor

Yes, I too like this suggested syntax.. when, and the use of not here, makes a lot of sense.

Contributor

ketzacoatl commented May 28, 2015

Yes, I too like this suggested syntax.. when, and the use of not here, makes a lot of sense.

@rafikk

This comment has been minimized.

Show comment
Hide comment
@rafikk

rafikk May 28, 2015

This seems inadequate as it only supports the conditional creation of resources. In my usage, I've found myself wanting conditional expressions (not statements) several times.

The use case has been creating heterogenous groups of ec2 instances. I.e., I'd like to create 12 instances where the first 4 are of type m3.large and the remaining are c4.xlarge. I've hacked around this for the time being by using a lookup table and creating an entry for each index, but it's pretty nasty.

For clarification, what I'm looking for is something like:

instance_type = "${if count.index > 3 then c4.xlarge else m3.large}"

rafikk commented May 28, 2015

This seems inadequate as it only supports the conditional creation of resources. In my usage, I've found myself wanting conditional expressions (not statements) several times.

The use case has been creating heterogenous groups of ec2 instances. I.e., I'd like to create 12 instances where the first 4 are of type m3.large and the remaining are c4.xlarge. I've hacked around this for the time being by using a lookup table and creating an entry for each index, but it's pretty nasty.

For clarification, what I'm looking for is something like:

instance_type = "${if count.index > 3 then c4.xlarge else m3.large}"
@ketzacoatl

This comment has been minimized.

Show comment
Hide comment
@ketzacoatl

ketzacoatl May 28, 2015

Contributor

It might make sense to consider these use cases separately, if only for the goal of getting the simpler implemented faster than the more complicated and nuanced case. when / not as a field on all resources is a fantastic start. Figuring out the rest seems to be too much to figure out in the immediate.. why block one for the other?

Contributor

ketzacoatl commented May 28, 2015

It might make sense to consider these use cases separately, if only for the goal of getting the simpler implemented faster than the more complicated and nuanced case. when / not as a field on all resources is a fantastic start. Figuring out the rest seems to be too much to figure out in the immediate.. why block one for the other?

@bortels

This comment has been minimized.

Show comment
Hide comment
@bortels

bortels May 30, 2015

Expanding on the referenced aws spot instances - I'd like to be able to express "spin up N instances in at least M availability zones, bidding the current bid price * X, But fallback to on-demand for any az where the bid price is > Y, or spot instance is unavailable". That seems complex (I have implemented it via a custom ruby script now), so maybe it's less a "we need conditionals" argument and more a "it would be nice if the aws provider abstracted instance types and did the "right thing").

Maybe the whole conditional thing could be handled by some sort of "call to external" hook to work the logic? It's a cop-out, in a way, but perhaps the most flexible in the end. If the suggested "when" could call an arbitrary script, with current context, you could kinda wedge in anything you needed as a shim, in only the spots where declarative is problematic. Can terraform do external calls like that already? I came into this from the side googling for spot instance solutions and seeing if terraform had support, so I'm not super familiar with current functionality there.

bortels commented May 30, 2015

Expanding on the referenced aws spot instances - I'd like to be able to express "spin up N instances in at least M availability zones, bidding the current bid price * X, But fallback to on-demand for any az where the bid price is > Y, or spot instance is unavailable". That seems complex (I have implemented it via a custom ruby script now), so maybe it's less a "we need conditionals" argument and more a "it would be nice if the aws provider abstracted instance types and did the "right thing").

Maybe the whole conditional thing could be handled by some sort of "call to external" hook to work the logic? It's a cop-out, in a way, but perhaps the most flexible in the end. If the suggested "when" could call an arbitrary script, with current context, you could kinda wedge in anything you needed as a shim, in only the spots where declarative is problematic. Can terraform do external calls like that already? I came into this from the side googling for spot instance solutions and seeing if terraform had support, so I'm not super familiar with current functionality there.

@rossedman

This comment has been minimized.

Show comment
Hide comment
@rossedman

rossedman Jun 4, 2015

👍 Really like @franklinwise suggestion. I think this is the way to go if control statements are added.

👍 Really like @franklinwise suggestion. I think this is the way to go if control statements are added.

@RJSzynal

This comment has been minimized.

Show comment
Hide comment
@RJSzynal

RJSzynal Jun 11, 2015

Just to add another use case similar to rafikk's example.
I'm using terraform to bring up a count of 'container instances' or 'nodes' in an AWS ECS cluster with consul running on each one. I want the first three to be run as servers but the rest to be agents.
Currently I have to duplicate the resource block which isn't very neat and any changes have to be done twice which allows too much room for human error.

It would be more flexible if I could do the following:
user_data = "${if count.index < 4 then SERVER_SCRIPT else AGENT_SCRIPT}"

Just to add another use case similar to rafikk's example.
I'm using terraform to bring up a count of 'container instances' or 'nodes' in an AWS ECS cluster with consul running on each one. I want the first three to be run as servers but the rest to be agents.
Currently I have to duplicate the resource block which isn't very neat and any changes have to be done twice which allows too much room for human error.

It would be more flexible if I could do the following:
user_data = "${if count.index < 4 then SERVER_SCRIPT else AGENT_SCRIPT}"

@rafikk

This comment has been minimized.

Show comment
Hide comment
@rafikk

rafikk Jun 16, 2015

@RJSzynal We have the same thing in our configuration. I really think support conditional logic inside of interpolation blocks is a must.

rafikk commented Jun 16, 2015

@RJSzynal We have the same thing in our configuration. I really think support conditional logic inside of interpolation blocks is a must.

@thegedge

This comment has been minimized.

Show comment
Hide comment
@thegedge

thegedge Jun 18, 2015

Contributor

tl;dr my five cents is that I'm for the high-level idea of better conditional / looping support, but would like to stick to keeping it declarative.

I like the child resource proposal by @apparentlymart (far more than the foreach, which feels too imperative). That being said, for that specific example I feel like the resource isn't really a child. What may be better is a top-level "group" construct, which supports count-style looping.

I'm also +1 for some basic conditional structure that gets interpolated. I don't like the imperative style "if then else" that's been recommended. I'd rather stick with a more procedural style thing that we already have, e.g., user_data = "${cond(var.use_foo, "foo", "bar")}", or perhaps some basic conditions user_data = "${cond(var.count = 1, "foo", "bar")}"

I agree with @rafikk with the lookup table being too verbose:

variable "lookup_table" {
  default = {
    "true" = "foo"
    "false" = "bar"
  }
}

variable "use_foo" {}

resource "aws_instance" "my_instance" {
  ami = "${lookup(var.lookup_table, var.use_foo)}"
  ...
}

I'm also not fond of the if block surrounding a resource, as things start looking too imperative. A better approach to stick with the declarative format would be something like:

resource "aws_instance" "my_instance" {
  ignore = "${var.disabled}"
}

I think setting count = 0 essentially does this, so this may be redundant. Sorry if I've repeated any ideas that may have already been expressed in earlier comments!

Contributor

thegedge commented Jun 18, 2015

tl;dr my five cents is that I'm for the high-level idea of better conditional / looping support, but would like to stick to keeping it declarative.

I like the child resource proposal by @apparentlymart (far more than the foreach, which feels too imperative). That being said, for that specific example I feel like the resource isn't really a child. What may be better is a top-level "group" construct, which supports count-style looping.

I'm also +1 for some basic conditional structure that gets interpolated. I don't like the imperative style "if then else" that's been recommended. I'd rather stick with a more procedural style thing that we already have, e.g., user_data = "${cond(var.use_foo, "foo", "bar")}", or perhaps some basic conditions user_data = "${cond(var.count = 1, "foo", "bar")}"

I agree with @rafikk with the lookup table being too verbose:

variable "lookup_table" {
  default = {
    "true" = "foo"
    "false" = "bar"
  }
}

variable "use_foo" {}

resource "aws_instance" "my_instance" {
  ami = "${lookup(var.lookup_table, var.use_foo)}"
  ...
}

I'm also not fond of the if block surrounding a resource, as things start looking too imperative. A better approach to stick with the declarative format would be something like:

resource "aws_instance" "my_instance" {
  ignore = "${var.disabled}"
}

I think setting count = 0 essentially does this, so this may be redundant. Sorry if I've repeated any ideas that may have already been expressed in earlier comments!

@hartfordfive

This comment has been minimized.

Show comment
Hide comment

👍

@apparentlymart

This comment has been minimized.

Show comment
Hide comment
@apparentlymart

apparentlymart Jul 24, 2015

Contributor

@thegedge sorry I just noticed your response to my earlier example even though you posted it a while back.

The special thing I was imagining for "child resources" is that they'd always have an implied dependency on their parent, so if you delete the instance then that always deletes the record along with it... perhaps "child" is the wrong word, but I was going for "this thing only exists to support the thing it's nested inside".

Contributor

apparentlymart commented Jul 24, 2015

@thegedge sorry I just noticed your response to my earlier example even though you posted it a while back.

The special thing I was imagining for "child resources" is that they'd always have an implied dependency on their parent, so if you delete the instance then that always deletes the record along with it... perhaps "child" is the wrong word, but I was going for "this thing only exists to support the thing it's nested inside".

@thegedge

This comment has been minimized.

Show comment
Hide comment
@thegedge

thegedge Jul 30, 2015

Contributor

@apparentlymart Ah yes, that would be nice. Could that also be solved with a "also destroy dependent resources" flag to terraform destroy? Maybe modeling these things as being intimately connected (i.e., an atomic unit) would be useful in other, not immediately obvious ways too.

Anyways, I'm also going to add in an example from my team, since I didn't do that in my last comment and that's what @phinze was interested in seeing! We want our app developers to build things on top of a terraformed "cloud", but we want them to be able to this with a minimal configuration that doesn't require much/any terraform knowledge.

Many of our apps follow a similar recipe: rails app that sometimes needs redis, memcache, an S3 bucket, a place to run the rails app, and/or a database, maybe some other things. We'd like to construct a module that allows our app developers to conditionally select what they need for their app. It would look something like this from the app developer's perspective:

## /my_app/main.tf
module "my_app" {
  source = "../app_template"
  rails_server = "puma"
  background_jobs = "resque"
  memcache = true
}

## /app_template/main.tf
...
resource "aws_elasticache_cluster" "memcache" {
  ignore = "${!var.memcache}"
  ...
}

resource "aws_elasticache_cluster" "redis" {
  ignore = "${var.background_jobs != "resque"}"
  ...
}
...
Contributor

thegedge commented Jul 30, 2015

@apparentlymart Ah yes, that would be nice. Could that also be solved with a "also destroy dependent resources" flag to terraform destroy? Maybe modeling these things as being intimately connected (i.e., an atomic unit) would be useful in other, not immediately obvious ways too.

Anyways, I'm also going to add in an example from my team, since I didn't do that in my last comment and that's what @phinze was interested in seeing! We want our app developers to build things on top of a terraformed "cloud", but we want them to be able to this with a minimal configuration that doesn't require much/any terraform knowledge.

Many of our apps follow a similar recipe: rails app that sometimes needs redis, memcache, an S3 bucket, a place to run the rails app, and/or a database, maybe some other things. We'd like to construct a module that allows our app developers to conditionally select what they need for their app. It would look something like this from the app developer's perspective:

## /my_app/main.tf
module "my_app" {
  source = "../app_template"
  rails_server = "puma"
  background_jobs = "resque"
  memcache = true
}

## /app_template/main.tf
...
resource "aws_elasticache_cluster" "memcache" {
  ignore = "${!var.memcache}"
  ...
}

resource "aws_elasticache_cluster" "redis" {
  ignore = "${var.background_jobs != "resque"}"
  ...
}
...
@glenjamin

This comment has been minimized.

Show comment
Hide comment
@glenjamin

glenjamin Sep 4, 2015

Contributor

An example I just ran into was attempting to create a list of users on AWS:

variable "devops" {
    default = "user1,user2,user3"
}

resource "aws_iam_group" "Administrators" {
    name = "Administrators"
    path = "/"
}

resource "aws_iam_user" "devops" {
    count = "${length(split(",", var.devops))}"
    name = "${element(split(",", var.devops), count.index)}"
}

This creates a reasonable create plan

+ aws_iam_user.users.0
    arn:       "" => "<computed>"
    name:      "" => "user1"
    path:      "" => "/"
    unique_id: "" => "<computed>"

+ aws_iam_user.users.1
    arn:       "" => "<computed>"
    name:      "" => "user2"
    path:      "" => "/"
    unique_id: "" => "<computed>"

+ aws_iam_user.users.2
    arn:       "" => "<computed>"
    name:      "" => "user3"
    path:      "" => "/"
    unique_id: "" => "<computed>"

But now user2 leaves.

-/+ aws_iam_user.users.1
    arn:       "arn:aws:iam::909704556315:user/user2" => "<computed>"
    name:      "user2" => "user3" (forces new resource)
    path:      "/" => "/"
    unique_id: "AIDAIOESVNFST3M3THMO6" => "<computed>"

And the list gets shuffled down, deleting a legitmate user along the way.

A loop and a way to use a variable in a resource identifier would resolve this.

Contributor

glenjamin commented Sep 4, 2015

An example I just ran into was attempting to create a list of users on AWS:

variable "devops" {
    default = "user1,user2,user3"
}

resource "aws_iam_group" "Administrators" {
    name = "Administrators"
    path = "/"
}

resource "aws_iam_user" "devops" {
    count = "${length(split(",", var.devops))}"
    name = "${element(split(",", var.devops), count.index)}"
}

This creates a reasonable create plan

+ aws_iam_user.users.0
    arn:       "" => "<computed>"
    name:      "" => "user1"
    path:      "" => "/"
    unique_id: "" => "<computed>"

+ aws_iam_user.users.1
    arn:       "" => "<computed>"
    name:      "" => "user2"
    path:      "" => "/"
    unique_id: "" => "<computed>"

+ aws_iam_user.users.2
    arn:       "" => "<computed>"
    name:      "" => "user3"
    path:      "" => "/"
    unique_id: "" => "<computed>"

But now user2 leaves.

-/+ aws_iam_user.users.1
    arn:       "arn:aws:iam::909704556315:user/user2" => "<computed>"
    name:      "user2" => "user3" (forces new resource)
    path:      "/" => "/"
    unique_id: "AIDAIOESVNFST3M3THMO6" => "<computed>"

And the list gets shuffled down, deleting a legitmate user along the way.

A loop and a way to use a variable in a resource identifier would resolve this.

@aavileli

This comment has been minimized.

Show comment
Hide comment
@aavileli

aavileli Sep 7, 2015

If you are building a module for any resource like RDS or any resource that change behaviour when additional attributes have been set then a evaluating function is required like the following which is available in cloudformation

"MyDB" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage" : "5",
"DBInstanceClass" : "db.m1.small",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : { "Ref" : "DBUser" },
"MasterUserPassword" : { "Ref" : "DBPassword" },
"DBParameterGroupName" : { "Ref" : "MyRDSParamGroup" },
"DBSnapshotIdentifier" : {
"Fn::If" : [
"UseDBSnapshot",
{"Ref" : "DBSnapshotName"},
{"Ref" : "AWS::NoValue"}
]
}
}
}

aavileli commented Sep 7, 2015

If you are building a module for any resource like RDS or any resource that change behaviour when additional attributes have been set then a evaluating function is required like the following which is available in cloudformation

"MyDB" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage" : "5",
"DBInstanceClass" : "db.m1.small",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : { "Ref" : "DBUser" },
"MasterUserPassword" : { "Ref" : "DBPassword" },
"DBParameterGroupName" : { "Ref" : "MyRDSParamGroup" },
"DBSnapshotIdentifier" : {
"Fn::If" : [
"UseDBSnapshot",
{"Ref" : "DBSnapshotName"},
{"Ref" : "AWS::NoValue"}
]
}
}
}

@jonhatalla

This comment has been minimized.

Show comment
Hide comment
@jonhatalla

jonhatalla Sep 10, 2015

AWS Elasticache Redis has a snapshot parameter. Sometimes we want to pass in a snapshot, and we set this param: snapshot_arns = ["${var.redis_snapshot}"]

Other times we do not want to pass in snapshot, however if you pass in an empty string, the elasticache cluster will not be built as its an invalid request. Even using modules and count=0, we cannot get around the conditional nature of this.

AWS Elasticache Redis has a snapshot parameter. Sometimes we want to pass in a snapshot, and we set this param: snapshot_arns = ["${var.redis_snapshot}"]

Other times we do not want to pass in snapshot, however if you pass in an empty string, the elasticache cluster will not be built as its an invalid request. Even using modules and count=0, we cannot get around the conditional nature of this.

@Ry7n

This comment has been minimized.

Show comment
Hide comment
@Ry7n

Ry7n Sep 11, 2015

The use case that I ran into today was wishing for:

resource "aws_elb" "maybe-ssl-elb" {
  listener {
    instance_port = 80
    instance_protocol = "http"
    lb_port = 80
    lb_protocol = "http"
  }
  if (not empty("${var.elb_server_certificate_arn}")) {
    listener {
      instance_port = 443
      instance_protocol = "https"
      lb_port = 443
      lb_protocol = "https"
      ssl_certificate_id = "${var.elb_server_certificate_arn}"
    }
  }
}

Ry7n commented Sep 11, 2015

The use case that I ran into today was wishing for:

resource "aws_elb" "maybe-ssl-elb" {
  listener {
    instance_port = 80
    instance_protocol = "http"
    lb_port = 80
    lb_protocol = "http"
  }
  if (not empty("${var.elb_server_certificate_arn}")) {
    listener {
      instance_port = 443
      instance_protocol = "https"
      lb_port = 443
      lb_protocol = "https"
      ssl_certificate_id = "${var.elb_server_certificate_arn}"
    }
  }
}
@davedash

This comment has been minimized.

Show comment
Hide comment
@davedash

davedash Sep 17, 2015

Contributor

I have a module which I was using for each environment "./app" now I've split it up into "./app-prod" "./app-stage" etc because some environments use user_data some do not. And the ones that do not I can't pass in a variable that says null. An if statement would allow us to do something like:

if "${var.userdata}" {
    user_data = "${var.userdata}"
}

Of course having a null or something would be good too. I might be missing something obvious.

Contributor

davedash commented Sep 17, 2015

I have a module which I was using for each environment "./app" now I've split it up into "./app-prod" "./app-stage" etc because some environments use user_data some do not. And the ones that do not I can't pass in a variable that says null. An if statement would allow us to do something like:

if "${var.userdata}" {
    user_data = "${var.userdata}"
}

Of course having a null or something would be good too. I might be missing something obvious.

@thegedge

This comment has been minimized.

Show comment
Hide comment
@thegedge

thegedge Sep 18, 2015

Contributor

I wanted to at least drop in an interesting way of doing if ! x.empty y else empty right now, which we use to choose whether or not we chef provision with a private or public IP:

host = "${replace(self.private_ip, replace(var.bastion_hosts, "/^$/", "/^.*$/"), "")}"

If var.bastion_hosts is empty, we end up replacing all of self.private_ip with an empty string. For the connection host parameter, this means terraform will pick the value for us. If it's non-empty, we search for var.bastion_hosts, which should never occur in self.private_ip, so we get the value of self.private_ip back.

UPDATE:
My assumption that an empty string would have terraform pick the host was wrong, so we had to go full ternary. Here's how you do if foo empty then bar else baz. If foo is a substring of baz this doesn't work.

${replace(replace(var.baz, replace(var.foo, "/^$/", "/^.*$/"), ""), "/^$/", var.bar)}
Contributor

thegedge commented Sep 18, 2015

I wanted to at least drop in an interesting way of doing if ! x.empty y else empty right now, which we use to choose whether or not we chef provision with a private or public IP:

host = "${replace(self.private_ip, replace(var.bastion_hosts, "/^$/", "/^.*$/"), "")}"

If var.bastion_hosts is empty, we end up replacing all of self.private_ip with an empty string. For the connection host parameter, this means terraform will pick the value for us. If it's non-empty, we search for var.bastion_hosts, which should never occur in self.private_ip, so we get the value of self.private_ip back.

UPDATE:
My assumption that an empty string would have terraform pick the host was wrong, so we had to go full ternary. Here's how you do if foo empty then bar else baz. If foo is a substring of baz this doesn't work.

${replace(replace(var.baz, replace(var.foo, "/^$/", "/^.*$/"), ""), "/^$/", var.bar)}
@ketzacoatl

This comment has been minimized.

Show comment
Hide comment
@ketzacoatl

ketzacoatl Sep 18, 2015

Contributor

nice pattern, thank you for sharing! I'll look forward to the day when this is not necessary, but I imagine it'll do for now (the simple use case, that is).

Contributor

ketzacoatl commented Sep 18, 2015

nice pattern, thank you for sharing! I'll look forward to the day when this is not necessary, but I imagine it'll do for now (the simple use case, that is).

@steve-jansen

This comment has been minimized.

Show comment
Hide comment
@steve-jansen

steve-jansen Sep 18, 2015

Contributor

@thegedge thanks for sharing, neat hack

Contributor

steve-jansen commented Sep 18, 2015

@thegedge thanks for sharing, neat hack

@brikis98

This comment has been minimized.

Show comment
Hide comment
@brikis98

brikis98 Oct 10, 2016

Contributor

If you're looking for a way to do if-statements in the meantime, I wrote up a blog post that captures a few of the basic techniques that are possible today: Terraform tips & tricks: loops, if-statements, and gotchas. It's not as nice as having built-in language support for conditionals, but it's surprising just how far you can go with creative use of the count parameter.

Contributor

brikis98 commented Oct 10, 2016

If you're looking for a way to do if-statements in the meantime, I wrote up a blog post that captures a few of the basic techniques that are possible today: Terraform tips & tricks: loops, if-statements, and gotchas. It's not as nice as having built-in language support for conditionals, but it's surprising just how far you can go with creative use of the count parameter.

@cmcconnell1

This comment has been minimized.

Show comment
Hide comment
@cmcconnell1

cmcconnell1 Oct 12, 2016

Thanks @brikis98 for creating the above page. It has been very helpful for me with this issue and others.

Thanks @brikis98 for creating the above page. It has been very helpful for me with this issue and others.

@Joosakur

This comment has been minimized.

Show comment
Hide comment
@Joosakur

Joosakur Oct 19, 2016

Yep, the page by @brikis98 is very helpful, thank you. There are some cases which are not probably covered by these workaround hacks though.

For example, I'd like to include https listener in aws_elb only if the (in that case mandatory) certificate id variable is not empty/null. Since the listener parameter has a type of object array, the string interpolation functions don't seem to apply to it easily.

Yep, the page by @brikis98 is very helpful, thank you. There are some cases which are not probably covered by these workaround hacks though.

For example, I'd like to include https listener in aws_elb only if the (in that case mandatory) certificate id variable is not empty/null. Since the listener parameter has a type of object array, the string interpolation functions don't seem to apply to it easily.

@jgartrel

This comment has been minimized.

Show comment
Hide comment
@jgartrel

jgartrel Nov 2, 2016

@phinze - My kingdom for a case statement:

  # Count = 1 only if 
  #   provider = aws AND nodes != 0 AND length(vpc_security_group_ids) is 0
  count = "${lookup(map("aws", "${signum(var.nodes)}"), "${var.provider}" , "0") * ( signum(length(var.vpc_security_group_ids)) - 1 ) * -1 }"

or a Ternary Operator
and some logic operators

Fantastic thread, and thank you!

As the Oracle would say ... "Oh, don't worry about it. As soon as you step outside that door, you'll start feeling better. You'll remember you don't believe in any of this logic crap. You're in control of your own DSL, remember?

jgartrel commented Nov 2, 2016

@phinze - My kingdom for a case statement:

  # Count = 1 only if 
  #   provider = aws AND nodes != 0 AND length(vpc_security_group_ids) is 0
  count = "${lookup(map("aws", "${signum(var.nodes)}"), "${var.provider}" , "0") * ( signum(length(var.vpc_security_group_ids)) - 1 ) * -1 }"

or a Ternary Operator
and some logic operators

Fantastic thread, and thank you!

As the Oracle would say ... "Oh, don't worry about it. As soon as you step outside that door, you'll start feeling better. You'll remember you don't believe in any of this logic crap. You're in control of your own DSL, remember?

@ahammond

This comment has been minimized.

Show comment
Hide comment
@ahammond

ahammond Nov 7, 2016

I think the ongoing lesson is "Don't write your own DSL. It will suck. You'll have to write a parser and editor/IDE support and those will suck, too." Saltstack was mentioned before in the context of preprocessors, but I think the fundamental lesson there is actually that it's domain specific language isn't a DSL, it is just a data-structure. You can encode that data-structure pretty much however you want (jinja pre-processed yaml is the standard, but there are plenty of other options). The important thing is that it renders down to a data-structure which is de-serialized and then drives the functionality. This seems to be a design paradigm that doesn't suck.

ahammond commented Nov 7, 2016

I think the ongoing lesson is "Don't write your own DSL. It will suck. You'll have to write a parser and editor/IDE support and those will suck, too." Saltstack was mentioned before in the context of preprocessors, but I think the fundamental lesson there is actually that it's domain specific language isn't a DSL, it is just a data-structure. You can encode that data-structure pretty much however you want (jinja pre-processed yaml is the standard, but there are plenty of other options). The important thing is that it renders down to a data-structure which is de-serialized and then drives the functionality. This seems to be a design paradigm that doesn't suck.

@mitchellh

This comment has been minimized.

Show comment
Hide comment
@mitchellh

mitchellh Nov 7, 2016

Member

@ahammond Which is the design paradigm of Terraform: we fully support JSON for this reason and projects out there do use full languages like Ruby to generate Terraform JSON. I'm not sure if you were saying Terraform didn't do this, but I want to make it clear that Terraform is a data structure currently and is not a DSL.

HCL can definitely be viewed as a DSL, but its really just a human-friendly language for writing data structures. We don't support conditionals or other non-data structure features yet so you can't consider it much more than that...

However, I've also learned the lesson (via Vagrant) of "don't use a full programming language" as well because people do really crazy things that make it difficult to safely load and use configuration.

Member

mitchellh commented Nov 7, 2016

@ahammond Which is the design paradigm of Terraform: we fully support JSON for this reason and projects out there do use full languages like Ruby to generate Terraform JSON. I'm not sure if you were saying Terraform didn't do this, but I want to make it clear that Terraform is a data structure currently and is not a DSL.

HCL can definitely be viewed as a DSL, but its really just a human-friendly language for writing data structures. We don't support conditionals or other non-data structure features yet so you can't consider it much more than that...

However, I've also learned the lesson (via Vagrant) of "don't use a full programming language" as well because people do really crazy things that make it difficult to safely load and use configuration.

@ahammond

This comment has been minimized.

Show comment
Hide comment
@ahammond

ahammond Nov 7, 2016

@mitchellh that's awesome! I'm going to take a look into the JSON format! I'm super-new to Terraform, found this issue while looking for a way to iterate that didn't suck.

ahammond commented Nov 7, 2016

@mitchellh that's awesome! I'm going to take a look into the JSON format! I'm super-new to Terraform, found this issue while looking for a way to iterate that didn't suck.

@apparentlymart apparentlymart referenced this issue in hashicorp/hil Dec 4, 2016

Merged

Comparison and Boolean Operations #38

@apparentlymart

This comment has been minimized.

Show comment
Hide comment
@apparentlymart

apparentlymart Dec 9, 2016

Contributor

FYI to those who are following this issue: Terraform 0.8 will have some basic support for conditionals and boolean operations in the interpolation language, giving us some first-class support for conditionals, albeit only inside interpolations for now.

Although we don't yet have an explicit when or if meta-attribute as was discussed above, this new conditional operator makes it easier to achieve that using count:

    # Create this only in production
    count = "${var.environment == "production" ? 1 : 0}"

Seems likely that a hypothetical future meta-attribute for explicitly enabling/disabling resources conditionally would build on the new boolean operators; for now, the ternary conditional operator should hopefully tidy up some of the existing tricks people were doing with setting count conditionally.

Contributor

apparentlymart commented Dec 9, 2016

FYI to those who are following this issue: Terraform 0.8 will have some basic support for conditionals and boolean operations in the interpolation language, giving us some first-class support for conditionals, albeit only inside interpolations for now.

Although we don't yet have an explicit when or if meta-attribute as was discussed above, this new conditional operator makes it easier to achieve that using count:

    # Create this only in production
    count = "${var.environment == "production" ? 1 : 0}"

Seems likely that a hypothetical future meta-attribute for explicitly enabling/disabling resources conditionally would build on the new boolean operators; for now, the ternary conditional operator should hopefully tidy up some of the existing tricks people were doing with setting count conditionally.

@mtougeron

This comment has been minimized.

Show comment
Hide comment
@mtougeron

mtougeron Dec 9, 2016

Contributor

@apparentlymart This is awesome, thanks!

Contributor

mtougeron commented Dec 9, 2016

@apparentlymart This is awesome, thanks!

@ketzacoatl

This comment has been minimized.

Show comment
Hide comment
@ketzacoatl

ketzacoatl Dec 9, 2016

Contributor

Very excited to leverage this with TF 0.8.x, thank you for all the great work here!

Contributor

ketzacoatl commented Dec 9, 2016

Very excited to leverage this with TF 0.8.x, thank you for all the great work here!

joeduffy added a commit to pulumi/pulumi that referenced this issue Dec 12, 2016

Add thoughts on a little language
This change includes a miniature spec for what we'd want out of a little
markup language that extends YAML/JSON with typing and minimal templating.

We've begun to reach the limits of what Go's templates give us; the usability
is quite poor: the order of template expansion is "confused" (as it must
happen before verification of stack properties); it is dumb textual copy-and-
paste, and thus knows nothing about the lexical and semantic rules; evaluation
of expressions that should produce actual objects inserted into the metadata
stream as-is must actually be serialized into text (problematic for the above
reasons); and, finally, as a result of all of this, failure modes are terrible.

But worse than this, we simply can't do what we need in many places.  For
instance, mapping a stack's properties onto the services that it creates works
in simple cases -- like strings, booleans, and ints -- but quickly breaks down
when referencing complex objects (for the same above reasons).  This is why
we've needed to special case property mapping in the aws/x/cf provider, but
clearly this won't generalize to all the compositional situations that arise.

It's worth nothing Hashicorp's HCL/HIL is closest to what we want.  (The
language used for Terraform.)  It isn't exactly what we want, however, for two
reasons.  First, it lacks conditionals and iteration.  This is likely to appear
at some point (see hashicorp/terraform#1604), and
indeed in this past week alone, a new C-like conditional operator (which I
actually don't love) got added to HIL:
hashicorp/hil@5fe4b10.
Second, and perhaps more importantly, its approach is to create a new language.
The design I list here is a natural extension that adds typechecking and
minimal templating to the existing YAML/JSON formats.  As a stand-alone
project, this whould have a much broader appeal.  And whether or not we use it
for Mu depends on whether we really want an entirely new markup language or not.

To cut to the chase, I'm shelving this for now.  I'm going to keep hacking my
way through the current Go templates plus special-casing for now.  My eye is
on the initial end-to-end prototype.  But, no doubt, we'll need to revisit this
immediately afterwards, make a decision, and make it happen.
@aavileli

This comment has been minimized.

Show comment
Hide comment
@aavileli

aavileli Dec 13, 2016

awaiting for TF .8

awaiting for TF .8

@ryno75

This comment has been minimized.

Show comment
Hide comment
@ryno75

ryno75 Dec 15, 2016

Contributor

IT'S HERE! YAHOOOOOOOOOOOOOOOOOOO!!!

Contributor

ryno75 commented Dec 15, 2016

IT'S HERE! YAHOOOOOOOOOOOOOOOOOOO!!!

@stack72

This comment has been minimized.

Show comment
Hide comment
@stack72

stack72 Dec 18, 2016

Contributor

Closing this as it has landed in Terraform 0.8.x :)

Contributor

stack72 commented Dec 18, 2016

Closing this as it has landed in Terraform 0.8.x :)

@stack72 stack72 closed this Dec 18, 2016

@tmatilai

This comment has been minimized.

Show comment
Hide comment
@tmatilai

tmatilai Dec 19, 2016

@stack72 I assume you refer to the new ternary operator? While that covers many of the use cases, it won't help with optionally specifying attributes, or "sub hashes" inside the resources. For example optionally declaring multiple listeners in an ELB. Would be nice to have an issue for tracking those, too.

But great work, the ternary operator for sure makes things easier especially in generic modules!

@stack72 I assume you refer to the new ternary operator? While that covers many of the use cases, it won't help with optionally specifying attributes, or "sub hashes" inside the resources. For example optionally declaring multiple listeners in an ELB. Would be nice to have an issue for tracking those, too.

But great work, the ternary operator for sure makes things easier especially in generic modules!

@apparentlymart

This comment has been minimized.

Show comment
Hide comment
@apparentlymart

apparentlymart Dec 19, 2016

Contributor

Given the age and large potential scope of this issue, I guess it makes sense for us to close it out and capture some more-specific use-cases in other issues, since this one was originally motivated with just collecting information on patterns people were using, and didn't have a tight enough scope that it would ever likely be closed by any real implementation work.

The trick there, of course, is that we previously intentionally consolidated several issues describing other use-cases over here, so there's a bunch of closed issues linked from here that capture some real use-cases we presumably don't want to "lose".

FWIW though, I understand that improved conditional stuff is still on the radar even if this particular issue is closed.

Contributor

apparentlymart commented Dec 19, 2016

Given the age and large potential scope of this issue, I guess it makes sense for us to close it out and capture some more-specific use-cases in other issues, since this one was originally motivated with just collecting information on patterns people were using, and didn't have a tight enough scope that it would ever likely be closed by any real implementation work.

The trick there, of course, is that we previously intentionally consolidated several issues describing other use-cases over here, so there's a bunch of closed issues linked from here that capture some real use-cases we presumably don't want to "lose".

FWIW though, I understand that improved conditional stuff is still on the radar even if this particular issue is closed.

@samstav samstav referenced this issue in samstav/terraform-mailgun-aws Jan 3, 2017

Closed

make MX records optional #1

@oillio

This comment has been minimized.

Show comment
Hide comment
@oillio

oillio Jan 20, 2017

@tmatilai - Would this feature request solve your issue? #7034

oillio commented Jan 20, 2017

@tmatilai - Would this feature request solve your issue? #7034

@tmatilai

This comment has been minimized.

Show comment
Hide comment
@tmatilai

tmatilai Jan 23, 2017

@oillio yeah, as far as I understand, that would cover the sub-resource/block case!

Even after that one use case would be conditional individual attributes, although I haven't personally been so much in need of those compared to sub-resources.

@oillio yeah, as far as I understand, that would cover the sub-resource/block case!

Even after that one use case would be conditional individual attributes, although I haven't personally been so much in need of those compared to sub-resources.

@aaroncaito

This comment has been minimized.

Show comment
Hide comment
@aaroncaito

aaroncaito Feb 26, 2017

Trying to get consistent output of a module using count to dictate the version of a resource to create.

module code:

resource "aws_s3_bucket" "lc_enabled" {
  count = "${var.lifecycle_enabled ? 1 : 0}"
  bucket  = "${var.bucket}"
  acl = "${var.acl}"
  lifecycle_rule {
      id = "log"
      prefix = "log/"
      enabled = true
      transition {
          days = 30
          storage_class = "STANDARD_IA"
      }
      transition {
          days = 60
          storage_class = "GLACIER"
      }
      expiration {
          days = 90
      }
  }
  versioning {
    enabled = "${var.versioning}"
  }
}
resource "aws_s3_bucket" "lc_disabled" {
  count = "${var.lifecycle_enabled ? 0 : 1 }"
  bucket  = "${var.bucket}"
  acl = "${var.acl}"
  versioning {
    enabled = "${var.versioning}"
  }
}

failing output code:

output "bucket.name" {
  value = "${var.lifecycle_enabled == 1 ? aws_s3_bucket.lc_enabled.id : aws_s3_bucket.lc_disabled.id}"
}

Apparently if any ternary values don't exist, the entire ternary will fail silently. the output above does not display at all. However if I'm using lifecycle_enabled = false and change the output to:

output "bucket.name" {
  value = "${var.lifecycle_enabled == 1 ? "something" : aws_s3_bucket.lc_disabled.id}"
}

Here because all these values exist the conditional parses correctly. Is this expected behavior? Ultimately I'm looking for a way to consolidate the output so if I'm overlooking a more direct method that'd be great.

Trying to get consistent output of a module using count to dictate the version of a resource to create.

module code:

resource "aws_s3_bucket" "lc_enabled" {
  count = "${var.lifecycle_enabled ? 1 : 0}"
  bucket  = "${var.bucket}"
  acl = "${var.acl}"
  lifecycle_rule {
      id = "log"
      prefix = "log/"
      enabled = true
      transition {
          days = 30
          storage_class = "STANDARD_IA"
      }
      transition {
          days = 60
          storage_class = "GLACIER"
      }
      expiration {
          days = 90
      }
  }
  versioning {
    enabled = "${var.versioning}"
  }
}
resource "aws_s3_bucket" "lc_disabled" {
  count = "${var.lifecycle_enabled ? 0 : 1 }"
  bucket  = "${var.bucket}"
  acl = "${var.acl}"
  versioning {
    enabled = "${var.versioning}"
  }
}

failing output code:

output "bucket.name" {
  value = "${var.lifecycle_enabled == 1 ? aws_s3_bucket.lc_enabled.id : aws_s3_bucket.lc_disabled.id}"
}

Apparently if any ternary values don't exist, the entire ternary will fail silently. the output above does not display at all. However if I'm using lifecycle_enabled = false and change the output to:

output "bucket.name" {
  value = "${var.lifecycle_enabled == 1 ? "something" : aws_s3_bucket.lc_disabled.id}"
}

Here because all these values exist the conditional parses correctly. Is this expected behavior? Ultimately I'm looking for a way to consolidate the output so if I'm overlooking a more direct method that'd be great.

@apparentlymart

This comment has been minimized.

Show comment
Hide comment
@apparentlymart

apparentlymart Feb 26, 2017

Contributor

Hi @aaroncaito!

That issue is being tracked over in hashicorp/hil#50. It's something I'd like to fix before too long since I know it's annoying and makes the conditional operator not as useful as it could otherwise be. It requires some adjustment to how the interpolation syntax evaluator works, so it's not a quick fix but something that is on my radar to take a look at.

Contributor

apparentlymart commented Feb 26, 2017

Hi @aaroncaito!

That issue is being tracked over in hashicorp/hil#50. It's something I'd like to fix before too long since I know it's annoying and makes the conditional operator not as useful as it could otherwise be. It requires some adjustment to how the interpolation syntax evaluator works, so it's not a quick fix but something that is on my radar to take a look at.

@aaroncaito

This comment has been minimized.

Show comment
Hide comment
@aaroncaito

aaroncaito Feb 26, 2017

thanks @apparentlymart in this case it'd be even better if we didn't need unique resource names when only one will be created. my output "thing.attribute" {value = "resource.SHARED_NAME.attribute"}. with no conditional eval required. I'll track that other issue as well as that will help other use cases.

aaroncaito commented Feb 26, 2017

thanks @apparentlymart in this case it'd be even better if we didn't need unique resource names when only one will be created. my output "thing.attribute" {value = "resource.SHARED_NAME.attribute"}. with no conditional eval required. I'll track that other issue as well as that will help other use cases.

@s-urbaniak s-urbaniak referenced this issue in coreos/tectonic-installer Feb 28, 2017

Closed

openstack: code sharing between nova and neutron #5

@pdecat

This comment has been minimized.

Show comment
Hide comment
@pdecat

pdecat Mar 2, 2017

Contributor

My workaround for the above S3 with LC enabled/disabled use case:

output "bucket.name" {
  value = "${coalesce(join("", aws_s3_bucket.lc_enabled.*.id), join("", aws_s3_bucket.lc_disabled.*.id))}"
}

Edit: the coalesce is superfluous:

output "bucket.name" {
  value = "${join("", aws_s3_bucket.lc_enabled.*.id, aws_s3_bucket.lc_disabled.*.id)}"
}

However, this usage of join() with multiple lists is not documented, so I'm not sure it is a good idea:

join(delim, list) - Joins the list with the delimiter for a resultant string.

https://www.terraform.io/docs/configuration/interpolation.html#join_delim_list_

Contributor

pdecat commented Mar 2, 2017

My workaround for the above S3 with LC enabled/disabled use case:

output "bucket.name" {
  value = "${coalesce(join("", aws_s3_bucket.lc_enabled.*.id), join("", aws_s3_bucket.lc_disabled.*.id))}"
}

Edit: the coalesce is superfluous:

output "bucket.name" {
  value = "${join("", aws_s3_bucket.lc_enabled.*.id, aws_s3_bucket.lc_disabled.*.id)}"
}

However, this usage of join() with multiple lists is not documented, so I'm not sure it is a good idea:

join(delim, list) - Joins the list with the delimiter for a resultant string.

https://www.terraform.io/docs/configuration/interpolation.html#join_delim_list_

@tamsky

This comment has been minimized.

Show comment
Hide comment
@tamsky

tamsky Mar 9, 2017

Contributor

@pdecat your workaround is similar to another workaround I saw recently:
#12453 (comment)

using join to turn the lists into strings to get past the type check

Contributor

tamsky commented Mar 9, 2017

@pdecat your workaround is similar to another workaround I saw recently:
#12453 (comment)

using join to turn the lists into strings to get past the type check

@gerph

This comment has been minimized.

Show comment
Hide comment
@gerph

gerph Sep 12, 2017

I have a case where I want to be able to define the AWS instance type in a variable, and - along side it - the size of the root volume. Since on AWS only EBS volumes can have a size, this becomes a problem. If I specify the instance type as a type that doesn't have EBS backed storage then no value will work for the volume_size, and AWS will always complain that you cannot do that.

That is...

resource "aws_instance" "foo" {
  ...
  root_block_device {
    volume_size = "0"
  }
}

(applies to both aws_instance, and aws_opsworks_instance, although it's strange that they don't share the code)

My workaround? Change the terraform code so that if the parameters within root_block_device are ineffective (ie 0 in the case of the size) the block device isn't added to the definition, and AWS is happy. It's not great, but recompiling the tool isn't a big deal as I've already got a number of patches applied.

gerph commented Sep 12, 2017

I have a case where I want to be able to define the AWS instance type in a variable, and - along side it - the size of the root volume. Since on AWS only EBS volumes can have a size, this becomes a problem. If I specify the instance type as a type that doesn't have EBS backed storage then no value will work for the volume_size, and AWS will always complain that you cannot do that.

That is...

resource "aws_instance" "foo" {
  ...
  root_block_device {
    volume_size = "0"
  }
}

(applies to both aws_instance, and aws_opsworks_instance, although it's strange that they don't share the code)

My workaround? Change the terraform code so that if the parameters within root_block_device are ineffective (ie 0 in the case of the size) the block device isn't added to the definition, and AWS is happy. It's not great, but recompiling the tool isn't a big deal as I've already got a number of patches applied.

@glasser

This comment has been minimized.

Show comment
Hide comment
@glasser

glasser Sep 12, 2017

Contributor

@gerph I am likely to do a similar hack to allow a single aws_launch_configuration to either have name or name_prefix — ie, so that normally my LCs can have autogenerated names but I can pin to an old name for a rollback if necessary.

Contributor

glasser commented Sep 12, 2017

@gerph I am likely to do a similar hack to allow a single aws_launch_configuration to either have name or name_prefix — ie, so that normally my LCs can have autogenerated names but I can pin to an old name for a rollback if necessary.

@arkaprava-jana

This comment has been minimized.

Show comment
Hide comment
@arkaprava-jana

arkaprava-jana Oct 4, 2017

I have two variables lob and role. As of now it is like this:

target = "${var.role == "hzc" ? "TCP:11410" : "HTTP:80/"}"

But my requirement is:

if lob==anything and role==hzc then "TCP:11410

else if lob==kolk and role==kata then "HTTPS:443/index.html"

else "TCP:8443"

How to map this in conditional statement?

I have two variables lob and role. As of now it is like this:

target = "${var.role == "hzc" ? "TCP:11410" : "HTTP:80/"}"

But my requirement is:

if lob==anything and role==hzc then "TCP:11410

else if lob==kolk and role==kata then "HTTPS:443/index.html"

else "TCP:8443"

How to map this in conditional statement?

@yfronto

This comment has been minimized.

Show comment
Hide comment
@yfronto

yfronto Oct 4, 2017

Contributor

@arkaprava-jana I assume by "anything" you mean "is defined" and not the text "anything"? You could nest conditionals to get something like:

target ="${var.role == "hzc" && var.lob != "" ? "TCP:11410" : var.lob == "kolk" && var.role == "kata" ? "HTTPS:443/index.html" : "TCP:8443"}"
Contributor

yfronto commented Oct 4, 2017

@arkaprava-jana I assume by "anything" you mean "is defined" and not the text "anything"? You could nest conditionals to get something like:

target ="${var.role == "hzc" && var.lob != "" ? "TCP:11410" : var.lob == "kolk" && var.role == "kata" ? "HTTPS:443/index.html" : "TCP:8443"}"
@arkaprava-jana

This comment has been minimized.

Show comment
Hide comment
@arkaprava-jana

arkaprava-jana Oct 4, 2017

Yes you are right. Looking for nested conditional example only.

Yes you are right. Looking for nested conditional example only.

@armenr

This comment has been minimized.

Show comment
Hide comment
@armenr

armenr Oct 26, 2017

I'm trying to get clever with the ternary operator and conditionals, but I think I have the wrong idea about them. I have the following code in an ec2 module I've built on top of the ec2_instance resource.

I have a var I've created called "total_instances".
I've set: count = "${var.total_instances}"

I've tried the following:

Name = "${count.index} = 1 ? {var.instance_name}-${var.environment} : {var.instance_name}${count.index+1}-${var.environment}}"

I've also tried:

Name = "${var.total_instances} = 1 ? {var.instance_name}-${var.environment} : {var.instance_name}${count.index+1}-${var.environment}}"

If the count of instances I pass to the module = 1, then I just want the instance to be tagged as "server-ops", but if the count I pass from my main.tf to the module is greater than 1, I want it to assign "server1-ops" as the name.

Any help or clarification would be very helpful, thank you!

armenr commented Oct 26, 2017

I'm trying to get clever with the ternary operator and conditionals, but I think I have the wrong idea about them. I have the following code in an ec2 module I've built on top of the ec2_instance resource.

I have a var I've created called "total_instances".
I've set: count = "${var.total_instances}"

I've tried the following:

Name = "${count.index} = 1 ? {var.instance_name}-${var.environment} : {var.instance_name}${count.index+1}-${var.environment}}"

I've also tried:

Name = "${var.total_instances} = 1 ? {var.instance_name}-${var.environment} : {var.instance_name}${count.index+1}-${var.environment}}"

If the count of instances I pass to the module = 1, then I just want the instance to be tagged as "server-ops", but if the count I pass from my main.tf to the module is greater than 1, I want it to assign "server1-ops" as the name.

Any help or clarification would be very helpful, thank you!

@jaygorrell

This comment has been minimized.

Show comment
Hide comment
@jaygorrell

jaygorrell Oct 26, 2017

@armenr I think the problem is yourr syntax... it should be ${condition ? true : false} all in one evaluation block. You may also want to use the format() interpolation to combine strings and variables easier but in this case the simplest is probably to only use the ternary to determine if the count is needed or not:
${var.instance_name}${var.total_instances > 1 ? count.index + 1 : ""}-${var.environment}

A format() example would be:
${var.total_instances > 1 ? format("%s-%02d-%s", var.instance_name, (count.index + 1), var.environment) : format("%s-%s", var.instance_name, var.environment)}

@armenr I think the problem is yourr syntax... it should be ${condition ? true : false} all in one evaluation block. You may also want to use the format() interpolation to combine strings and variables easier but in this case the simplest is probably to only use the ternary to determine if the count is needed or not:
${var.instance_name}${var.total_instances > 1 ? count.index + 1 : ""}-${var.environment}

A format() example would be:
${var.total_instances > 1 ? format("%s-%02d-%s", var.instance_name, (count.index + 1), var.environment) : format("%s-%s", var.instance_name, var.environment)}

@armenr

This comment has been minimized.

Show comment
Hide comment
@armenr

armenr Oct 26, 2017

@jaygorrell - Thanks! I'll fiddle with this and let you know how that works out. Thank you!

armenr commented Oct 26, 2017

@jaygorrell - Thanks! I'll fiddle with this and let you know how that works out. Thank you!

@megakoresh

This comment has been minimized.

Show comment
Hide comment
@megakoresh

megakoresh Mar 27, 2018

I don't know if I am late to the party, but I am currently facing similar issues to what is being discussed here:

One use case that does not work with the current implementation is assigning pre-allocated Floating IPs vs allocating them. The scenario is like this:

Production has a fixed amount of servers with pre-allocated IP addresses that need to remain the same all the time.

Testing has the same infrastructure but can have more/less servers and Floating IPs are allocated and released on demand.

(I use openstack as an example here, but I imagine there are quite many parallel scenarios regardless of platform)
The following does not work:

resource "openstack_networking_floatingip_v2" "tf_instance_ips" {
  count = "${length(var.floating_ips) > 0 ? 0 : var.count}"
  pool  = "${var.floating_ip_pool_name}"
}

resource "openstack_compute_floatingip_associate_v2" "tf_instance_ips" {
  count       = "${var.count}"
  floating_ip = "${length(var.floating_ips) > 0 ? element(var.floating_ips, count.index) : element(openstack_networking_floatingip_v2.tf_instance_ips.*.address, count.index)}"
  instance_id = "${element(openstack_compute_instance_v2.tf_instance.*.id, count.index)}"
}

This will throw
element() may not be used with an empty list
because of #11210
However even when that is fixed, it is much easier to read and cleaner to have something like this:

if "${length(var.floating_ips) > 0}" {
  resource "openstack_compute_floatingip_associate_v2" "tf_instance_ips" {
    count       = "${var.count}"
    floating_ip = "${element(var.floating_ips, count.index)}"
    instance_id = "${element(openstack_compute_instance_v2.tf_instance.*.id, count.index)}"
  }
}

if "${length(openstack_networking_floatingip_v2.tf_instance_ips.*.address) > 0}" {
  resource "openstack_networking_floatingip_v2" "tf_instance_ips" {
    count = "${length(var.floating_ips) > 0 ? 0 : var.count}"
    pool  = "${var.floating_ip_pool_name}"
  }
  resource "openstack_compute_floatingip_associate_v2" "tf_instance_ips" {
    count       = "${var.count}"
    floating_ip = "${element(openstack_networking_floatingip_v2.tf_instance_ips.*.address, count.index)}"
    instance_id = "${element(openstack_compute_instance_v2.tf_instance.*.id, count.index)}"
  }
}

PS: The workaround for the above use case is to use coalescelist function, but it is sub-optimal for example in cases where type of the input value changes conditionally or when depending on condition different sets of resources need to be spawned, while remaining resources are not different.

resource "openstack_compute_floatingip_associate_v2" "tf_instance_ips" {
  count       = "${var.count}"
  floating_ip = "${element(coalescelist(var.floating_ips, openstack_networking_floatingip_v2.tf_instance_ips.*.address), count.index)}"
  instance_id = "${element(openstack_compute_instance_v2.tf_instance.*.id, count.index)}"
}

megakoresh commented Mar 27, 2018

I don't know if I am late to the party, but I am currently facing similar issues to what is being discussed here:

One use case that does not work with the current implementation is assigning pre-allocated Floating IPs vs allocating them. The scenario is like this:

Production has a fixed amount of servers with pre-allocated IP addresses that need to remain the same all the time.

Testing has the same infrastructure but can have more/less servers and Floating IPs are allocated and released on demand.

(I use openstack as an example here, but I imagine there are quite many parallel scenarios regardless of platform)
The following does not work:

resource "openstack_networking_floatingip_v2" "tf_instance_ips" {
  count = "${length(var.floating_ips) > 0 ? 0 : var.count}"
  pool  = "${var.floating_ip_pool_name}"
}

resource "openstack_compute_floatingip_associate_v2" "tf_instance_ips" {
  count       = "${var.count}"
  floating_ip = "${length(var.floating_ips) > 0 ? element(var.floating_ips, count.index) : element(openstack_networking_floatingip_v2.tf_instance_ips.*.address, count.index)}"
  instance_id = "${element(openstack_compute_instance_v2.tf_instance.*.id, count.index)}"
}

This will throw
element() may not be used with an empty list
because of #11210
However even when that is fixed, it is much easier to read and cleaner to have something like this:

if "${length(var.floating_ips) > 0}" {
  resource "openstack_compute_floatingip_associate_v2" "tf_instance_ips" {
    count       = "${var.count}"
    floating_ip = "${element(var.floating_ips, count.index)}"
    instance_id = "${element(openstack_compute_instance_v2.tf_instance.*.id, count.index)}"
  }
}

if "${length(openstack_networking_floatingip_v2.tf_instance_ips.*.address) > 0}" {
  resource "openstack_networking_floatingip_v2" "tf_instance_ips" {
    count = "${length(var.floating_ips) > 0 ? 0 : var.count}"
    pool  = "${var.floating_ip_pool_name}"
  }
  resource "openstack_compute_floatingip_associate_v2" "tf_instance_ips" {
    count       = "${var.count}"
    floating_ip = "${element(openstack_networking_floatingip_v2.tf_instance_ips.*.address, count.index)}"
    instance_id = "${element(openstack_compute_instance_v2.tf_instance.*.id, count.index)}"
  }
}

PS: The workaround for the above use case is to use coalescelist function, but it is sub-optimal for example in cases where type of the input value changes conditionally or when depending on condition different sets of resources need to be spawned, while remaining resources are not different.

resource "openstack_compute_floatingip_associate_v2" "tf_instance_ips" {
  count       = "${var.count}"
  floating_ip = "${element(coalescelist(var.floating_ips, openstack_networking_floatingip_v2.tf_instance_ips.*.address), count.index)}"
  instance_id = "${element(openstack_compute_instance_v2.tf_instance.*.id, count.index)}"
}
@mthak

This comment has been minimized.

Show comment
Hide comment
@mthak

mthak Jun 5, 2018

how do i get something like this in terraform
Properties:
{% if PostgresSnapshot is defined and RDSSnapshot != "" %}
DBSnapshotIdentifier:
Ref: PostgresSnapshot2
{% else %}
DBName: {{ DbName |default('sample', true) }}
MasterUsername:
Ref: MasterU
MasterUserPassword:
Ref: MasterUserPW
{% endif %}

mthak commented Jun 5, 2018

how do i get something like this in terraform
Properties:
{% if PostgresSnapshot is defined and RDSSnapshot != "" %}
DBSnapshotIdentifier:
Ref: PostgresSnapshot2
{% else %}
DBName: {{ DbName |default('sample', true) }}
MasterUsername:
Ref: MasterU
MasterUserPassword:
Ref: MasterUserPW
{% endif %}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment