adding wildcard interpolation #6351

mzupan · 2016-04-26T15:27:30Z

This adds a wildcard search for lookup() function

Usage

variable "cidr" {
  default = {
    prod = "10.10.0.0/16"
    "*" = "10.11.0.0/16"
  }
}

resource "aws_vpc" "main" {
  cidr_block = "${lookup(var.cidr, var.env)}"
  enable_dns_hostnames = true
  enable_dns_support = true
}

Example output

$ ./terraform plan -var 'env=prod'
Refreshing Terraform state prior to plan...

+ aws_vpc.main
    cidr_block:                "" => "10.10.0.0/16"
    default_network_acl_id:    "" => "<computed>"
    default_security_group_id: "" => "<computed>"
    dhcp_options_id:           "" => "<computed>"
    enable_classiclink:        "" => "<computed>"
    enable_dns_hostnames:      "" => "1"
    enable_dns_support:        "" => "1"
    instance_tenancy:          "" => "<computed>"
    main_route_table_id:       "" => "<computed>"

Then a search on the wildcard

$ ./terraform plan -var 'env=dev'

+ aws_vpc.main
    cidr_block:                "" => "10.11.0.0/16"
    default_network_acl_id:    "" => "<computed>"
    default_security_group_id: "" => "<computed>"
    dhcp_options_id:           "" => "<computed>"
    enable_classiclink:        "" => "<computed>"
    enable_dns_hostnames:      "" => "1"
    enable_dns_support:        "" => "1"
    instance_tenancy:          "" => "<computed>"
    main_route_table_id:       "" => "<computed>"

I think the test I included passes.. But feel free to implement this as you see. This references #6284 for use in 0.7

If this isn't the correct branch let me know.

This replaces this plugin system with the extracted hashicorp/go-plugin library. This doesn't introduce any new features such as binary flattening but opens us up to that a lot more easily and removes a lot of code from TF in favor of the upstream lib. This will introduce a protocol change that will cause all existing plugins to have to be recompiled to work properly. There is no actual API changes so they just have to recompile, but it is technically backwards incompatible.

This introduces the terraform state list command to list the resources within a state. This is the first of many state management commands to come into 0.7. This is the first command of many to come that is considered a "plumbing" command within Terraform (see "plumbing vs porcelain": http://git.661346.n2.nabble.com/what-are-plumbing-and-porcelain-td2190639.html). As such, this PR also introduces a bunch of groundwork to support plumbing commands. The main changes: - Main command output is changed to split "common" and "uncommon" commands. - mitchellh/cli is updated to support nested subcommands, since terraform state list is a nested subcommand. - terraform.StateFilter is introduced as a way in core to filter/search the state files. This is very basic currently but I expect to make it more advanced as time goes on. - terraform state list command is introduced to list resources in a state. This can take a series of arguments to filter this down. Known issues, or things that aren't done in this PR on purpose: - Unit tests for terraform state list are on the way. Unit tests for the core changes are all there.

I decided to split this up from the terraform state rm command to make the diff easier to see. These changes will also be used for terraform state mv. This adds a `Remove` method to the `*terraform.State` struct. It takes a list of addresses and removes the items matching that list. This leverages the `StateFilter` committed last week to make the view of the world consistent across address lookups. There is a lot of test duplication here with StateFilter, but in Terraform style: we like it that way.

This adds a field terraform_version to the state that represents the Terraform version that wrote that state. If Terraform encounters a state written by a future version, it will error. You must use at least the version that wrote that state. Internally we have fields to override this behavior (StateFutureAllowed), but I chose not to expose them as CLI flags, since the user can just modify the state directly. This is tricky, but should be tricky to represent the horrible disaster that can happen by enabling it. We didn't have to bump the state format version since the absense of the field means it was written by version "0.0.0" which will always be older. In effect though this change will always apply to version 2 of the state since it appears in 0.7 which bumped the version for other purposes.

… into the main terraform binary

This commit rectifies the fact that the original binary state is referred to as V1 in the source code, but the first version of the JSON state uses StateVersion: 1. We instead make the code refer to V0 as the binary state, and V1 as the first version of JSON state.

This commit adds the groundwork for supporting module outputs of types other than string. In order to do so, the state version is increased from 1 to 2 (though the "public-facing" state version is actually as the first state file was binary). Tests are added to ensure that V2 (1) state is upgraded to V3 (2) state, though no separate read path is required since the V2 JSON will unmarshal correctly into the V3 structure. Outputs in a ModuleState are now of type map[string]interface{}, and a test covers round-tripping string, []string and map[string]string, which should cover all of the types in question. Type switches have been added where necessary to deal with the interface{} value, but they currently default to panicking when the input is not a string.

This changes the representation of maps in the interpolator from the dotted flatmap form of a string variable named "var.variablename.key" per map element to use native HIL maps instead. This involves porting some of the interpolation functions in order to keep the tests green, and adding support for map outputs. There is one backwards incompatibility: as a result of an implementation detail of maps, one could access an indexed map variable using the syntax "${var.variablename.key}". This is no longer possible - instead HIL native syntax - "${var.variablename["key"]}" must be used. This was previously documented, (though not heavily used) so it must be noted as a backward compatibility issue for Terraform 0.7.

sl1pm4t · 2016-05-08T23:47:02Z

This will be a useful addition.

gozer · 2016-05-26T16:22:00Z

@jen20 PR just closed, no merges? Curious as to what is going on with this much desired feature?

astlock · 2016-11-19T22:17:15Z

+1 to the previous comment!

astlock · 2016-11-20T11:25:34Z

It seems I found a simple solution, and correct me if I am wrong, but there is no need for any additional PR to use "wildcard interpolation."
Usage:

variable "amis" {
  default = {
    "prod"  = "ami-6134f33"
    "default" = "default-ami"
  }
}

resource "aws_launch_configuration" "after" {
  image_id              = "${lookup(var.amis, var.env, var.amis["default"])}"
...

or the same with "*"

variable "amis" {
  default = {
    "prod"  = "ami-6134f33"
    "*" = "default-ami"
  }
}

resource "aws_launch_configuration" "after" {
  image_id              = "${lookup(var.amis, var.env, var.amis["*"])}"
...

So, with the third argument in lookup function it will return default/*/your value if no key is found in the map.

mzupan · 2016-11-20T13:44:20Z

It just got implemented differently in 0.7

lookup(var.amis, var.env, 'Default-AMI')

ghost · 2020-04-19T02:35:06Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

mitchellh and others added 17 commits April 24, 2016 18:29

deps: Vendor github.com/hashicorp/go-plugin

6cbdc21

terraform: fix some issues around filtering single counts

a161445

add command/state show

4b5da61

website: update docs for state show

4c2208a

deps: vendor columnize

a607092

command/state: pattern => address

6cf50b7

Implemented internal plugin calls; which allows us to compile plugins…

c757eca

… into the main terraform binary

deps: Update github.com/hashicorp/hil/...

b13b722

core: Add Cobbler provider to internal plugin list

8b53139

adding wildcard interpolation

5417812

mzupan mentioned this pull request Apr 26, 2016

adding a * catchall for var interpolation #6284

Closed

stack72 added enhancement core labels Apr 26, 2016

jen20 force-pushed the dev-0.7 branch 3 times, most recently from 63ef309 to 39c4b95 Compare May 3, 2016 22:35

jen20 force-pushed the dev-0.7 branch from 18f25c0 to f1d099b Compare May 5, 2016 23:16

jen20 closed this May 10, 2016

hashicorp locked and limited conversation to collaborators Apr 19, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

adding wildcard interpolation #6351

adding wildcard interpolation #6351

mzupan commented Apr 26, 2016

sl1pm4t commented May 8, 2016

gozer commented May 26, 2016

astlock commented Nov 19, 2016

astlock commented Nov 20, 2016 •

edited

mzupan commented Nov 20, 2016

ghost commented Apr 19, 2020

adding wildcard interpolation #6351

adding wildcard interpolation #6351

Conversation

mzupan commented Apr 26, 2016

sl1pm4t commented May 8, 2016

gozer commented May 26, 2016

astlock commented Nov 19, 2016

astlock commented Nov 20, 2016 • edited

mzupan commented Nov 20, 2016

ghost commented Apr 19, 2020

astlock commented Nov 20, 2016 •

edited