Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terraform 0.12.1 trates "null" as literal value instead of using default variable #21702

Closed
bohdanyurov-gl opened this issue Jun 12, 2019 · 33 comments

Comments

@bohdanyurov-gl
Copy link

bohdanyurov-gl commented Jun 12, 2019

Terraform Version

Terraform v0.12.1
+ provider.acme v1.3.0
+ provider.archive v1.2.2
+ provider.aws v2.11.0
+ provider.local v1.2.2
+ provider.null v2.1.2
+ provider.random v2.1.2
+ provider.template v2.1.2
+ provider.tls v2.0.1

Terraform Configuration Files

module "instance" {
  source = "./instance"

  foo = null
}

variable "foo" {
  default = "bar"
}

Debug Output

Crash Output

Expected Behavior

Default value is used

Actual Behavior

image
image

Steps to Reproduce

Additional Context

References

https://www.terraform.io/docs/configuration/expressions.html

Finally, there is one special value that has no type:

null: a value that represents absence or omission. If you set an argument of a resource or module to null, Terraform behaves as though you had completely omitted it — it will use the argument's default value if it has one, or raise an error if the argument is mandatory. null is most useful in conditional expressions, so you can dynamically omit an argument if a condition isn't met.
@apparentlymart
Copy link
Member

apparentlymart commented Jun 12, 2019

Hi @bohdanyurov-gl! Sorry for this inconsistency, and thanks for reporting it.

The key subtlety which that text is implying but not explicitly stating is that count is not "an argument of a resource"... it is a so-called "meta-argument" which is handled by Terraform Core itself, and has some special behaviors associated with it due to how it affects the construction of the graphs Terraform uses.

In particular, the presence of count cannot be conditional because its presence causes references to the resource to produce a sequence of objects rather than a single object, and so the meaning of downstream references (which is analyzed and validated statically) depends on whether it is set.

I assume your goal here was for count to be zero if the variable is null. If so, one way to write that is coalesce(var.iam_enable_ssm_access, false) to provide a default value of false when that variable is not set.

We'll use this issue to represent finding a way to be more explicit about this requirement in the documentation. Thanks again for reporting this!

@jcarlson
Copy link

jcarlson commented Aug 20, 2019

I'm having a similar issue, but I think I'm using a null value more inline with what the documentation says.

$ terraform -v
Terraform v0.12.5
+ provider.alks v1.3.0
+ provider.archive v1.2.0
+ provider.aws v2.21.1
+ provider.external v1.2.0
+ provider.null v2.1.0
+ provider.random v2.2.0
+ provider.template v2.1.0
# main.tf
variable "env" {
  default = "dev"
  type    = "string"
}

module "rds" {
  source = "./modules/rds"

  instance_class = var.env == "prod" ? "db.m5.large" : null
}
# modules/rds/main.tf
variable "instance_class" {
  default = "db.t2.small"
  type    = string
}

resource "aws_db_instance" "main" {
  instance_class = var.instance_class
}
Error: "instance_class": required field is not set

  on ../modules/rds/main.tf line 15, in resource "aws_db_instance" "main":
  15: resource "aws_db_instance" "main" {

So in this case, it looks like Terraform is taking the null value provided to the rds module and using it as a literal value, all the way to the aws_db_instance resource, rather than picking up the default value specified in the sub-module.

Is this expected behavior?

EDIT: my intent here is to say, "if this is production, use a larger instance, otherwise, whatever default the module provides is fine"

@theneva
Copy link

theneva commented Aug 28, 2019

I encountered something that might be related:

I have this a Google Cloud SQL instance that may be either zonal or regional based on a bool var.is_regional:

resource "google_sql_database_instance" "sql_instance" {
  #
  settings {
	#
    location_preference {
      zone = var.is_regional ? null : var.cluster_location
    }
  }
}

When var.is_regional is set to true, Terraform does not seem to set a preferred zone.

The resource was automatically assigned the zone europe-west1-d by Google Cloud. However, Terraform treats the null in the config as a literal value(?), so now I always get this diff when running terraform plan or terraform apply:

~ location_preference {
  - zone = "europe-west1-d" -> null
}

I don't know how to best handle this case. The best outcome for me would be to remove the whole block if var.is_regional is true, but I don't believe that's possible.

@idrennanvmware
Copy link

idrennanvmware commented Sep 5, 2019

We are seeing the same behavior on vSphere as @theneva is reporting Terraform keeps thinking the resource has changed from the sane defaults

`
- boot_delay = 0 -> null
boot_retry_delay = 10000
- boot_retry_enabled = false -> null
- cpu_performance_counters_enabled = false -> null
- cpu_reservation = 0 -> null
- custom_attributes = {} -> null
- efi_secure_boot_enabled = false -> null
- enable_disk_uuid = false -> null
- enable_logging = false -> null
- extra_config = {} -> null

'

Small sample of the attributes affected, above.

Edit: Turns out that this was all linked to a null value (like above) being passed to Organization Name and Full_Name (this appears to be new behavior in TF 0.12 but I will confirm) which caused the resource to be destroyed and recreated. Setting those variables instead of defaults resolved the issue.

windows_options { organization_name = "${var.win_organization_name}" full_name = "${var.win_full_name}" <Snip> }

@gp42
Copy link

gp42 commented Oct 8, 2019

I'm having a similar issue, but I think I'm using a null value more inline with what the documentation says.

$ terraform -v
Terraform v0.12.5
+ provider.alks v1.3.0
+ provider.archive v1.2.0
+ provider.aws v2.21.1
+ provider.external v1.2.0
+ provider.null v2.1.0
+ provider.random v2.2.0
+ provider.template v2.1.0
# main.tf
variable "env" {
  default = "dev"
  type    = "string"
}

module "rds" {
  source = "./modules/rds"

  instance_class = var.env == "prod" ? "db.m5.large" : null
}
# modules/rds/main.tf
variable "instance_class" {
  default = "db.t2.small"
  type    = string
}

resource "aws_db_instance" "main" {
  instance_class = var.instance_class
}
Error: "instance_class": required field is not set

  on ../modules/rds/main.tf line 15, in resource "aws_db_instance" "main":
  15: resource "aws_db_instance" "main" {

So in this case, it looks like Terraform is taking the null value provided to the rds module and using it as a literal value, all the way to the aws_db_instance resource, rather than picking up the default value specified in the sub-module.

Is this expected behavior?

EDIT: my intent here is to say, "if this is production, use a larger instance, otherwise, whatever default the module provides is fine"

yup, experiencing similar behaviour while trying to use module defaults. Any suggestions to this 🤷‍♂ ? (tf v0.12.9)

@krzysztof-magosa
Copy link

krzysztof-magosa commented Oct 30, 2019

@theneva I believe you could use dynamic and write such condition in for_each block so it returns empty list in case you don't want block to be created. Hacky but could work.

@theneva
Copy link

theneva commented Oct 30, 2019

Clever! I'm currently checking out Pulumi (because of this class of problems, and support for Kubernetes custom resource definitions), but might give it a shot if we decide to stick with Terraform 😄

@scodeman
Copy link

scodeman commented Nov 20, 2019

Hi @bohdanyurov-gl! Sorry for this inconsistency, and thanks for reporting it.

The key subtlety which that text is implying but not explicitly stating is that count is not "an argument of a resource"... it is a so-called "meta-argument" which is handled by Terraform Core itself, and has some special behaviors associated with it due to how it affects the construction of the graphs Terraform uses.

In particular, the presence of count cannot be conditional because its presence causes references to the resource to produce a sequence of objects rather than a single object, and so the meaning of downstream references (which is analyzed and validated statically) depends on whether it is set.

I assume your goal here was for count to be zero if the variable is null. If so, one way to write that is coalesce(var.iam_enable_ssm_access, false) to provide a default value of false when that variable is not set.

We'll use this issue to represent finding a way to be more explicit about this requirement in the documentation. Thanks again for reporting this!

Hi @apparentlymart, I think this issue should still be labelled as "bug" and not "documentation" as this is happening on regular arguments as shown in @bohdanyurov-gl foo example ( independently of count).

Example

#test.tf
module instance {
  source = "./instance"
  foo = null
}
#instance/variables.tf
variable foo {
  default = "bar"
}
#instance/main.tf
resource "null_resource" instance_foo {
  provisioner "local-exec" {
    command = "echo ${var.foo}"
  }
}

gives the following error after terraform apply on test.tf

module.instance.null_resource.instance_foo: Creating...

Error: Invalid template interpolation value: The expression result is null. Cannot include a null value in a string template.

@kevinburke1
Copy link
Contributor

kevinburke1 commented Dec 18, 2019

I can reproduce this with the use case in the comment immediately above mine. I'm working around it with

variable "image" {
  type = string
  // when this issue is resolved, set this to the actual default image we want
  // https://github.com/hashicorp/terraform/issues/21702
  default = ""
}

And then inline in the module definition

    image = coalesce(var.image, "ubuntu/bionic")

or whatever default value you want to actually use.

@mahsoud
Copy link

mahsoud commented Feb 13, 2020

reproduced with Terraform v0.12.20

@AntonChernysh
Copy link

AntonChernysh commented Feb 14, 2020

I'm having issues where terraform is not omitting property when "null" returned from lookup() like this: ip_restriction = lookup(site_config.value, "ip_restriction", null)
This construction works fine for all string properties and not for this particular property that expects a list of objects.
This property successfully ignored when I set it's value to "null" (without lookup).
Is this related?

@dimitriydotsenko
Copy link

dimitriydotsenko commented Feb 25, 2020

reproduced with Terraform v0.12.21

@baskicom
Copy link

baskicom commented Mar 18, 2020

Is there a fix / workaround ?

@ankurbhardwaj87
Copy link

ankurbhardwaj87 commented Mar 24, 2020

When running terraform plan on version0.12.23,i am facing following issue,default value is being replaced with null. This was working fine in terraform0.11.11
Current
{
- action = "allow"
- cidr_block = "172.16.12.0/23"
- from_port = 0
- icmp_code = 0
- icmp_type = 0
- ipv6_cidr_block = ""
- protocol = "-1"
- rule_no = 106
- to_port = 0
},

New
+ {
+ action = "allow"
+ cidr_block = "172.16.12.0/23"
+ from_port = 0
+ icmp_code = null
+ icmp_type = null
+ ipv6_cidr_block = null
+ protocol = "-1"
+ rule_no = 106
+ to_port = 0
},

@scodeman
Copy link

scodeman commented Mar 25, 2020

@jbardin, @bohdanyurov-gl , @apparentlymart - could we set it back to "bug" as described above

@lucas-caylent
Copy link

lucas-caylent commented Apr 2, 2020

i see the same behavior on 0.12.24 :

logging_config {
    bucket          = var.s3_bucket_access_logs == "" ? null : var.s3_bucket_access_logs
    prefix          = var.s3_access_logs_prefix == "" ? null : var.s3_access_logs_prefix
    include_cookies = var.log_cookies == "" ? null : var.log_cookies
  }

return Error: "default_cache_behavior.0.lambda_function_association.0.lambda_arn": required field is not set

with all three variables defaulted to ""

and same behavior if the assignment is direct

bucket          = var.s3_bucket_access_logs == "" ? null : var.s3_bucket_access_logs

with values defaulted to null

@sean-abbott
Copy link

sean-abbott commented May 15, 2020

On 0.12.25, expanding a bit on @scodeman 's response. I'm trying to use this to safely wrap a base module with a few differences. A minimal example:

$ tree /tmp/terraform_minimum_case/
/tmp/terraform_minimum_case/
├── base
│   └── main.tf
└── wrapper
    └── main.tf
$ cat terraform_minimum_case/base/main.tf 
resource "null_resource" base {
  provisioner "local-exec" {
    command = "echo hello ${var.foo}"
  }
}

variable foo {
  default = "world"
}

$ cat terraform_minimum_case/wrapper/main.tf 
module "wrapped_base" {
  source = "/tmp/terraform_minimum_case/base"

  foo = var.bar == "" ? null : var.bar
}

variable bar {
  default = ""
}

$ terraform apply
Error: Invalid template interpolation value: The expression result is null. Cannot include a null value in a string template.
$ terraform apply -var 'bar=mundo'
module.wrapped_base.null_resource.base (local-exec): Executing: ["/bin/sh" "-c" "echo hello mundo"]
module.wrapped_base.null_resource.base (local-exec): hello mundo

This also happens for required resources, such as the aws_ami data resource. (i.e. Null values are not allowed for this attribute value. when the default is clearly set in the base module as I have this set up)

This seems to be what null is supposed to be for, so this does indeed seem like a bug.

@saguziel
Copy link

saguziel commented Sep 18, 2020

This is still broken on 0.13.2

@GeoffMillerAZ
Copy link

GeoffMillerAZ commented Sep 29, 2020

This is still broken on 0.13.3

@nwsparks
Copy link

nwsparks commented Dec 3, 2020

Is this solved by module_variable_optional_attrs in the 0.14 release here? https://github.com/hashicorp/terraform/releases/tag/v0.14.0

@thllxb
Copy link

thllxb commented Dec 3, 2020

Is this solved by module_variable_optional_attrs in the 0.14 release here? https://github.com/hashicorp/terraform/releases/tag/v0.14.0

I dont think so. That's a different feature for a different purpose.

@ifenglin
Copy link

ifenglin commented Jan 25, 2021

Still broken in v0.13.5

@AbrohamLincoln
Copy link

AbrohamLincoln commented Feb 2, 2021

Still an issue in 0.14.5.

michcio1234 added a commit to datarevenue-berlin/OpenMLOps that referenced this issue Feb 15, 2021
When null is passed to a submodule, Terraform uses this value (null)
instead of submodule's default.
hashicorp/terraform#21702
@latacora-tomekr
Copy link

latacora-tomekr commented Mar 5, 2021

This is also more of an issue with the support for optional variables where those variables are set to null if not provided.

@matt-byrne
Copy link

matt-byrne commented Apr 9, 2021

This is still an issue in 0.14.10, and I don't see any notes about it in any of the 0.15 RC builds so safe to assume it will still be an issue there too.

@ricardobf
Copy link

ricardobf commented Apr 22, 2021

Still an issue in 0.15.0.

@mark-hubers
Copy link

mark-hubers commented May 27, 2021

I think the trick around this problem with null is doing something like this with a mix of thing in your local to do part of the conditional operator that deal with the var that can be null.

local {
   create_backup_kms_key   = var.enable_backups && var.backup_kms_key_id == null
}

resource "aws_kms_key" "backup" {
  count = local.create_backup_kms_key ? 1 : 0
  ...
}

@matt-byrne
Copy link

matt-byrne commented May 27, 2021

@mark-hubers That would only be a workaround if you wanted to maintain a different block for every possible combination of variables that could be null or not.

This issue is about being able to nullify individual input variables and have them use their defaults.

@scodeman
Copy link

scodeman commented Jun 18, 2021

@jbardin, @bohdanyurov-gl , @apparentlymart - could we set it back to "bug" as described above

Re

@jbardin
Copy link
Member

jbardin commented Jun 18, 2021

Thanks @scodeman, The behavior related to module varibales is tracked in #24142, though because it is a breaking change it cannot be altered within any Terraform 1.x releases. The issue here is tagged as documention, as the meta arguments do work as designed.

@brunojcm-vgw
Copy link

brunojcm-vgw commented Jun 18, 2021

The issue is reported for 2 years, and now we heard "it's too late for 1.0", this is so frustrating. 1.0 is exactly where we would expect to see such a silly mistake fixed, but it seems no thorough review of all open issues that would be a breaking change was performed.

let's hope this is fixed in the next 2 years for TF 2.0

@jbardin
Copy link
Member

jbardin commented Jul 12, 2021

Since the initial exchange here was around confusion about null values being applied to boolean operators, conditional expressions, or count itself, and the new error output is quite specific about the values being passed in, I'm going to close this issue.

For those who want to follow the issue about the behavior of null when passed to modules, that will be tracked in #24142.

Thanks!

@jbardin jbardin closed this as completed Jul 12, 2021
@github-actions
Copy link

github-actions bot commented Aug 12, 2021

I'm going to lock this issue because it has been closed for 30 days . This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests