Confusing error message "map of object is required" or "list of object is required" for a variable default value

[apparentlymart]

Terraform Version

Terraform v0.12.0-alpha1

Terraform Configuration Files

variable "example" {
  default = {
    "foo" = ["a", "b", "c"]
    "bar" = ["d", "e"]
    "baz" = []
  }
}

Expected Behavior

When encountering a variable block without an explicit type argument, Terraform should use the type of the default value to infer the variable's type.

In the above case, the expected inferred type would be map(list(string)).

Actual Behavior

Officially, Terraform versions prior to v0.12 only supported flat lists and maps for variable values. However, due to some implementation bugs in validation it was possible to introduce more complex data structures as variable defaults, including structures that would violate the usual rule that all elements of a list or map must be of the same type. While this usually led to downstream errors while working with that invalid expression, certain simple expressions could also pass validation.

Terraform v0.12 now does more complete validation of variable values in order to improve the user experience for callers of modules, allowing for more precise and actionable error messages to be returned. For compatibility with existing usage, Terraform v0.12 attempts to infer a reasonable type specification for any default value, but in v0.12-alpha1 the type inference mechanism is not able to correctly infer more complex structures like lists of lists, maps of lists, etc.

When inference fails, Terraform may produce a confusing error message such as "object is required", even though the value appears to be an object. When Terraform presents this message, what it means to say is that the object it found is not of the expected type, which results from Terraform being unable to decide on a suitable object type to use for validation.

Workaround

To work around this for v0.12-alpha1 testing, set an explicit type argument to avoid the need for the automatic type detection heuristic:

variable "example" {
  type = map(list(string))

  default = {
    "foo" = ["a", "b", "c"]
    "bar" = ["d", "e"]
    "baz" = []
  }
}

As long as the given default can be converted to the specified type, the default value should be accepted. Note that the map, list and set type kinds require all of their elements to be of the same type. For more information on the 0.12 language type system, see the Types and Values draft documentation.

[apparentlymart]

Over in #19278 we can see a different variation of this problem: Terraform was able to infer a type from the default successfully but it inferred an overly-specific type. In that case, it inferred an object with an exact set of keys rather than a map(string) as the user intended.

While our usual behavior for type inference is to choose the most precise type that matches the value, I think the requirements for variable here deserve a slightly different behavior since we are using the default value as an example of what type is intended, and so the exact type of the example will often be too specific.

Perhaps then the appropriate behavior is to run an additional step at the end of the normal type inference if the normal inferencer selected a tuple or object type: if all of the elements can be unified to a single element type, simplify the inferred type to be a list or map type instead.

For example:

• ["a", "b"] would be initially inferred as tuple([string,string]). All of the elements types are already the same, so we can simplify to list(string).
• ["a", true] would be initially inferred as tuple([string,bool]). Since bool can always convert to string, the unifier would tell us to simplify this to list(string) and then convert the second element to "true".
• ["a", []] would be initially inferred as tuple([string, tuple([])]). Since string and tuple types have no types in common, unification would fail and the inferred type would be retained exactly as-is.

The effect of this is that users that are intending to use the new structural types will need to define type in many cases in order to specify exactly what is expected, while existing configurations that are presuming the v0.11 type system will infer closer to what the author presumably intended.

Since prior versions of Terraform did not support these structural types at all, it may be best for us to require a typed-by-example variable to reduce to a collection type and produce a good error message if not explaining how to set the type argument to express the exact type desired. That way we can avoid silently choosing a more general type than a module author intended. When we work on this issue, we should experiment with the different permuations here with some real-world examples and see which behavior seems to achieve the best results.

[apparentlymart]

I did some more testing after #19690 is merged and it seems as if that has actually addressed the core problems here. Looking more closely now, I realize that the type unification improvements in cty have fixed the rough edges here by allowing Terraform to find a suitable type without any special additional rules.

With that said, then, I'm going to close this out now and we'll make a fresh issue if we find more obscure problems in this area during further testing.

To verify this I used a child module with the following content:

variable "list" {
  default = ["a", "b", 1]
}
variable "map" {
  default = {
    a = "a"
    b = 1
    c = "c"
  }
}
variable "list_of_map" {
  default = [
    {},
    {
      a = 1
      b = "b"
      c = "c"
    },
  ]
}

I then called it with some values that have similar-but-not-identical types:

module "child" {
  source = "./child"

  list = [1, "b"]
  map = {
    a = 1
    b = "b"
  }
  list_of_map = [
    {
      a = 1
      b = "b"
    }
  ]
}

Terraform now accepts this without error.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.