[BUG] Terramate panics when using ternary expression with tm_hcl_expression() function

[ottramst]

Describe the bug

Terramate seems to panic when running generate with a ternary expression mixed with the tm_hcl_expression() function.

Example:

globals {
  database = global.module_rds_enabled ? tm_hcl_expression("module.rds.db_instance_database_name") : tm_hcl_expression("module.rds_aurora.cluster_database_name")
}

To Reproduce

Use the next configuration

terramate {
  required_version = "~> 0.1.42"

  config {
    git {
      check_untracked   = false
      check_uncommitted = false
      check_remote      = false
    }
  }
}

globals {
  module_rds_enabled = true

  database = global.module_rds_enabled ? tm_hcl_expression("module.rds.db_instance_database_name") : tm_hcl_expression("module.rds_aurora.cluster_database_name")
}

generate_hcl "_test.tf" {
  content {
    database = global.database
  }
}

stack {
  name = "test-stack"
}

and run terramate generate

Expected behavior

The above expression should return database = module.rds.db_instance_database_name when global.module_rds_enabled = true or database = module.rds_aurora.cluster_database_name when global.module_rds_enabled = false

Log Output

Logs from panic:

panic: cannot produce tokens for customdecode.ExpressionVal(&hclsyntax.ScopeTraversalExpr{Traversal:hcl.Traversal{hcl.TraverseRoot{isTraverser:hcl.isTraverser{}, Name:"module", SrcRange:hcl.Range{Filename:"<generated-hcl>\x00module.rds_aurora.cluster_database_name", Start:hcl.Pos{Line:1, Column:1, Byte:0}, End:hcl.Pos{Line:1, Column:7, Byte:6}}}, hcl.TraverseAttr{isTraverser:hcl.isTraverser{}, Name:"rds_aurora", SrcRange:hcl.Range{Filename:"<generated-hcl>\x00module.rds_aurora.cluster_database_name", Start:hcl.Pos{Line:1, Column:7, Byte:6}, End:hcl.Pos{Line:1, Column:18, Byte:17}}}, hcl.TraverseAttr{isTraverser:hcl.isTraverser{}, Name:"cluster_database_name", SrcRange:hcl.Range{Filename:"<generated-hcl>\x00module.rds_aurora.cluster_database_name", Start:hcl.Pos{Line:1, Column:18, Byte:17}, End:hcl.Pos{Line:1, Column:40, Byte:39}}}}, SrcRange:hcl.Range{Filename:"<generated-hcl>\x00module.rds_aurora.cluster_database_name", Start:hcl.Pos{Line:1, Column:1, Byte:0}, End:hcl.Pos{Line:1, Column:40, Byte:39}}})

goroutine 1 [running]:
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x164a628?, 0xc00019d140?}}, {0x14aba00?, 0xc00085f170?}}, {0xc000af6c00, 0x14, 0x20})
        github.com/hashicorp/hcl/v2@v2.14.1/hclwrite/generate.go:298 +0x120b
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x164a820?, 0xc00085f250?}}, {0x14dac80?, 0xc000609b30?}}, {0xc000c4b680, 0x10, 0x10})
        github.com/hashicorp/hcl/v2@v2.14.1/hclwrite/generate.go:284 +0xf93
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x164a820?, 0xc00085f260?}}, {0x14dac80?, 0xc000609b90?}}, {0xc000c4b680, 0xc, 0x10})
        github.com/hashicorp/hcl/v2@v2.14.1/hclwrite/generate.go:284 +0xf93
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x164a820?, 0xc00085f280?}}, {0x14dac80?, 0xc000609bf0?}}, {0xc000025ae0, 0x4, 0x4})
        github.com/hashicorp/hcl/v2@v2.14.1/hclwrite/generate.go:284 +0xf93
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x164a820?, 0xc000ac4970?}}, {0x14dac80?, 0xc000bb1aa0?}}, {0x0, 0x0, 0x0})
        github.com/hashicorp/hcl/v2@v2.14.1/hclwrite/generate.go:284 +0xf93
github.com/hashicorp/hcl/v2/hclwrite.TokensForValue({{{0x164a820?, 0xc000ac4970?}}, {0x14dac80?, 0xc000bb1aa0?}})
        github.com/hashicorp/hcl/v2@v2.14.1/hclwrite/generate.go:24 +0x35
github.com/mineiros-io/terramate/hcl/eval.(*engine).evalVar(0xc000a00768)
        github.com/mineiros-io/terramate/hcl/eval/partial.go:901 +0x41a
github.com/mineiros-io/terramate/hcl/eval.(*engine).evalIdent(0xc000a00768)
        github.com/mineiros-io/terramate/hcl/eval/partial.go:609 +0x136
github.com/mineiros-io/terramate/hcl/eval.(*engine).evalExpr(0xc000a00768)
        github.com/mineiros-io/terramate/hcl/eval/partial.go:354 +0x1278
github.com/mineiros-io/terramate/hcl/eval.(*engine).Eval(0xc000a00768)
        github.com/mineiros-io/terramate/hcl/eval/partial.go:115 +0xee
github.com/mineiros-io/terramate/hcl/eval.(*Context).PartialEval(0xc0000153c8, {0x164a548?, 0xc0000ad0e0?})
        github.com/mineiros-io/terramate/hcl/eval/eval.go:126 +0x230
github.com/mineiros-io/terramate/generate/genhcl.copyBody(0xc000502090?, 0xc000426dc0, {0x164a5f0, 0xc0000153d0})
        github.com/mineiros-io/terramate/generate/genhcl/genhcl.go:310 +0x84b
github.com/mineiros-io/terramate/generate/genhcl.appendBlock(0xc000a012d0?, 0xc0001506c0, {0x164a5f0, 0xc0000153d0})
        github.com/mineiros-io/terramate/generate/genhcl/genhcl.go:338 +0xa8
github.com/mineiros-io/terramate/generate/genhcl.copyBody(0x164a740?, 0xc000426f20, {0x164a5f0, 0xc0000153d0})
        github.com/mineiros-io/terramate/generate/genhcl/genhcl.go:322 +0xa9a
github.com/mineiros-io/terramate/generate/genhcl.Load(0xc000304780?, {{0xc000037770, 0x46}, {0xc0006ed500, 0x6d, 0x6d}}, {0x164c798?, 0xc000741dc0}, 0x1?)
        github.com/mineiros-io/terramate/generate/genhcl/genhcl.go:226 +0xad4
github.com/mineiros-io/terramate/generate.loadStackCodeCfgs(0xc00055e948?, {{0xc000037770, 0x46}, {0xc0006ed500, 0x6d, 0x6d}}, 0x164c798?, 0xc000741dc0?)
        github.com/mineiros-io/terramate/generate/generate.go:963 +0xe7
github.com/mineiros-io/terramate/generate.Do.func1({{0xc000037770, 0x46}, {0xc0006ed500, 0x6d, 0x6d}}, 0xc000741dc0, 0xc0007f5e00?)
        github.com/mineiros-io/terramate/generate/generate.go:159 +0x46e
github.com/mineiros-io/terramate/generate.forEachStack(0xc000304790?, {0xc000320840, 0x56}, 0xc000a036a0)
        github.com/mineiros-io/terramate/generate/generate.go:686 +0xacd
github.com/mineiros-io/terramate/generate.Do(0xc0006fe098, {0xc000320840?, 0x1f4?})
        github.com/mineiros-io/terramate/generate/generate.go:138 +0x85
github.com/mineiros-io/terramate/cmd/terramate/cli.(*cli).generate(0xc0006fe000, {0xc000320840?, 0x0?})
        github.com/mineiros-io/terramate/cmd/terramate/cli/cli.go:600 +0x4a
github.com/mineiros-io/terramate/cmd/terramate/cli.(*cli).run(0xc0006fe000)
        github.com/mineiros-io/terramate/cmd/terramate/cli/cli.go:392 +0x4d8
github.com/mineiros-io/terramate/cmd/terramate/cli.Exec({0xc0001ac010, 0x1, 0x1}, {0x1647ca0, 0xc0001a0000}, {0x1647cc0, 0xc0001a0008}, {0x1647cc0, 0xc0001a0010})
        github.com/mineiros-io/terramate/cmd/terramate/cli/cli.go:202 +0xdd
main.main()
        github.com/mineiros-io/terramate/cmd/terramate/main.go:30 +0x78

Environment (please complete the following information):

• OS: Mac OSX
• OS Version: 12.6 (21G115)
• Git Version: git version 2.37.1 (Apple Git-137.1)
• Terramate Version: 0.1.42

[katcipis]

Thanks for taking the time to explain your issue @ottramst.

This is a known (but not well documented) limitation. Globals today are always about fully evaluated information, there is no partial evaluation on them, given that they can't hold arbitrary un-evaluated expressions as their final value. The problem is not related to ternary, even this simple example will fail with a similar error:

globals {
  data = tm_hcl_expression("expr")
}

generate_hcl "test.tf" {
  content {
    data = global.data
  }
}

We will be reviewing globals behavior soon, but for now I will only document better the contexts where tm_hcl_expression can be used safely. We should also remove the function from the global evaluation context, so you are going to get a much nicer "function doesn't exist" error than the not so friendly panic.

Given the provided example, if it is similar to what you are trying to achieve, this should work:

globals {
  module_rds_enabled = true

  database = global.module_rds_enabled ? "module.rds.db_instance_database_name" : "module.rds_aurora.cluster_database_name"
}

generate_hcl "_test.tf" {
  content {
    database = tm_hcl_expression(global.database)
  }
}

stack {
  name = "test-stack"
}