HCL expressions

[traut]

Description

Investigate and document the extent to which the HCL expressions are supported in Fabric configuration files.

For example, some of the features might be:

• HCL-native templating
• loops
• variables
• ...

[Andrew-Morozko]

Consider using more of hcl.EvalContext. May simplify current ref parsing/eval and local contexts

[traut]

I think this issue should be about clarification -- what HCL features are supported by the parser out of the box?

[Andrew-Morozko]

We have all of the hcl native features working, here's an example:

document "example" {
  data inline "hcl_expression_showcase" {
    arithmetic = "1 + 2 = ${1 + 2}"
    // "arithmetic": "1 + 2 = 3",

    logic = "true and false is ${true && false}"
    // "logic": "true and false is false",

    conditionals = "2 is ${2 % 2 == 0 ? "even" : "odd"}"
    // "conditionals": "2 is even"

    // technically, this is a tuple
    loop_over_list = [ for el in [1, 2, 3]: el * 2 ]
    // "loop_over_list": [
    //     2,
    //     4,
    //     6
    // ],

    loop_over_tuple = [ for el in [1, "two", 3]: "value is ${el}" ]
    // "loop_over_tuple": [
    //     "value is 1",
    //     "value is two",
    //     "value is 3"
    // ],

    // technically, this is an object
    loop_over_map = [ for k, v in {"a": 1, "b": 2, "c": 3}: "key ${k}: value ${v}" ]
    // "loop_over_map": [
    //     "key a: value 1",
    //     "key b: value 2",
    //     "key c: value 3"
    // ],

    loop_over_object = [ for k, v in {"a": 1, "b": "two", "c": 3}: "key ${k}: value ${v}" ]
    // "loop_over_object": [
    //     "key a: value 1",
    //     "key b: value two",
    //     "key c: value 3"
    // ],

    loop_creating_object = { for v in [1, 2, 3]: "${v}" => v * 2 }
    // "loop_creating_object": {
    //     "1": 2,
    //     "2": 4,
    //     "3": 6
    // },

    loop_with_filter = { for v in [1, 2, 3, 4]: "${v}" => v * 2 if v % 2 == 0 }
    // "loop_with_filter": {
    //     "2": 4,
    //     "4": 8
    // },

    loop_with_grouping = { for v in [1, 2, 3, 4]: (v%2 == 0 ? "evens" : "odds") => v... }
    // "loop_with_grouping": {
    //     "evens": [
    //         2,
    //         4
    //     ],
    //     "odds": [
    //         1,
    //         3
    //     ]
    // },

    splat_expression = ([
      {
        id: 1,
        name: "foo",
      },
      {
        id: 2,
        name: "bar",
      },
      {
        id: 3,
        name: "baz",
      },
    ])[*].name
    // "splat_expression": [
    //     "foo",
    //     "bar",
    //     "baz"
    // ],

    template_directives = <<EOT
%{~ for v in [1, 2, 3, 4] }
%{~ if v % 2 == 0 ~}
${v} is even
%{ else ~}
${v*2} is doubled ${v}
%{ endif ~}
%{ endfor ~}
EOT
    // "template_directives": "2 is doubled 1\n2 is even\n6 is doubled 3\n4 is even\n"
  }
}

The only function that we're currently exposing is from_env_variable:

  data inline "hcl_function_showcase" {
    my_home = from_env_variable("HOME")
    // "my_home": "/Users/andrew",
  }

We can trivially extend our list with 82 functions by including cty stdlib or we can write our own, it's reasonably straightforward stuff.

The main problem is that we have practically no way to give the user the ability to reference queries/data block values/etc, our whole system is built to expose them to jq or go templates. #73 was the last time I attempted the switchover to a fully hcl-based logic, but it spiraled out of my control and out of reasonable scope 😓

[traut]

oh wow, this is pretty cool!

Do the loops work with blocks as well? We are reinventing them with #142 because, as you said, we rely on the internal data structure, but it is a separate logical step.

We have 3 stages where mutations can happen:

• HCL parsing: mutations by HCL native features listed above. Only the template and the environment are known, no data is fetched.
• Data preparation: mutation with dynamic blocks (Dynamic blocks #142) based on the data. The template, the environment, and the data required for the template are known.
• Content rendering: mutation with dependencies (Dependencies between content blocks #98). The template, the environment, the data required for the template, and the content produced (partially) are known.

Every stage has its pros and cons and can be utilized independently from the others, giving us quite the flexibility to adapt the templates / documents on the fly.

[Andrew-Morozko]

Do the loops work with blocks as well?

No, they aren't built-in into hcl itself, but (I assume) this hcl extention is responsible for the dynamic blocks in hashicorp products. IDK how easy they would be to integrate into our existing system, but it's a possibility worth considering.

[traut]

Understood, thank you! I'll add a doc page about it and close this issue