Language design: take a look at Nix lang

[danbst]

Though I understand it is too late to make any language design changes, but take a look at Nix language on how it solved the problem of "configuration lang" + "template lang". Basically, it defined both the same.

This allowed to substitute "templates" with "functions", make "variables" (let-bindings in Nix terminology), have JSONish dictionaries, simple syntax and so on.

For example, here are examples from README.md translated to Nix language. You can see that it is slightly more verbose than HCL, a lot less verbose than JSON, but has a builtin function declaration, named function arguments, namespaces and interpolation.

{
  io_mode = "async";
  service.http.web_proxy = {
    listen_addr = "127.0.0.1:8080";
    process.main = {
      command = [ "/usr/local/bin/awesome-app" "server" ];
    };
    process.mgmt = {
      command = [ "/usr/local/bin/awesome-app" "mgmt" ];
    };
  };
}

and

{ addend, upper, name }:
rec {
  # Arithmetic with literals and application-provided variables
  sum = 1 + addend;

  # String interpolation and templates
  message = "Hello, ${name}!";

  # Application-provided functions
  shouty_message = upper message;
}

Nix lang proved to be very useful both for package specification and configuration. For example, take a look at Consul package expression file: https://github.com/NixOS/nixpkgs/blob/32340793aafec24dcef95fee46a21e634dd63457/pkgs/servers/consul/default.nix and system profile configuration: https://github.com/NixOS/nixpkgs/blob/32340793aafec24dcef95fee46a21e634dd63457/nixos/modules/profiles/hardened.nix

[apparentlymart]

Hi @danbst! Thanks for sharing this.

We are familiar with this Nix format and indeed it has some interesting similarities and differences to HCL. The main constraint on the design of HCL2 was existing use of HCL1+HIL, so there's a limit to how much direct inspiration we could take from other similar languages, but I think HCL2's expression language in particular has similar capabilities to the one in the Nix language.

A notable omission in HCL2 in comparison to Nix language is a native syntax for declaring functions, which is intentionally not built in since it's a lot of extra mandatory weight on the language that would apply to all calling applications. However, there is an experimental extension package that allows applications to opt-in to allowing users to define functions without extending the syntax -- it just uses the normal HCL block/argument constructs to form a function definition.

Making the application responsible for wiring up this mechanism allows it to be integrated with other concepts in the calling application. For example, while there are no immediate-term plans to offer user-defined functions in Terraform, to do so effectively would require Terraform itself to be aware of the functions so that they could interact well with Terraform's concept of modules, and have a Terraform-native-feeling way to share functions between different modules. I don't yet know exactly what that would look like, but having the functions syntax built directly into the language would limit the degree to which calling applications could adapt the configuration conventions to fit their needs.

We use GitHub issues for tracking bugs and enhancements, so since this issue doesn't represent something specific we can act on I'm going to close it out. However, I do appreciate you sharing these links and your thoughts on the Nix language!