Support HCL files

[sixfeetover]

Would love to see support for HCL. HCL is used for a lot of configuration files as part of the Hashicorp stack (Vault, Terraform, etc).

[jvehent]

We don't use HCL at Mozilla, so it's unlikely we'll write support for it. However, we're happy to help review patches from the community.

[negz]

Note that all HCL can be represented as JSON (i.e. you can configure at least Terraform using JSON rather than HCL). We use sops to decrypt JSON files and pass those to Terraform, i.e.:

terraform apply -var-file=<(sops -d secret.tfvars.json)

There are also some HCL to JSON (and vice versa) converters around that might help workaround this use case.

[carlpett]

@sixfeetover @negz As another Terraform solution, I just created https://github.com/carlpett/terraform-sops, which lets you use a sops-encrypted file as a data source

[sixfeetover]

@carlpett Oh, very clever 👍

[jvehent]

This is awesome! @autrilla: could we host this under sops/terraform and add a use-case examples in the examples folder?

[autrilla]

@jvehent yeah, that'd be great, as long as @carlpett is OK with having their work go in our repo.

[starkers]

I'd really like to use sops for .tfvars files specifically..

Although yes its perfectly possible to implement a fancy hcl2json > sops process its just ugly and because of json we loose the ability to put comments in etc..

[carlpett]

@autrilla I'd be perfectly fine with that, but I think a better goal would be to have it hosted in the "official third-party" organization terraform-providers. That way, it'll be possible to download automatically, rather than doing a manual install.

@starkers I actually started off by looking at that, but found this approach seemed easier to use to me. But maybe I missed something! How was your intended workflow?

[negz]

@starkers Take a look at #292 (comment) - Terraform will happily load JSON encoded tfvars files.

From https://github.com/hashicorp/hcl:

The API for HCL allows JSON as an input so that it is also machine-friendly (machines can generate JSON instead of trying to generate HCL).

[autrilla]

@carlpett yeah, that sounds better. Do you know what the process looks like to get it included there?

[so0k]

@carlpett thanks for sharing, as we were already using mozilla/sops - your provider allows our infra team to easily git clone and terraform apply without having to worry about custom make rules or tfvars

I did have to do the following to get this working on OSX:

go get github.com/carlpett/terraform-sops
mv $GOPATH/bin/terraform-sops $HOME/.terraform.d/plugins/terraform-provider-sops

[abeluck]

@carlpett any progress on getting terraform-sops hosted with terraform-providers? What needs to be done to make that happen?

[carlpett]

Thanks for poking on this @abeluck, I had let it slip for quite a while. Back when this discussion was started, the "Terraform Provider Development Program" was just about to start, and there was no clear way to enroll in it. I see now though that it is possible to sign up via a web form, so I'll get on it :)
If Hashicorp agrees it is useful, there will be a bit of reviewing and adapting things (mainly repo structure, I think) so they mesh well with their automation.

[binlab]

It will be nice to have

[robertwe]

hi everyone, do we have any update on this issue?

[abeluck]

@robertwe

sops has no builtin support for HCL files.

If you're using terraform you can use @carlpett's terraform-provider-sops to load sops files (json or yaml) into terraform. It works very well.

Also shout out to terragrunt which lets you decrypt sops files as part of your terraform+terragrunt workflow.

[robertwe]

@abeluck thx for info. I'm aware of that. I also considered this option.
There is also a possibility to load yaml file directly in terraform.
The deal is I was searching for some super simple solution.