Implementing Hashicorp Vault Transit encryption

[binlab]

Are you considering adding the Hashicorp Vault Transit backend for state file encryption as an alternative to symmetric encryption with a static passcode?

[dee-kryvenko]

Pardon me responding with a question, but are you considering using this backend as a long term state management solution? I'm not opposing to that, just asking. If we're talking Git-based state management long term, adding more encryption features would make total sense. For the time being I was considering this approach as an interim solution to the common chicken-egg problem (either using it only temporarily or using it only for 0-ring stacks such as s3+dynamodb itself). From that standpoint, Vault would not exist yet anyway. So I guess to answer your question we need to understand if there's any interest in using this backend as a long term state management solution.

[binlab]

Yes, sure. You are totally right about the chicken-egg problem. But there is another problem with Terraform - lack of client-side encryption for state file. This proposal was created four years ago (2016) and still not implemented. Although all know everything about the problem of data leakage through the state file. Strange.

On my mind use Git as a backend actually good idea, it cheap, available not only in clouds, give versioning from the box. But all this only subject to strong encryption (RSA 4096+), a closed repository, and right ACL. Subject to all these conditions, it is in no way inferior to any of the others. And finally, Git behind SSH with certificate-based authorization even more securely than others.

Regarding encryption via Vault Transit and chicken/egg problem, this can be easily solved.
We can create Vault by Terraform and encrypt it by a static passphrase, store it to Vault and when use Vault Transit for other elements of infrastructure and this solves the issue of collaboration. In addition, through Vault, we can provide fast secure authorization in SSH and Git server for state and code separately.

[dee-kryvenko]

Good to know there is interest in using this backend type for long term operations!

Stepping aside from the chicken-egg problem then, let's assume Vault is there and available for us from the get-go. Why not just use it natively as a backend?

I just wanted to make sure I'm not wasting time implementing redundant feature. I'm totally onboard with having more options for encrypting state at rest with this backend, and I'm not quite happy with a shared symmetric key being the only option atm - but maybe there's other options worth looking into apart from Vault.

[dee-kryvenko]

I just stumbled upon https://github.com/mozilla/sops - sounds like this is something we can use under the hood and it supports Vault Transit, KMS, PGP and more. I haven't looked into the code yet, but it's a Go module so it should be possible. Seems to lack some syntax sugar getsops/sops#903 but at this point I don't know if it's a blocker or just means more work.

[dee-kryvenko]

Ok it's been a long way coming, but I have finally found some time for this. Added initial integration with sops. For now I added only 3 bindings - PGP, AWS KMS and Hashicorp Vault Transit. sops does more than that, and it will be easy to integrate with it, but I personally do not have a use case for it at the moment, nor - active environment to test it with.

@binlab on the off-chance you are still interested in this project two years later, please check out latest release https://github.com/plumber-cd/terraform-backend-git/releases/tag/v0.1.0.