-
Notifications
You must be signed in to change notification settings - Fork 9.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature request: Vault backend #16066
Comments
/cc @seanherron @consultantRR @apparentlymart |
hello @jamiehannaford that's actually idea i used to do so over credhub (credhub is a vault like made for cloud foundry PaaS ecosystem). You can found it here: https://github.com/orange-cloudfoundry/terraform-secure-backend I've actually no bad feeling about it and I would be happy if someone can point me issues by doing that that. This can give some ideas and framework to do the same with vault. |
Hi @jamiehannaford, Storing the state directly in Vault is indeed something that was discussed before, and is in principle possible. Vault unfortunately lacks some other primitives that are required to implement a "good" backend, such as locking. There were also some concerns that Vault is not really intended for storing "large" objects, and state data can end up being quite large. This concern also applies somewhat to Consul, which has a value size limit, but due to the encryption this limit can be hit sooner and at a less-predictable time. So with all of that said, a Vault backend is possible in principle but these limitations led to encryption being proposed as an orthogonal idea in #9556. We're not closed to the idea of a Vault backend, but worried that it may not meet expectations and thus might not be as useful as expected. |
Thanks for the great comment @apparentlymart! Understanding the context here is super useful. Do you think it's better to close this in favour of #9556? Maybe we could leave this for posterity, at least until an encryption solution is implemented. |
Given that we don't yet have a specific path forward -- just a few proposals -- I think we can leave this issue here to represent this particular proposal, including the tradeoffs around it. When we get to solving this problem (still planned, but not something we're working on immediately) it'll be useful to have this context. |
This might be a good use case for the transit engine. Don't worry about trying to store the values in Vault, just have Vault manage encrypting the blob. |
I read (well, likely skimmed) that ticket in looking into this and failed to grasp that. I will comment no further! Thank you. |
I wouldn't mind handling locking through consul with state files in vault. That's how we run our vault servers already (we use consul for HA and s3 for durability). |
Nice idea! Any moves in this direction? |
A lock mechanism can easily be implemented in vault with versioned secret backends. On an apply, write the value |
I replied this in #31539, however github did not mention here. The idea of vault not being built for large resources a bit of a silly argument. How large can a state file get ? And why should Vault be limited in size. If I'm going to use Vault for storing secrets/sensitive data, it should cover all use cases. State file can't possibly be GB or even 10s of MB ? To me the response doesn't seem valid. We should be allowed to use Vault as a backend (And I understand in priniciple i possible). If Vault is not an option, then Hashicorp did wrong. This may seem like strong criticism, but think about this objectively. |
I agree with @tjad If there's a reason a size limit should be/is implemented, I'm sure Terraform could be coded to work around that (e.g. a "state-part1", "state-part2"..."state-partN" kind of situation could be a decent way to work around any size limits that Vault or Consul might have.) I would think that storing state as an encrypted blob could be a fairly easy integration for a Terraform+Vault integration, and as @eedwards-sk said, Vault could offload the locking to Consul and work around it, along with the size limits. There's even an HTTP Response Code that Vault's REST API could use for this case if Consul has the record locked, and Terraform could handle that the same way it handles any other state file lock. (Though, I realize that that means updating Vault and Terraform, which expands the scope of this request). I, for one, would love to see this feature implemented. Has there been any movement on it? |
From what I've read, I still don't understand if it's possible to store the state of |
Is it possible to add vault as a backend to terraform? Currently we have consul and etcd, which can work with vault, but nothing exists out of the box as an encrypted data backend. This was already suggested here in a thread which discusses how secrets/sensitive data should be stored in state files. It seems that the terraform provider allows you to populate arbitrary configuration into vault, but I'd like something for tfstate.
There exists an encryption proposal which might solve the same problem, but perhaps it'd be easier to delegate the heavy lifting to Vault and have terraform use its HTTP(S) API. Is this a feasible solution or does there exist a better alternative? If so, would it require an enormous effort to implement?
The text was updated successfully, but these errors were encountered: