Support Consul ACL resources

[hashibot]

This issue was originally opened by @sheldonh as hashicorp/terraform#2331. It was migrated here as part of the provider split. The original body of the issue is below.

---

I've love support for Consul ACL resources.

I've taken a quick look, and there's a bit of a decision to be made. Either I have to

• use Consul's /v1/acl/list endpoint to scroll through all tokens, looking for the one that matches the policy I have in tfstate, or
• save tokens to tfstate so I can use /v1/acl/info/<id>.

I'm keen to hear what people think of this trade-off. My instinct is that relying on /v1/acl/list will turn into a mess. In fact, it might not even be possible when there are multiple tokens with the same policy.

So we might be stuck with the less secure idea of saving tokens to tfstate?

[hashibot]

This comment was originally opened by @jamtur01 as hashicorp/terraform#2331 (comment). It was migrated here as part of the provider split. The original comment is below.

---

When/If we can encrypt state (c.f. hashicorp/terraform#516) then this feels a lot more palatable.

[hashibot]

This comment was originally opened by @apparentlymart as hashicorp/terraform#2331 (comment). It was migrated here as part of the provider split. The original comment is below.

---

Perhaps at some point Consul will get the idea of "token accessors" like Vault has, so that it's possible to talk about a token without holding a token. This was mentioned in passing in hashicorp/consul#2334 but I wasn't able to find a top-level issue about it.

I feel kinda inclined to just make sure that the Vault provider has reasonable support for Vault's Consul Backend and for now suggest that folks should be issuing Consul ACLs through that, but it does feel a little harsh to say "if you want to manage Consul ACLs with Terraform then you need to deploy Vault first".

[hashibot]

This comment was originally opened by @daveadams as hashicorp/terraform#2331 (comment). It was migrated here as part of the provider split. The original comment is below.

---

The Consul issue you're looking for, @apparentlymart, is hashicorp/consul#2027.

[therealbill]

So to be clear, the proposal here would be a way to define Consul ACLs in terraform, yes? More specifically the ability to define bits such as the following (from the Consul ACL docs):

key "" {
  policy = "read"
}
key "foo" {
  policy = "write"
}
key "bar" {
  policy = "deny"
}

[daveadams]

@therealbill , yes. That would be a great thing, and should be easy enough to implement. But there's a practical holdup in that ACLs consist of just three fields: the policy itself, the name, and the ID. The policy and the name are not guaranteed to be unique. The ID is unique, but unfortunately the ID is the credential itself, so since that would get stored in tfstate, and make the whole thing terribly insecure. Before Terraform could directly support a Consul ACL resource that I would use, there would need to be a separate identifier in Consul itself, thus the link to consul ticket 2027 (or you could use the mechanism the AWS provider uses for access keys and have the Terraform user provide a PGP key with which to encrypt the ID).

[daveadams]

I've personally managed a workaround using a null_resource and a script, which assumes there won't be more than one Consul ACL with the same name. This is not a safe assumption, because there can be. However, since I'm also the Consul administrator it works in my environment.

[therealbill]

@daveadams thanks for the clarification. I may have to look at some workarounds myself in the meantime. I want to bootstrap a cluster w/some standard ACL setups and would prefer to keep it in terraform. Someday, perhaps. ;)

[daveadams]

FWIW, here's my workaround module:

main.tf: https://gist.github.com/daveadams/1ba4828220911a126dbdf93629c037a1
manage-acl.rb: https://gist.github.com/daveadams/7f088e3ac338b54551a5124ad34baed2

As I mentioned, it relies on the ACL Name field being unique. But it works almost as well as a real resource would. We have some Consul config hardcoded into the manage-acl script in our internal repo, but you could also pass those things in from the module call.

[therealbill]

Thanks, I'll take a look.

[ashald]

It looks like there are plans for native support of an ACL resource but meantime I implemented it as a custom terraform provider - hope it will be useful for others as well https://github.com/Ashald/terraform-provider-consulacl

The provider itself has 100% test coverage including integration tests and we've been using it for couple of week (since alpha version) to manage ACL keys on our production cluster constellation and it works without an issue. Tokens themselves are not disclosed in plan but are stored in the state file.

[therealbill]

The route I took eventually was to use Terraform to define the ACL in Vault which then manages Consul without the sensitive information winding up in the state file. May not work for everyone, especially those not using Vault, but so far it does for me. Took a bit of wrangling because the docs are unclear on that, and I’d love to see a proper TF resource for a consul ACL in Vault rather than a generic secret. I’ll be writing up a blog post on the process in the next week or so.

[pearkes]

@ashald Thanks for that. There is work planned on the ACL API to enable non-sensitive token identifiers that will then allow us to officially add ACL support to this provider. Perhaps when that lands upstream we can collaborate on merging your resource here, and updating to that latest API?

[ashald]

@pearkes if that will be of any help I will be more than happy to do so!

Also, given the plans for HCL2.0 I was planning to update syntax so that it will be possible to copy-paster token rules into Terraform resource as is to simplify migration.

So far I've been using SHA256 hashes of tokens as IDs to not disclose them in logs.

As for sensitive values themselves, we need to ensure concrete token values - for that we've been using RSA encrypted token values that are decrypted in runtime using rsadecrypt interpolation function. I wonder what are your plans on this once token accessors are implemented? Do you plan to keep the functionality to create tokens with given secret values?

[jsmilani]

Consul has released 1.4.0 rc1 and it seems to have added "Accessor ID - The token's public identifier." This sounds like the missing piece to implement this feature.

https://www.consul.io/docs/guides/acl.html#acl-tokens

[pearkes]

@jsmilani yes! We're super excited 😄. We are internally planning on building token and policy resources and data sources but would happily accept a PR for it too. It is a lot of work -- I can support anyone doing it.

Just would need to let us know so we don't overlap.

[pearkes]

FYI I just made a comment on a related PR that I thought was worth cross-linking here: #60 (comment).

[melo]

@remilapeyre Just to be clear: this issue is being closed because #60 covers this request?

[remilapeyre]

Yes, I think it solves the issue but we can reopen the issue if something is missing.