Allow creating tokens with pregenerated secret_ids

[brknstrngz]

Hi,

I am aware of how contentious this is due to sensitive data ending up in the state file, so I am just testing the water.

In a scenario where IDs for the master token and agent tokens are programatically generated and distributed to Consul, Nomad and Vault nodes over an out of band channel along with their configuration, agent tokens can then be subsequently provisioned configured in Consul with their appropriate policies using this provider and the master token, once server quorum is reached. This then allows Nomad and Vault to join the Consul cluster with the appropriate rights and register themselves.

This is the rough equivalent of distributing the token IDs ahead of the time in the same fashion and then passing -secret= explicitly to consul acl token create on the master, but less clunky.

Perhaps this can also be placed under a boldface warning sign as we have for consul_acl_token_secret_id, suggesting storing the state remotely?

Thanks!

[remilapeyre]

Hi @brknstrngz, you are correct that this would end up saving the token in the state file which is why we did not offer this possibility in the first place.

At the moment the ACL bootstrap more manual that I would like it to be, but there is work going on in Consul to use JWT to simplify this part: hashicorp/consul#7972. Once this is ready we should be able to use this to securely add nodes to the cluster.

As of today, I don't have a better solution, if you need tokens with a specific secret_id you could create them manually as you already do and import them in Terraform using terraform import consul_acl_token.my_token ... and then migrate to the JWT method to introduce new nodes when it's ready.

[brknstrngz]

As of today, I don't have a better solution, if you need tokens with a specific secret_id you could create them manually as you already do and import them in Terraform using terraform import consul_acl_token.my_token ... and then migrate to the JWT method to introduce new nodes when it's ready.

Hi @remilapeyre, thank you for the swift reply!

My understanding of this is that there needs to be an existing Consul cluster that the provider configuration points to, where tokens to be imported reside. Is that correct? If so, I may be in a chicken and egg situation - at the point in time where my token secret ids are generated the Consul cluster has not yet been provisioned.

My workflow is as follows:

1. Generate the master token for the Consul servers and the agent tokens and policies for all Consul nodes (servers and clients), Nomad servers and clients and Vault, outside of Terraform
2. Have Terraform provision cloud VMs
3. Use a configuration management tool (Salt, in this case) to set the agent token IDs generated in step 1 in the Consul, Nomad and Vault configuration files on all nodes
4. Wait for Consul server quorum
5. Using the tokens and policies generated in step 1, programatically generate a new Terraform state containing the configuration for this provider (pointing to the newly running Consul cluster) and all consul_acl_policy and consul_acl_token resources, then apply that state.

I hope this makes more sense now.

[hashicorp-cla]

All committers have signed the CLA.

[andyrepton]

Hi, just to revive this a little bit, I've put together almost an identical PR myself, with some documentation explaining the risks of storing the tokens in the state file. With appropriate warnings, would this be acceptable to merge to match existing Consul API functionality?

For what it’s worth, I’ve tested that if you do not provide the secret_id in your resource, it will be marked as ‘null’ in your state file, so it shouldn’t catch people off guard.