asm secret version "secret_string_wo_version" rejects ephemeral values

[rjhornsby]

Terraform and AWS Provider Version

╰─± terraform version
Terraform v1.13.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v6.14.1
+ provider registry.terraform.io/hashicorp/vault v5.3.0

Affected Resource(s) or Data Source(s)

• aws_secretsmanager_secret_version

Expected Behavior

The property secret_string_wo_version should accept ephemeral values, ie ephemeral.vault_kv_secret_v2.some_secret.version

Actual Behavior

Attempting to do so results in a TF error telling us that we can't write an ephemeral value to the secret_string_wo_version property. The error message suggests a conflict with the wo portion of the property name.

│ Error: Invalid use of ephemeral value
│
│   with aws_secretsmanager_secret_version.some_secret,
│   on account_default_config.tf line 45, in resource "aws_secretsmanager_secret_version" "some_secret":
│   45:   secret_string_wo_version = ephemeral.vault_kv_secret_v2.some_secret.version
│
│ Ephemeral values are not valid for "secret_string_wo_version", because it is not a write-only attribute and must be persisted to state

I've also tried unsuccessfully to "cheat" and sha256() the vault secret version, but the provider/resource still rejects it for the same reason.

The only other solution I can come up with to comply with the documentation for the property

Used together with secret_string_wo to trigger an update. Increment this value when an update to secret_string_wo is required.

is to hardcode a version value into the TF and change it in the TF code every time the vault secret changes -- but that's an extremely poor solution, if it can even be called that. Open to other suggestions, but it seems like the secret_string_wo_version property behavior is broken.

Relevant Error/Panic Output

│ Error: Invalid use of ephemeral value
│
│   with aws_secretsmanager_secret_version.some_secret,
│   on account_default_config.tf line 45, in resource "aws_secretsmanager_secret_version" "some_secret":
│   45:   secret_string_wo_version = ephemeral.vault_kv_secret_v2.some_secret.version
│
│ Ephemeral values are not valid for "secret_string_wo_version", because it is not a write-only attribute and must be persisted to state

Sample Terraform Configuration

Click to expand configuration

ephemeral "vault_kv_secret_v2" "some_secret" {
  mount = "a/secret/vault"
  name  = "key/to/a/great/secret"
}

resource "aws_secretsmanager_secret" "some_secret" {
  name                           = "some_secret"
  description                    = "secret to everlasting life"
  recovery_window_in_days        = 7
  force_overwrite_replica_secret = true
}

resource "aws_secretsmanager_secret_version" "some_secret" {
  secret_id        = aws_secretsmanager_secret.some_secret.id
  secret_string_wo = ephemeral.vault_kv_secret_v2.some_secret.data["cheat_code"]
  secret_string_wo_version = ephemeral.vault_kv_secret_v2.some_secret.version
}

Steps to Reproduce

1. Plan the configuration
2. See error message

Debug Logging

Click to expand log output

GenAI / LLM Assisted Development

n/a

Important Facts and References

This may be related to #42383 or #41635.

Would you like to implement a fix?

No

[github-actions]

Community Guidelines

This comment is added to every new Issue to provide quick reference to how the Terraform AWS Provider is maintained. Please review the information below, and thank you for contributing to the community that keeps the provider thriving! 🚀

Voting for Prioritization

• Please vote on this Issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize it.
• Please see our prioritization guide for additional information on how the maintainers handle prioritization.
• Please do not leave +1 or other comments that do not add relevant new information or questions; they generate extra noise for others following the Issue and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.
• For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.

[apparentlymart]

I think this issue is describing essentially the same thought process I tumbled through in #41371 (comment). 🤔

I included a different idea for addressing this at the end of that comment, in case that's interesting/helpful.

In the meantime I found a way to work around it with a weird utility provider: apparentlymart/memory's memory resource type. This resource adds the option of storing the most recent "version" number in the Terraform state so that at least you only need to provide a value for it when you're actively trying to update the secret, and it'll remain unchanged otherwise.

---

In this particular case, where there's a separate remote system that's already storing a version number, a variant of that "memory" idea would be for the hashicorp/vault provider to offer a data source that only returns the metadata for a secret, without returning the secret information itself, and then use that data source place of the memory resource type:

// INVALID: This is a hypothetical resource type that doesn't currently exist
// in the hashicorp/vault provider.
data "vault_kv_secret_v2_metadata" "example" {
  mount = "a/secret/vault"
  name  = "key/to/a/great/secret"
}

ephemeral "vault_kv_secret_v2" "example" {
  mount   = data.vault_kv_secret_v2_metadata.example.mount
  name    = data.vault_kv_secret_v2_metadata.example.name
  version = data.vault_kv_secret_v2_metadata.example.version
}

# (...aws_secretsmanager_secret resource as before...)

resource "aws_secretsmanager_secret_version" "example" {
  secret_id        = aws_secretsmanager_secret.example.id
  secret_string_wo = ephemeral.vault_kv_secret_v2.example.data["example"]
  secret_string_wo_version = data.vault_kv_secret_v2.example.version

  lifecycle {
    precondition {
      # This is completely optional, but would make sure that the
      # data and ephemeral resources both agree on which version
      # they are using in case future incorrect changes to the module
      # might cause them to get desynchronized somehow
      condition     = ephemeral.vault_kv_secret_v2.example.version == data.vault_kv_secret_v2.example.version
      error_message = "Fetched secret has version ${ephemeral.vault_kv_secret_v2.example.version}, but was expecting version ${data.vault_kv_secret_v2.example.version}."
    }
  }
}

In this shape of things, each resource has a distinct purpose:

• data.kv_secret_v2_metadata.example determines what is the current version number of the secret, but doesn't actually return any secret data so it's okay for its result to persist in the state.

  This effectively uses Vault's own metadata as the "memory" for what version was most recently used, so it doesn't need to be maintained separately as either an input variable or as the value in a memory resource.

• ephemeral.vault_kv_secret_v2.example obtains a lease for the chosen secret version and returns the actual secret material as an ephemeral value, so it doesn't get saved in the state.

• aws_secretsmanager_secret_version.example causes the Secrets Manager copy of the secret to be updated whenever the source secret has changed in Vault, using the version number returned by data.kv_secret_v2_metadata.example to make that decision.

  The secret data is in a write-only attribute, so again it doesn't get saved to the state.

I'm not going to try to argue that this isn't very clunky, but it seems like it would work using the mechanisms that today's Terraform supports, with just an additional data source in hashicorp/vault to return the secret metadata without the actual secret data.

The main thing this misses (compared to some the other two patterns I was playing with) is a way to avoid requesting the secret from Vault when the version number hasn't changed, but that's a pretty marginal thing in this setup where Vault is also the authority on whether a password change is needed and so some Vault credentials would be needed to make that decision either way.

(I personally prefer the model with the separate input variable to force the secret update, because that means the secret update can require some special elevated credentials that aren't used for routine work, but I expect different teams will have different needs/wants in that area.)

---

FWIW, I don't think it would be possible for secret_string_wo_version to accept an ephemeral value directly, as proposed in the original issue text, because that would mean the ephemeral value would need to be stored in the state to allow the provider to compare with it next time, and the whole point of ephemeral values is that they aren't stored in the state.