[Bug]: vault_kv_secret_v2 fails to update secret data modified outside terraform

[nhavens]

Terraform Core Version

1.5.6

Terraform Vault Provider Version

3.19.0

Vault Server Version

1.12.3+ent

Affected Resource(s)

vault_kv_secret_v2 resources that specifies data_json, but the secret's key-value pairs have been modified externally to terraform

Expected Behavior

The vault provider should restore the secret's key-value pairs to the state described in code.

Actual Behavior

terraform apply completes without making any changes.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

terraform {
  required_providers {
    vault = {
      source  = "hashicorp/vault"
      version = "3.19.0"
    }
  }
}

resource "vault_kv_secret_v2" "this" {
  mount               = "secrets"
  name                = "foo/example"
  delete_all_versions = true
  data_json = jsonencode(
    {
      "testkey" = "testvalue"
    }
  )
  custom_metadata {
    max_versions = 1
  }
}

Steps to Reproduce

1. terraform apply
2. Modify the secret in Vault, adding a second key-value pair (vault kv patch -mount=secrets foo/example testkey2=testvalue2)
3. terraform apply - nothing changes

Debug Output

Note the discrepancy between data and data_json in the output of the final command below.

$ terraform apply
vault_kv_secret_v2.this: Refreshing state... [id=secrets/data/foo/example]

Terraform used the selected providers to generate the following execution plan.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # vault_kv_secret_v2.this will be created
  + resource "vault_kv_secret_v2" "this" {
      + data                = (sensitive value)
      + data_json           = (sensitive value)
      + delete_all_versions = true
      + disable_read        = false
      + id                  = (known after apply)
      + metadata            = (known after apply)
      + mount               = "secrets"
      + name                = "foo/example"
      + path                = (known after apply)

      + custom_metadata {
          + max_versions = 1
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

vault_kv_secret_v2.this: Creating...
vault_kv_secret_v2.this: Creation complete after 0s [id=secrets/data/foo/example]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
$ vault kv patch -mount=secrets foo/example testkey2=testvalue2
Data was written to secrets/data/foo/example but we recommend that you add the "patch" capability to your ACL policy in order to use HTTP PATCH in the future.
====== Secret Path ======
secrets/data/foo/example

======= Metadata =======
Key                Value
---                -----
created_time       2023-08-28T22:00:15.285059327Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            2
$ vault kv get -mount=secrets foo/example
====== Secret Path ======
secrets/data/foo/example

======= Metadata =======
Key                Value
---                -----
created_time       2023-08-28T22:00:15.285059327Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            2

====== Data ======
Key         Value
---         -----
testkey     testvalue
testkey2    testvalue2
$ terraform apply
vault_kv_secret_v2.this: Refreshing state... [id=secrets/data/foo/example]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and
found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
$ terraform show -json | jq '.values.root_module.resources[] | .values'
{
  "cas": null,
  "custom_metadata": [
    {
      "cas_required": false,
      "data": {},
      "delete_version_after": 0,
      "max_versions": 1
    }
  ],
  "data": {
    "testkey": "testvalue",
    "testkey2": "testvalue2"
  },
  "data_json": "{\"testkey\":\"testvalue\"}",
  "delete_all_versions": true,
  "disable_read": false,
  "id": "secrets/data/foo/example",
  "metadata": {
    "created_time": "2023-08-28T22:00:15.285059327Z",
    "custom_metadata": "null",
    "deletion_time": "",
    "destroyed": "false",
    "version": "2"
  },
  "mount": "secrets",
  "name": "foo/example",
  "namespace": null,
  "options": null,
  "path": "secrets/data/foo/example"
}
$ 

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

No