unknown or unsupported field awskms found in configuration when configs are stored in JSON

[timurkhisamov]

awskms field didn't parsed when configs are stored in JSON format
When I use JSON format to store our configurations, I found in logs next:

JSON configs:

{
    "api_addr": "http://{{ GetPrivateIP }}:8200",
    "cluster_addr": "http://{{ GetPrivateIP }}:8201",
    "cluster_name": "infra-jp-aws",
    "listener": {
        "tcp": {
            "address": "{{ GetPrivateIP }}:8200",
            "cluster_address": "{{ GetPrivateIP }}:8201",
            "tls_disable": true
        }
    },
    "log_level": "info",
    "seal": {
        "awskms": {
            "kms_key_id": "alias/kms-unseal-key"
        }
    },
    "storage": {
        "consul": {
            "address": "http://localhost:8500",
        }
    },
    "telemetry": {
        "disable_hostname": true,
        "prometheus_retention_time": "60s"
    },
    "ui": true
}

But when I use HCL format, error didn't reproduce:

HCL configs:

api_addr = "http://{{ GetPrivateIP }}:8200"
cluster_addr = "http://{{ GetPrivateIP }}:8201"
cluster_name = "infra-jp-aws"
listener "tcp" {
  address = "{{ GetPrivateIP }}:8200"
  cluster_address = "{{ GetPrivateIP }}:8201"
  tls_disable = true
}
log_level = "info"
seal "awskms" {
  kms_key_id = "alias/kms-unseal-key"
}
storage "consul" {
  address = "http://localhost:8500"
telemetry = {
  disable_hostname = true
  prometheus_retention_time = "60s"
}
ui = true

To Reproduce
Steps to reproduce the behavior:

1. Run vault using json config with awskms unsealing option.
2. See error

Expected behavior
A clear and concise description of what you expected to happen.

Environment:

• Vault Server Version (retrieve with vault status):

Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    3
Threshold                2
Version                  1.11.3
Build Date               2022-08-26T10:27:10Z
Storage Type             consul
Cluster Name             infra-jp-aws
Cluster ID               bbceb8a1-34e9-54ce-6280-7efac01f3f45
HA Enabled               true
HA Cluster               https://10.0.11.239:8201
HA Mode                  active
Active Since             2022-09-05T08:55:57.087708264Z

• Vault CLI Version (retrieve with vault version):

Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    3
Threshold                2
Version                  1.11.3
Cluster Name             infra-jp-aws
Cluster ID               bbceb8a1-34e9-54ce-6280-7efac01f3f45
HA Enabled               true
HA Cluster               https://10.0.11.239:8201
HA Mode                  active

• Server Operating System/Architecture:

Linux hashi-0 5.11.0-1027-aws #30~20.04.1-Ubuntu SMP Thu Jan 13 11:46:53 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

[zombiezen]

I'm seeing similar behavior with the service_registration "consul" stanza on Vault 1.11.2

[zombiezen]

I did a little more investigating and it seems Vault accepts the JSON configuration and the warning is spurious. (To be clear, this should be fixed to prevent confusion.) In my case, I was using the gcpckms seal and the consul service registration. I was able to confirm that it was still using the GCP KMS key and registering with Consul.

[VioletHynes]

Hi folks! Just wanted to leave a little update here. I believe I've found the source of the problem. I can confirm that the warnings are spurious and that a fix will be coming soon.

In particular, this affects json configuration like this:

    "foo": {
        "bar": {
            "baz": "xyz"
        }
    }

and it will in some cases produce a spurious warning about bar.

[VioletHynes]

Closing this issue, as this should be fixed by #17660

Thanks for the report! I appreciated the detail.