ManagedIdentityCredential authentication failed

[nomaster]

Describe the bug
After updating to version 1.18.5, Vault cannot log in to Azure anymore.

To Reproduce

Azure steps:

1. Provision Azure Key Vault
2. Configure Federated Identity Credential
3. Assign Key Vault permissions to identity

Kubernetes steps:

1. Install Vault on Azure Kubernetes Services
2. Add pod label azure.workload.identity/use: "true"
3. Configure raft with seal "azurekeyvault" {}
4. Start Vault server

Error message

error parsing Seal configuration: error fetching Azure Key Vault wrapper key information: ManagedIdentityCredential authentication failed. the requested identity isn't assigned to this resource
GET http://169.254.169.254/metadata/identity/oauth2/token
RESPONSE 400: 400 Bad Request

Expected behavior
Vault uses Azure Workload Identity to authenticate to Azure.

Environment:

• Vault Server Version (retrieve with vault status): 1.18.5
• Vault CLI Version (retrieve with vault version): 1.18.5
• Server Operating System/Architecture: Linux/x86_64

Vault server configuration file(s):

ui = true
listener "tcp" {
  address = "[::]:8200"
  cluster_address = "[::]:8201"
  tls_cert_file = "/vault/userconfig/vault-tls/tls.crt"
  tls_key_file  = "/vault/userconfig/vault-tls/tls.key"
  tls_client_ca_file = "/vault/userconfig/vault-tls/ca.crt"
}
seal "azurekeyvault" {
}
storage "raft" {
  performance_multiplier = 1
  path = "/vault/data"
  retry_join {
    auto_join = "provider=k8s label_selector=\"app.kubernetes.io/name=vault,component=server\" namespace=\"vault\" "
    auto_join_scheme = "https"
    leader_tls_servername = "vault"
    leader_ca_cert_file = "/vault/userconfig/vault-tls/ca.crt"
    leader_client_cert_file = "/vault/userconfig/vault-tls/tls.crt"
    leader_client_key_file = "/vault/userconfig/vault-tls/tls.key"
  }
}
service_registration "kubernetes" {}
telemetry {
  prometheus_retention_time = "30s"
  disable_hostname = true
}
plugin_directory = "/usr/local/libexec/vault/"
disable_mlock = true

Additional context
Version 1.18.4 works fine with the same configuration

[heatherezell]

Thank you for the report! This may be a regression that was introduced when attempting to fix managed identity based auth for Azure. For now, we recommend downgrading to 1.18.4. I'll keep an eye out for when a fix can be expected and reply back here when I have more information. Thanks again!

[heatherezell]

This should be resolved in 1.18.6. Thanks again for your report! Let me know if you're still having issues.

[heatherezell]

Wanted to issue a correction to my last comment: 1.18.6 is out for Enterprise, but Community Edition users who are impacted should upgrade to 1.19. Thanks!

[nomaster]

Thanks!

Unfortunately, we cannot test the fix, since we ran into another issue:

• No longer possible to use label_selector in auto_join field #29863

Will check again with the next release.

[keisari-ch]

hi,

was not able to make it work with following versions :

1.18.4, 1.18.5 and 1.19

looping with the following messages :

Connecting...
2025-03-26T17:19:15.23544  Connecting to the container 'ca-vault-dev-sn'...
2025-03-26T17:19:15.25235  Successfully Connected to container: 'ca-vault-dev-sn' [Revision: 'ca-vault-dev-sn--eobscsp-8999d946c-r4r5r', Replica: 'ca-vault-dev-sn--eobscsp']
2025-03-26T17:19:14.137467738Z Couldn't start vault with IPC_LOCK. Disabling IPC_LOCK, please use --cap-add IPC_LOCK
2025-03-26T17:19:14.259660586Z Error initializing storage of type azure: failed to obtain auth token from IMDS "AzurePublicCloud": adal: Refresh request failed. Status Code = '400'. Response body: {"statusCode":400,"message":"Unable to load the proper Managed Identity.","correlationId":"61d5b385-53c9-4bea-96f9-1bb6a2d02929"} Endpoint http://localhost:42356/msi/token?api-version=2017-09-01&resource=https%3A%2F%2Fstorage.azure.com%2F
2025-03-26T17:19:14.259660489Z 2025-03-26T17:19:14.249Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
2025-03-26T17:19:14.259689250Z 2025-03-26T17:19:14.249Z [INFO]  storage.azure: accountKey not set, using managed identity auth

[nomaster]

Works for us now with version 1.19.2 and 1.19.3 and the workaround from the other issue:

• No longer possible to use label_selector in auto_join field #29863 (comment)