Unable to Access Azure Key Vault from Azure Batch Nodes Despite Necessary Permissions.

[Poludasu]

We are experiencing issues accessing Azure Key Vault secrets from a Python script stored in Azure Data Lake Storage. These scripts run on Azure Batch nodes with Linux OS. Despite granting the necessary permissions to the Batch nodes and configuring the identity as a system-assigned managed identity, we cannot access the Key Vault secrets. The pool allocation mode is set to BatchService.

We are using DefaultAzureCredential() to obtain credentials and access the secrets. However, we encounter the following error:

Error:
DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource. Error: Unexpected response "{'error': 'invalid_request', 'error_description': 'Identity not found'}"
SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
AzureCliCredential: Azure CLI not found on path
AzurePowerShellCredential: PowerShell is not installed
AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
Traceback (most recent call last):
File "test_variable_pipeline.py", line 27, in
retrieved_secret = client.get_secret(secret_name)
File "/usr/local/lib/python3.8/dist-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/azure/keyvault/secrets/_client.py", line 64, in get_secret
bundle = self._client.get_secret(
File "/usr/local/lib/python3.8/dist-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/azure/keyvault/secrets/_generated/_operations/_operations.py", line 751, in get_secret
pipeline_response: PipelineResponse = self._client._pipeline.run( # pylint: disable=protected-access
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/_base.py", line 230, in run
return first_node.send(pipeline_request)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/_base.py", line 86, in send
response = self.next.send(request)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/_base.py", line 86, in send
response = self.next.send(request)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/_base.py", line 86, in send
response = self.next.send(request)
[Previous line repeated 2 more times]
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/policies/_redirect.py", line 197, in send
response = self.next.send(request)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/policies/_retry.py", line 531, in send
response = self.next.send(request)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/policies/_authentication.py", line 135, in send
request_authorized = self.on_challenge(request, response)
File "/usr/local/lib/python3.8/dist-packages/azure/keyvault/secrets/_shared/challenge_auth_policy.py", line 137, in on_challenge
self.authorize_request(request, scope, tenant_id=challenge.tenant_id)
File "/usr/local/lib/python3.8/dist-packages/azure/core/pipeline/policies/_authentication.py", line 113, in authorize_request
self._token = self._credential.get_token(*scopes, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/azure/identity/_credentials/default.py", line 225, in get_token
token = super().get_token(*scopes, claims=claims, tenant_id=tenant_id, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/azure/identity/_credentials/chained.py", line 124, in get_token
raise ClientAuthenticationError(message=message)
azure.core.exceptions.ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource. Error: Unexpected response "{'error': 'invalid_request', 'error_description': 'Identity not found'}"
SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
AzureCliCredential: Azure CLI not found on path
AzurePowerShellCredential: PowerShell is not installed
AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.

We have connected to a Batch node and executed the following lines of code:

from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient

credential = DefaultAzureCredential()
key_vault_url = "https://.vault.azure.net/"
client = SecretClient(vault_url=key_vault_url, credential=credential)

secret_name = ""
retrieved_secret = client.get_secret(secret_name)
print(f"Secret: {retrieved_secret.value}")

We are unsure where the issue lies and are looking for guidance on resolving this problem or alternative methods to achieve the same result. Any assistance would be greatly appreciated.

[github-actions]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[xiangyan99]

Thanks for reaching out.

Please follow https://learn.microsoft.com/en-us/azure/batch/managed-identity-pools to configure and use managed identity in batch pools

[github-actions]

Hi @Poludasu. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text "/unresolve" to remove the "issue-addressed" label and continue the conversation.

[github-actions]

Hi @Poludasu, since you haven’t asked that we /unresolve the issue, we’ll close this out. If you believe further discussion is needed, please add a comment /unresolve to reopen the issue.