Login fails when run on a remote machine: PermissionError: [WinError 5]

[MatisseHack]

Describe the bug

az login fails when run on a remote machine with the following output:

ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: [WinError 5] : ''
Traceback (most recent call last):
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 231, in invoke
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 658, in execute
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 721, in _run_jobs_serially
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 692, in _run_job
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 328, in __call__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/profile/custom.py", line 149, in login
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/_profile.py", line 160, in login
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/auth/identity.py", line 173, in login_with_service_principal
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/application.py", line 1647, in acquire_token_for_client
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/oauth2cli/oauth2.py", line 753, in obtain_token_for_client
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/oauth2cli/oidc.py", line 115, in _obtain_token
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/oauth2cli/oauth2.py", line 789, in _obtain_token
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/application.py", line 581, in <lambda>
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/token_cache.py", line 307, in add
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/token_cache.py", line 113, in add
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/token_cache.py", line 184, in __add
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal_extensions/token_cache.py", line 49, in modify
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal_extensions/persistence.py", line 163, in save
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal_extensions/windows.py", line 82, in protect
PermissionError: [WinError 5] : ''

The same command works if I connect to the machine via RDP and run it manually. I wonder if the issue has something to do with being run non-interactively? In my case I'm running this script on a remote machine using Packer.

To Reproduce

Run the following script on a remote machine:

Invoke-WebRequest -Uri https://aka.ms/installazurecliwindows -OutFile .\AzureCLI.msi
Start-Process msiexec.exe -Wait -ArgumentList '/I AzureCLI.msi /quiet'

$env:Path = [Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine)

az login --service-principal --username "***" --password "***" --tenant "***"

Expected behavior

az login should complete successfully.

Environment summary

• Install method: MSI
• CLI version: 2.31.0
• OS version: Windows Server 2019
• Shell type: PowerShell

Additional context

az login works if encryption is turned off first.

az config set core.encrypt_token_cache=false

It seems like the underlying issue is with the CryptProtectData function returning ERROR_ACCESS_DENIED.

[jiasli]

Thanks @MatisseHack for the extremely clear issue description and helpful analysis. According to https://docs.microsoft.com/en-us/windows/win32/debug/system-error-codes--0-499-, error code 5 is indeed ERROR_ACCESS_DENIED.

Perhaps CryptProtectData doesn't allow a headless/non-GUI environment? @rayluo

[rayluo]

Thanks @MatisseHack for the extremely clear issue description and helpful analysis. According to https://docs.microsoft.com/en-us/windows/win32/debug/system-error-codes--0-499-, error code 5 is indeed ERROR_ACCESS_DENIED.

Perhaps CryptProtectData doesn't allow a headless/non-GUI environment?

That seems like to be a common trend on most platforms.

• Windows: We observed DPAPI failure on Windows headless mode.
• Linux: The same link above also mentioned that headless mode on Linux would generally fail, (although later we found a workaround to run encryption on Linux in a technically headless docker container).
• MacOS: Encryption on a headless machine would allegedly fail, too.

Perhaps, MSAL EXtensions could and should catch all those different exceptions per platform, and then throw one unified exception (such as EncryptionError), so that our downstream apps (such as Azure CLI) could catch EncryptionError and provide more meaningful error message than the vague PermissionError: [WinError 5] : ''? CCing @bgavrilMS

[bgavrilMS]

Yea, we had to allow a fallback to a plaintext file on all OSes, as requested by PowerShell. There is a never ending stream of corner cases where encryption at rest just isn't working. It's not related to headless mode, both DPAPI and KeyChain are available in headless mode. DPAPI I think is not available in virtualized Windows box.

[petemounce]

+1 azure CLI 2.30.0 and 2.34.0.

This blocks me while remotely provisioning on-premise machines via ansible (or anything really).

I (probably; about to embark on this) have to work around the issue by authenticating as the service principal locally then copying up my ~/.azure to the remote nodes.

That in turn means I need to carefully account for not accidentally publishing my own personal directory (leaking creds I really wouldn't want on my infra) and I'm not sure how to do that yet.

[jiasli]

@petemounce, though I don't fully understand your scenario, but copying ~/.azure to another computer is not a supported use case. You should run az login on the "remote nodes", instead of copying it from somewhere else.

[petemounce]

Are you able to suggest a way to run az login non-interactively for a service-principal without the password secret flag?

As in, supply that secret some other way, one that is less prone to leak into a log and become not-secret?

[rayluo]

Are you able to suggest a way to run az login non-interactively for a service-principal without the password secret flag?

As in, supply that secret some other way, one that is less prone to leak into a log and become not-secret?

This page describes a read -s technique that might be what you want.

[petemounce]

I don't think so; both the bash and powershell variants are interactive. My scenario is headless and non-interactive; an automated process with no human interaction.

[rayluo]

Then you would want this: https://docs.microsoft.com/en-us/cli/azure/authenticate-azure-cli#sign-in-with-a-managed-identity

[petemounce]

@rayluo thanks for the steer. I'm unfamiliar with Azure so far. When I read https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview#how-can-i-use-managed-identities-for-azure-resources it seems to suggest managed identities can only be used when working within Azure resources (I mean; inside Azure-hosted things).

Is that accurate? I'm working with on-premise machines, and so I'm not sure this is applicable. This is my first foray into Azure (from AWS and GCP) though; I'm very much still in learning mode.

[jiasli]

it seems to suggest managed identities can only be used when working within Azure resources (I mean; inside Azure-hosted things).

This is true.

For a on-premise headless/non-interactive environment, we recommend using service principal login as described in the link @rayluo shared: https://docs.microsoft.com/en-us/cli/azure/authenticate-azure-cli#sign-in-with-a-service-principal

If you care about credential leak, please make sure Bash doesn't echo the command themselves. That it, DON'T RUN:

set -v
set -x

Also make sure logging is NOT turned on via logging.enable_log_file: https://docs.microsoft.com/en-us/cli/azure/azure-cli-configuration#cli-configuration-values-and-environment-variables