ACS k8s get-credentials encrypted private key

[nerdondon]

When running the az acs kubernetes get-credentials command it errors out saying PasswordRequiredException: Private key file is encrypted. I am on macOS and my private key is password protected.

Please find the logs here:

user@host: az acs kubernetes get-credentials -n $CLUSTER_NAME -g $RESOURCE_GROUP          
Private key file is encrypted
Traceback (most recent call last):
  File "/Users/williamseandon/Library/Python/2.7/lib/python/site-packages/azure/cli/main.py", line 35, in main
    cmd_result = APPLICATION.execute(args)
  File "/Users/williamseandon/Library/Python/2.7/lib/python/site-packages/azure/cli/core/application.py", line 146, in execute
    result = expanded_arg.func(params)
  File "/Users/williamseandon/Library/Python/2.7/lib/python/site-packages/azure/cli/core/commands/__init__.py", line 295, in _execute_command
    result = op(client, **kwargs) if client else op(**kwargs)
  File "/Users/williamseandon/Library/Python/2.7/lib/python/site-packages/azure/cli/command_modules/acs/custom.py", line 534, in k8s_get_credentials
    '.kube/config', path_candidate)
  File "/Users/williamseandon/Library/Python/2.7/lib/python/site-packages/azure/cli/command_modules/acs/acs_client.py", line 21, in SecureCopy
    ssh.connect(host, username=user, key_filename=os.path.join(home, '.ssh', 'id_rsa'))
  File "/Users/williamseandon/Library/Python/2.7/lib/python/site-packages/paramiko/client.py", line 380, in connect
    look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host)
  File "/Users/williamseandon/Library/Python/2.7/lib/python/site-packages/paramiko/client.py", line 621, in _auth
    raise saved_exception
PasswordRequiredException: Private key file is encrypted

A quick search leads to http://stackoverflow.com/questions/15579117/paramiko-using-encrypted-private-key-file-on-os-x . Looks like there may bean issue with paramiko and macOS's keyring? Let me know if I'm completely off base. I was able to copy the kube config from the master node via scp.

[sheerun]

I don't want get-credentials to use my password-protected key from ~/.ssh/id_rsa, but rather unprotected one that I located in ~/.ssh/azure/id_rsa

How to configure it?

[nerdondon]

@sheerun You can configure that when you run the az acs create command. Specifically, the parameter you are looking for is --ssh-key-value and you specify the path to your alternative key. See az acs create -h for a full listing of parameteres and their descriptions.

[sheerun]

I'm afraid I never used az acs create. I tried to create acs from the web interface, then was asked to generate credentials with cli.. The commands I used are:

az account set --subscription="..."
az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/..."
az acs kubernetes get-credentials --resource-group=azure-container-service --name=containerservice-azure-container-service

I makes much more sense to configure ssh path in get-credentials as it's one-time only aciton.

[brendandburns]

@sheerun You should now be able to use --ssh-key-file

See here for the code:

https://github.com/Azure/azure-cli/blob/master/src/command_modules/azure-cli-acs/azure/cli/command_modules/acs/custom.py#L522

I'm not sure if that has made it into the official release yet or not, but if it hasn't it will shortly.

Thanks!

[brendandburns]

And @nerdondon yeah, there is some issue between parimiko and encrypted keys. For now you need to use an unecrypted key, I'm looking into figuring out the issue.

Thanks!
--brendan

[tirithen]

I tried creating an unencrypted key and referencing it but it seem to fail on authenticating against my public key (makes sense to me since I have not uploaded any public key to Azure? Could not find any UI for it). I could never under any circumstance decrypt my main ssh key so this would have to be a separate one but why does it not work?

As I read the log the there is a second failed attempt with my main encrypted key, it's the first failure that is with my unencrypted azure key.

How long until you support proper encrypted keys same way as GitHub/GitLab or any other system does it?

$ az acs kubernetes get-credentials --resource-group=$RESOURCE_GROUP --name=$SERVICE_NAME --ssh-key-file=/home/user/bin/azure-cli/id_rsa --debug                                                                                                                                   
Command arguments ['acs', 'kubernetes', 'get-credentials', '--resource-group=cds-rg', '--name=cds-service', '--ssh-key-file=/home/user/bin/azure-cli/id_rsa']
Installed command modules ['acr', 'acs', 'appservice', 'cloud', 'component', 'configure', 'container', 'context', 'feedback', 'network', 'profile', 'resource', 'role', 'storage', 'vm']
Registered application event handler 'CommandTableParams.Loaded' at <function add_id_parameters at 0x7fd695c26378>
Registered application event handler 'CommandTable.Loaded' at <function add_id_parameters at 0x7fd695c26378>
Loaded module 'acr' in 0.196 seconds.
Loaded module 'acs' in 0.103 seconds.
Loaded module 'appservice' in 0.064 seconds.
Loaded module 'cloud' in 0.001 seconds.
Loaded module 'component' in 0.001 seconds.
Loaded module 'configure' in 0.003 seconds.
Loaded module 'container' in 0.001 seconds.
Loaded module 'context' in 0.001 seconds.
Loaded module 'feedback' in 0.001 seconds.
Loaded module 'network' in 0.138 seconds.
Loaded module 'profile' in 0.001 seconds.
Loaded module 'resource' in 0.003 seconds.
Loaded module 'role' in 0.015 seconds.
Loaded module 'storage' in 0.052 seconds.
Loaded module 'vm' in 0.009 seconds.
Loaded all modules in 0.588 seconds. (note: there's always an overhead with the first module loaded)
Application event 'CommandTable.Loaded' with event data {'command_table': OrderedDict([('acr credential show', <azure.cli.core.commands.CliCommand object at 0x7fd6973f5358>), ('acr credential renew', <azure.cli.core.commands.CliCommand object at 0x7fd695bfe470>), ('acr check-name', <azure.cli.core.commands.CliCommand object at 0x7fd6973ff0f0>), ('acr list', <azure.cli.core.commands.CliCommand object at 0x7fd6962f8b70>), ('acr create', <azure.cli.core.commands.CliCommand object at 0x7fd695c12668>), ('acr delete', <azure.cli.core.commands.CliComm [...]
Application event 'CommandParser.Loaded' with event data {'parser': AzCliCommandParser(prog='az', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)}
Application event 'CommandTableParams.Loaded' with event data {'command_table': OrderedDict([('acr credential show', <azure.cli.core.commands.CliCommand object at 0x7fd6973f5358>), ('acr credential renew', <azure.cli.core.commands.CliCommand object at 0x7fd695bfe470>), ('acr check-name', <azure.cli.core.commands.CliCommand object at 0x7fd6973ff0f0>), ('acr list', <azure.cli.core.commands.CliCommand object at 0x7fd6962f8b70>), ('acr create', <azure.cli.core.commands.CliCommand object at 0x7fd695c12668>), ('acr delete', <azure.cli.core.commands.CliComm [...]
Application event 'CommandParser.Parsed' with event data {'command': 'acs kubernetes get-credentials', 'args': Namespace(_command_package='acs', _jmespath_query=None, _log_verbosity_debug=False, _log_verbosity_verbose=False, _output_format='json', _parser=AzCliCommandParser(prog='az acs kubernetes get-credentials', usage=None, description='Create a new Acs.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True), _validators=[], command='acs kubernetes get-credentials', func=<function create_command.<locals>. [...]
Getting management service client client_type=ComputeManagementClient
msrest.pipeline : Adding 'log_request' callback before event: 'request'
msrest.pipeline : Callback to overwrite original call: False
msrest.pipeline : Adding 'log_response' callback after event: 'response'
msrest.pipeline : Callback to overwrite original call: False
adal-python : c3f36691-ef35-4f2f-b8cf-4c136346a2cc - Authority:Performing instance discovery: https://login.microsoftonline.com/*some id*
adal-python : c3f36691-ef35-4f2f-b8cf-4c136346a2cc - Authority:Performing static instance discovery
adal-python : c3f36691-ef35-4f2f-b8cf-4c136346a2cc - Authority:Authority validated via static instance discovery
adal-python : c3f36691-ef35-4f2f-b8cf-4c136346a2cc - TokenRequest:Getting token from cache with refresh if necessary.
adal-python : c3f36691-ef35-4f2f-b8cf-4c136346a2cc - OAuth2Client:finding with query: {"_clientId": "*some id*", "userId": "*some email*"}
adal-python : c3f36691-ef35-4f2f-b8cf-4c136346a2cc - OAuth2Client:Looking for potential cache entries:
adal-python : c3f36691-ef35-4f2f-b8cf-4c136346a2cc - OAuth2Client:{"_clientId": "*some id*", "userId": "*some email*"}
adal-python : c3f36691-ef35-4f2f-b8cf-4c136346a2cc - OAuth2Client:Found 2 potential entries.
adal-python : c3f36691-ef35-4f2f-b8cf-4c136346a2cc - OAuth2Client:Resource specific token found.
adal-python : c3f36691-ef35-4f2f-b8cf-4c136346a2cc - OAuth2Client:Returning token from cache lookup, AccessTokenId: b'*some token*', RefreshTokenId: b'*some token*'
msrest.pipeline : Configuring request: timeout=100, verify=True, cert=None
msrest.pipeline : Configuring redirects: allow=True, max=30
msrest.pipeline : Configuring proxies: ''
msrest.pipeline : Evaluate proxies against ENV settings: True
msrest.pipeline : Configuring retry: max_retries=3, backoff_factor=0.8, max_backoff=90
msrest.http_logger : Request URL: 'https://management.azure.com/subscriptions/*some id*/resourceGroups/cds-rg/providers/Microsoft.ContainerService/containerServices/cds-service?api-version=2016-09-30'
msrest.http_logger : Request method: 'GET'
msrest.http_logger : Request headers:
msrest.http_logger :     'User-Agent': 'python/3.6.0 (Linux-4.9.6-1-ARCH-x86_64-with-arch-Arch-Linux) requests/2.13.0 msrest/0.4.4 msrest_azure/0.4.7 computemanagementclient/0.32.1 Azure-SDK-For-Python AZURECLI/0.1.1b3'
msrest.http_logger :     'Accept-Encoding': 'gzip, deflate'
msrest.http_logger :     'Accept': 'application/json'
msrest.http_logger :     'Connection': 'keep-alive'
msrest.http_logger :     'Authorization': 'Bearer *some token*'
msrest.http_logger :     'x-ms-client-request-id': '*some id*'
msrest.http_logger :     'CommandName': 'acs kubernetes get-credentials'
msrest.http_logger :     'Content-Type': 'application/json; charset=utf-8'
msrest.http_logger :     'accept-language': 'en-US'
msrest.http_logger : Request body:
msrest.http_logger : None
requests.packages.urllib3.connectionpool : Starting new HTTPS connection (1): management.azure.com
requests.packages.urllib3.connectionpool : https://management.azure.com:443 "GET /subscriptions/*some id*/resourceGroups/cds-rg/providers/Microsoft.ContainerService/containerServices/cds-service?api-version=2016-09-30 HTTP/1.1" 200 None
msrest.http_logger : Response status: 200
msrest.http_logger : Response headers:
msrest.http_logger :     'Cache-Control': 'no-cache'
msrest.http_logger :     'Pragma': 'no-cache'
msrest.http_logger :     'Transfer-Encoding': 'chunked'
msrest.http_logger :     'Content-Type': 'application/json; charset=utf-8'
msrest.http_logger :     'Content-Encoding': 'gzip'
msrest.http_logger :     'Expires': '-1'
msrest.http_logger :     'Vary': 'Accept-Encoding'
msrest.http_logger :     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
msrest.http_logger :     'x-ms-served-by': '*some id*'
msrest.http_logger :     'x-ms-request-id': '*some id*'
msrest.http_logger :     'Server': 'Microsoft-HTTPAPI/2.0, Microsoft-HTTPAPI/2.0'
msrest.http_logger :     'x-ms-ratelimit-remaining-subscription-reads': '14988'
msrest.http_logger :     'x-ms-correlation-request-id': '*some id*'
msrest.http_logger :     'x-ms-routing-request-id': 'WESTEUROPE:20170206T134401Z:*some id*'
msrest.http_logger :     'Date': 'Mon, 06 Feb 2017 13:44:01 GMT'
msrest.http_logger : Response content:
msrest.http_logger : Body contains chunked data.
paramiko.transport : starting thread (client mode): 0x94460320
paramiko.transport : Local version/idstring: SSH-2.0-paramiko_2.1.1
paramiko.transport : Remote version/idstring: SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu1
paramiko.transport : Connected (version 2.0, client OpenSSH_7.2p2)
paramiko.transport : kex algos:['curve25519-sha256@libssh.org', 'ecdh-sha2-nistp256', 'ecdh-sha2-nistp384', 'ecdh-sha2-nistp521', 'diffie-hellman-group-exchange-sha256', 'diffie-hellman-group14-sha1'] server key:['ssh-rsa', 'rsa-sha2-512', 'rsa-sha2-256', 'ecdsa-sha2-nistp256', 'ssh-ed25519'] client encrypt:['chacha20-poly1305@openssh.com', 'aes128-ctr', 'aes192-ctr', 'aes256-ctr', 'aes128-gcm@openssh.com', 'aes256-gcm@openssh.com'] server encrypt:['chacha20-poly1305@openssh.com', 'aes128-ctr', 'aes192-ctr', 'aes256-ctr', 'aes128-gcm@openssh.com', 'aes256-gcm@openssh.com'] client mac:['umac-64-etm@openssh.com', 'umac-128-etm@openssh.com', 'hmac-sha2-256-etm@openssh.com', 'hmac-sha2-512-etm@openssh.com', 'hmac-sha1-etm@openssh.com', 'umac-64@openssh.com', 'umac-128@openssh.com', 'hmac-sha2-256', 'hmac-sha2-512', 'hmac-sha1'] server mac:['umac-64-etm@openssh.com', 'umac-128-etm@openssh.com', 'hmac-sha2-256-etm@openssh.com', 'hmac-sha2-512-etm@openssh.com', 'hmac-sha1-etm@openssh.com', 'umac-64@openssh.com', 'umac-128@openssh.com', 'hmac-sha2-256', 'hmac-sha2-512', 'hmac-sha1'] client compress:['none', 'zlib@openssh.com'] server compress:['none', 'zlib@openssh.com'] client lang:[''] server lang:[''] kex follows?False
paramiko.transport : Kex agreed: diffie-hellman-group14-sha1
paramiko.transport : Cipher agreed: aes128-ctr
paramiko.transport : MAC agreed: hmac-sha2-256
paramiko.transport : Compression agreed: none
paramiko.transport : kex engine KexGroup14 specified hash_algo <built-in function openssl_sha1>
paramiko.transport : Switch to new keys ...
paramiko.transport : Adding ssh-rsa host key for mig.westeurope.cloudapp.azure.com: b'*some id*'
paramiko.transport : Trying key b'*some id*' from /home/user/bin/azure-cli/id_rsa
paramiko.transport : userauth is OK
paramiko.transport : Authentication (publickey) failed.
paramiko.transport : Trying SSH agent key b'*some id*'
paramiko.transport : userauth is OK
paramiko.transport : Authentication (publickey) failed.
Private key file is encrypted
Traceback (most recent call last):
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/azure/cli/main.py", line 37, in main
    cmd_result = APPLICATION.execute(args)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/azure/cli/core/application.py", line 157, in execute
    result = expanded_arg.func(params)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 333, in _execute_command
    result = op(client, **kwargs) if client else op(**kwargs)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/azure/cli/command_modules/acs/custom.py", line 535, in k8s_get_credentials
    _k8s_get_credentials_internal(name, acs_info, path, ssh_key_file)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/azure/cli/command_modules/acs/custom.py", line 552, in _k8s_get_credentials_internal
    '.kube/config', path_candidate, key_filename=ssh_key_file)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/azure/cli/command_modules/acs/acs_client.py", line 20, in SecureCopy
    ssh.connect(host, username=user, key_filename=key_filename)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/paramiko/client.py", line 380, in connect
    look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/paramiko/client.py", line 621, in _auth
    raise saved_exception
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/paramiko/client.py", line 595, in _auth
    key = pkey_class.from_private_key_file(filename, password)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/paramiko/pkey.py", line 196, in from_private_key_file
    key = cls(filename=filename, password=password)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/paramiko/rsakey.py", line 45, in __init__
    self._from_private_key_file(filename, password)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/paramiko/rsakey.py", line 163, in _from_private_key_file
    data = self._read_private_key_file('RSA', filename, password)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/paramiko/pkey.py", line 268, in _read_private_key_file
    data = self._read_private_key(tag, f, password)
  File "/home/user/bin/azure-cli/lib/python3.6/site-packages/paramiko/pkey.py", line 310, in _read_private_key
    raise PasswordRequiredException('Private key file is encrypted')
paramiko.ssh_exception.PasswordRequiredException: Private key file is encrypted

[tirithen]

My colleague also have problems, but he is on Windows (I'm on Arch Linux). How can we upload our public key to Azure before running $ az acs kubernetes get-credentials ? Is there an other manual way of downloading these credentials?

[abeven]

+1 Having this same issue. Also, what would be the process to specify a password if I am using an encrypted key?

[brendandburns]

I need to dig into the parimiko SSH library to figure this out, apologies for the delay...

[jpoon]

Colleague recently hit the same issue -- as a temporary workaround to obtaining the kubeconfig, you can ssh into the master node and scp the config from .kube/config to your local machine.

[ashb]

Paramiko has support for getting the key from the ssh-agent which would be my preferred way of dealing with encrypted keys.

[dottorblaster]

Is this issue still open? I see this closed but I'm getting:

$ az acs kubernetes get-credentials --resource-group=k8s-dottorblaster --name=containerservice-k8s-dottorblaster
Private key file is encrypted
Traceback (most recent call last):
  File "/Users/blaster/lib/azure-cli/lib/python2.7/site-packages/azure/cli/main.py", line 36, in main
    cmd_result = APPLICATION.execute(args)
  File "/Users/blaster/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/application.py", line 203, in execute
    result = expanded_arg.func(params)
  File "/Users/blaster/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 278, in __call__
    return self.handler(*args, **kwargs)
  File "/Users/blaster/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 473, in _execute_command
    reraise(*sys.exc_info())
  File "/Users/blaster/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 450, in _execute_command
    result = op(client, **kwargs) if client else op(**kwargs)
  File "/Users/blaster/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/custom.py", line 690, in k8s_get_credentials
    _k8s_get_credentials_internal(name, acs_info, path, ssh_key_file)
  File "/Users/blaster/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/custom.py", line 711, in _k8s_get_credentials_internal
    '.kube/config', path_candidate, key_filename=ssh_key_file)
  File "/Users/blaster/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/acs_client.py", line 49, in SecureCopy
    ssh.connect(host, username=user, pkey=pkey)
  File "/Users/blaster/lib/azure-cli/lib/python2.7/site-packages/paramiko/client.py", line 381, in connect
    look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host)
  File "/Users/blaster/lib/azure-cli/lib/python2.7/site-packages/paramiko/client.py", line 622, in _auth
    raise saved_exception
PasswordRequiredException: Private key file is encrypted

I have a passphrase protected keyfile.

[squillace]

+1 -- ditto. passphrase protected keyfile.

EDIT: I'm on WSL for Windows here. :-|

[squillace]

Update: Same event on most recent CLI on mac as well.

[squillace]

My bad: ignore above comments: user error.

[dottorblaster]

@squillace what was the error? Maybe it's worth sharing. Did you get the PasswordRequiredException

[squillace]

No. I get the same Private key file is encrypted line. paramiko.ssh_exception.PasswordRequiredException: Private key file is encrypted. I thought I had it figured out, but I didn't. I"ll try again tomorrow. This is on WSL and mac.

[dottorblaster]

Hope you'll find a solution. I'm still stuck here, no luck.

[berndverst]

I followed these instructions: https://docs.microsoft.com/en-us/azure/container-service/container-service-kubernetes-walkthrough

@squillace I'm also getting PasswordRequiredException: Private key file is encrypted on Mac. Latest CLI, latest everything (updated via homebrew).

[berndverst]

Workaround:

mkdir $HOME/.kube
scp azureuser@<master-dns-name>:.kube/config $HOME/.kube/config

where <master-dns-name> is <dns-prefix>.<location>.cloudapp.azure.com

[devigned]

This PR may fix the continued pain of this issue #3612

[dottorblaster]

OMG you are the best.

[marcote]

Another workaround:
Fire up the ssh-agent and make it load the keys.

Background:
I'm using a brand new Mac laptop and I hit the very same issue. After reading the comments, I assed the ssh-agent plugin into zsh, reload the shell.

Executed again: az kubernetes get-credentials ...
And the error is gone.

[dtrapezoid]

@marcote I added ssh-agent: $ ssh-add ~/.ssh/id_rsa then plugins=(... ssh-agent) into my zshrc profile, reloaded. Still got the same error: PasswordRequiredException: Private key file is encrypted

I had previously installed with homebrew but uninstalled and grabbed the latest version of azure-cli w/curl.

Is anyone else still struggling with this same issue?

[dtrapezoid]

If I do az acs kubernetes get-credentials --resource-group=myResourceGroup --name=myK8sCluster --ssh-key-file=~/.ssh/id_rsapassing the path to my id_rsa, I get Private key file ~/.ssh/id_rsa does not exist - this is after adding it to the ssh-agent, with ssh-add ~/.ssh/id_rsa as noted in previous comment.

Am I missing something?

[campbelldgunn]

@dtrapezoid I had the same issue. What I found to fix it was to do a ssh-add ~/.ssh/<private_key_file_name> then you need to specify the get-credentials without the ssh-key-file arg.

Hope this helps.