[2.1.5]sshd error because waagent deletes all ssh_host_* keys and only creates the specified key pair during provisioning

[yuxisun1217]

Description of problem:
If Provisioning.RegenerateSshHostKeyPair=y, waagent deletes all the ssh_host_* keys and only generate the key pair of specified type during provisioning, which makes sshd record error logs in the /var/log/messages.

Version-Release number of selected component (if applicable):
WALinuxAgent-2.1.5

RHEL Version:
RHEL-7.3 internal build

How reproducible:
100%

Steps to Reproduce:

1. Provision a RHEL7.3 VM on Azure with the following parameters in the /etc/waagent.conf:
   Provisioning.RegenerateSshHostKeyPair=y
   Provisioning.SshHostKeyPairType=rsa
2. Check /var/log/messages

Actual results:
/var/log/messages:
Aug 9 17:31:42 localhost sshd[10656]: error: Could not load host key: /etc/ssh/ssh_host_ecdsa_key
Aug 9 17:31:42 localhost sshd[10656]: error: Could not load host key: /etc/ssh/ssh_host_ed25519_key

Expected results:
No error logs in the messages

Maybe we can let waagent only delete the key pairs with the specified type? (Such as, if Provisioning.SshHostKeyPairType=rsa, only delete ssh_host_rsa_key*)

[brendandixon]

@szarkos Steve? Any thoughts?

[szarkos]

On a clean image there should not be any host keys anyway. I think it's a good default to clear any old host keys during provisioning, we don't want every VM on Azure that was created from the same image to have the same host keys.

Anyway, in RHEL 6/7 all the host keys are regenerated by the init script when you (re)start the ssh service. So in the 2.0 agent we created an rsa key as a generic operation just to be safe, and then restarted sshd as the last thing https://github.com/Azure/WALinuxAgent/blob/archive/2.0/waagent#L5585. So on RHEL this had the effect of generating any other keys that RHEL wants.

In 2.1 it looks like the SSH service is restarted from ProvisionHandler.config_user_account(), but probably restart_ssh_service() should be moved to the end of ProvisionHandler.run() after reg_ssh_host_key() is called.

[brendandixon]

@yuxisun1217 @szarkos Makes sense to move the call. This would move https://github.com/Azure/WALinuxAgent/blob/master/azurelinuxagent/pa/provision/default.py#L145 to just after https://github.com/Azure/WALinuxAgent/blob/master/azurelinuxagent/pa/provision/default.py#L69.

[yuxisun1217]

Hi @brendandixon ,
I use v2.1.5.2 and try. It works well. :)

[brendandixon]

@yuxisun1217 Yah! Glad to hear it. Thank you for testing.

[hglkrijger]

Great, thanks!

[shaochieh]

@yuxisun1217 it's happening again on walinuxagent version 2.2.21+really2.2.20-0ubuntu1~16.04.11. I understand @szarkos but the host keys are not regenerated. so i guess i should leave Provisioning.RegenerateSshHostKeyPair=n?