allow forcing a new ssh key pair

[dafydd-t]

Currently the pdc-agent uses an existing SSH key pair and certificate if they exist and are valid. We should allow users to force a new ssh key pair and certificate to be generated.

Implement a --force-key-file-overwrite flag that generates a new SSH key pair and requests a new certificate using the new public key.

[Hronom]

Please make it somehow automaticly regenerate certificate if it fails to use already existing.

My current workaround is to add -ssh-key-file parameter that is differ from default, for example /home/pdc/.ssh/grafana_pdc_v1

[dafydd-t]

Please make it somehow automaticly regenerate certificate if it fails to use already existing.

@Hronom What is the reason that the existing key fails? There is already logic for refreshing if the cert is no longer valid, so I'm wondering what the other failure modes are.

[Hronom]

@dafydd-t hello, I'm using version 0.0.13 of dcoker image. After some time I get this error:

level=error caller=ssh.go:122 msg="ssh client exited. restarting"
level=info caller=keymanager.go:52 msg="starting key manager"
level=info caller=keymanager.go:156 msg="new certificate required: certificate validity has expired"
level=info caller=keymanager.go:204 msg="generating new certificate"
level=error caller=client.go:177 msg="unknown response from PDC API" code=502
level=error caller=ssh.go:102 msg="could not check or generate certificate" error="failed to generate new certificate: key signing request failed: internal error"

If I restart containers it starting to work.

Is it fixed in 0.0.15 I see it released 2 days ago?

[dafydd-t]

@Hronom 0.0.14 introduces some retries for the key signing request. This helps in some cases, but we've also identified a change required internally. That should get sorted in the next day or two.

[Hronom]

Thanks @dafydd-t , just a side note, we found this approach (grafana-pdc-agent) very usefull and very easy to connect with Grafana Cloud, interested in active development of this and adding new datasource in it. Thanks for this great tool!