logging in with ssh certificates

[oseiberts11]

Hi! I'm trying out your ssh_exporter. You may have noticed that I ran into a few small snags in your README :)

Right now I am trying to actually log in a user. We are not using private keys directly for authentication, but key certificates. The ssh server uses TrustedUserCAKeys /etc/ssh/trusted_user_ca_keys (and HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub), in its configuration. Users also have ed25519 keys.

I found an article about this here: https://berndbausch.medium.com/ssh-certificates-a45bdcdfac39
https://www.lorier.net/docs/ssh-ca.html also looks good (much more compact though: gives very specific commands to run but with less explanation).

I haven't found a way yet to log in with this sort of configuration. Am I missing something?

Futhermore, if I specify a known_hosts file, I see the same error as in golang/go#33366 : level=error ts=2021-09-30T12:26:26.813Z caller=collector.go:128 target=hostname:22 msg="Failed to establish SSH connection" err="ssh: handshake failed: ssh: no authorities for hostname: hostname:22".

If I leave the known_hosts file out, then a "normal" authentication failure: level=error ts=2021-09-30T12:47:15.063Z caller=collector.go:128 target=hostname:22 msg="Failed to establish SSH connection" err="ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain".

[treydock]

For the known_hosts issue, I think you have to also define host_key_algorithms setting and there might be some other quirks around hostname matching. I don't recall the details but I remember when I tested that feature that it was not robust like regular SSH for host key matching, that it was very specific like either the host key name had to match exactly and/or you have to tell the Golang SSH code which Key algorithm to match against because if it gets back RSA and first one in known_hosts is ed25519, it would fail.

I will look more into the SSH certificates and see if that's possible.

[treydock]

So it looks like based on one example I found: https://stackoverflow.com/a/52139555 , that to support SSH with certificates would require the host running this exporter to have access to both the certificate public and private keys. Will such a solution work for you having to give the exporter access to both public and private keys for the certificate?

[oseiberts11]

Thanks for finding that example. It looks like it is just the thing for this use case.

It confuses me a bit with the file names it uses, since they don't seem to be the same as the "normal" ones (I have id_ed25519, id_ed25519-cert.pub and id_ed25519.pub as files).

I think the private key that the example talks about must be my private key, not the private key of the CA. But what confuses me here is the talk about "creating a signer from the private key": somehow that seems more like something a CA would do (and hence use the CA private key).

Using my own private key is ok for me. As I understand it, the ssh client program also still uses it somehow when connecting to the server. For example, if you specify an identity file with the -i option, you specify the private key, even if you wish to use the certificate. (It finds the certificate automagically somehow). The ssh program doesn't know the CA private key.

The main advantage we have with the certificates is that we don't have to pre-create users with authorized_key files. If the secret key is used in the process, or not, isn't what we are trying to avoid. (In fact, the certificate is labeled ".pub", so by itself it could never serve to log in).

If there are different code paths to follow for both login procedures (with or without certificate), it would be perfectly reasonable to me if you decide based on the presence of a certificate_file key or something like that. Automatically finding a certificate (or finding out if it would be useful) would not be a requirement.

Does that make sense?

[oseiberts11]

I dabbled myself a bit (I'm not so used to go) and this is the result: #14
I did a single test to a single host and it didn't report failure any more.

[treydock]

@oseiberts11 I released #14 in v1.3.0-rc.0: https://github.com/treydock/ssh_exporter/releases/tag/v1.3.0-rc.0. If this works for you I will promote to v1.3.0.

[oseiberts11]

Thanks! I gave it a whirl, and it passed the tests and my own test as well. Looks good!

[treydock]

Released v1.3.0 with #14 that closes this issue.