Use Identity::from_pem

[mkpankov]

Close #1731

---

This change is 

[mkpankov]

@jgrund making sure this works with a "proper" certificate proves to be harder than I thought.

I can't use LetsEncrypt because they want a public-facing web-server or DNS record, which doesn't make sense for our vagrant cluster or the isolated production environment it mimics.

I currently think the only way to actually setup this in production is to use self-signed certificates.

At one of my previous employers we had corporate certificates derived from custom Root CA, and that root CA had to be installed manually on each machine and browser separately to add it to chain of trust. So that root CA was self-signed, and I don't think using non-self-signed certificate for this kind of setup makes sense.

Point is, I'm not sure anyone buys proper certificates from 3-rd party public authority for TLS on intranet. If that's actually what we want to support, I guess we should get a proper certificate and PK from a provider like digicert.

[ip1981]

There is no such thing like a "proper" certificate (except algorithms and expiration). A certificate is either trusted or not (via a chain of certificates, also known as intermediate certificates). That is whether a certificate is signed by a certificate which is signed by a certificate and so on ... by a certificate which you trust :)

A root CA is always self-signed, if some "root CA" is not self-signed, it's an intermediate certificate.

[mkpankov]

I'm saying the same, so it's not clear what we wanted to test with "not a self-signed client certificate". We aren't expecting our clients to obtain certificates with publicly trusted root CA (which are usually paid, not free), are we?

[mkpankov]

I've removed danger_accept_invalid_certs from ClientBuilder invocation and the tests pass, so this should work in all scenarios.

[johnsonw]

Hi @mkpankov,

The integration tests are picking up the following error from the rust-iml-agent on the storage servers when detecting the filesystem :

Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.472#033[0m #033[33m WARN#033[0m rustls::session: Sending fatal alert BadCertificate
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.472#033[0m #033[33m WARN#033[0m iml_agent::reader: Got a manager read Error reqwest::Error { kind: Request, url: "https://adm.local/agent2/message/?server_boot_time=2020-07-13T10%3A09%3A13.000000%2B00%3A00Z&client_start_time=2020-07-13T10%3A09%3A53.579173%2B00%3A00Z", source: hyper::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: WebPKIError(UnknownIssuer) } }) }. Will retry in 5 seconds.
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.472#033[0m #033[32m INFO#033[0m iml_agent::http_comms::session: Terminating all sessions
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.603#033[0m #033[32m INFO#033[0m iml_agent::poller: sending session create request for ntp
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::http_comms::agent_client: Requesting new session for: PluginName("ntp").
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::poller: sending session create request for action_runner
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::http_comms::agent_client: Requesting new session for: PluginName("action_runner").
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::poller: sending session create request for ostpool
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::http_comms::agent_client: Requesting new session for: PluginName("ostpool").
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::poller: sending session create request for stats
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::http_comms::agent_client: Requesting new session for: PluginName("stats").
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::poller: sending session create request for device
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::http_comms::agent_client: Requesting new session for: PluginName("device").
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::poller: sending session create request for postoffice
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.604#033[0m #033[32m INFO#033[0m iml_agent::http_comms::agent_client: Requesting new session for: PluginName("postoffice").
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.616#033[0m #033[33m WARN#033[0m rustls::session: Sending fatal alert BadCertificate
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.616#033[0m #033[33m WARN#033[0m rustls::session: Sending fatal alert BadCertificate
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.616#033[0m #033[32m INFO#033[0m iml_agent::poller: session create request for postoffice failed: Reqwest(reqwest::Error { kind: Request, url: "https://adm.local/agent2/message/", source: hyper::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: WebPKIError(UnknownIssuer) } }) })
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.616#033[0m #033[32m INFO#033[0m iml_agent::poller: session create request for device failed: Reqwest(reqwest::Error { kind: Request, url: "https://adm.local/agent2/message/", source: hyper::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: WebPKIError(UnknownIssuer) } }) })
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.618#033[0m #033[33m WARN#033[0m rustls::session: Sending fatal alert BadCertificate
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.618#033[0m #033[32m INFO#033[0m iml_agent::poller: session create request for ostpool failed: Reqwest(reqwest::Error { kind: Request, url: "https://adm.local/agent2/message/", source: hyper::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: WebPKIError(UnknownIssuer) } }) })
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.618#033[0m #033[33m WARN#033[0m rustls::session: Sending fatal alert BadCertificate
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.619#033[0m #033[32m INFO#033[0m iml_agent::poller: session create request for ntp failed: Reqwest(reqwest::Error { kind: Request, url: "https://adm.local/agent2/message/", source: hyper::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: WebPKIError(UnknownIssuer) } }) })
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.619#033[0m #033[33m WARN#033[0m rustls::session: Sending fatal alert BadCertificate
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.619#033[0m #033[32m INFO#033[0m iml_agent::poller: session create request for stats failed: Reqwest(reqwest::Error { kind: Request, url: "https://adm.local/agent2/message/", source: hyper::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: WebPKIError(UnknownIssuer) } }) })
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.625#033[0m #033[33m WARN#033[0m rustls::session: Sending fatal alert BadCertificate
Jul 13 10:15:39 oss1 iml-agent-daemon: #033[2mJul 13 10:15:39.625#033[0m #033[32m INFO#033[0m iml_agent::poller: session create request for action_runner failed: Reqwest(reqwest::Error { kind: Request, url: "https://adm.local/agent2/message/", source: hyper::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: WebPKIError(UnknownIssuer) } }) })

Ultimately, it fails to detect the filesystem.

[mkpankov]

@johnsonw thanks Will. Did you go to the machine to see that or it's possible to see in Jenkins logs?

[mkpankov]

I've split the patches and this branch passes the tests.

[jgrund]

There are a few outstanding comments, otherwise LGTM

[johnsonw]

@johnsonw thanks Will. Did you go to the machine to see that or it's possible to see in Jenkins logs?

I was able to get the information from the sos reports (part of the build artifacts)

[mkpankov]

Follow-up issue: #2071