Installer: self-signs certs cause installation failure

[sehz]

steps reproduce it:

1. Create Kubernetes cluster with self-signed certs
2. Installer will set up the cluster but SC and SPU pods are running correctly.

Please see: https://gist.github.com/tarokkk/be24d9c1fde100b21150fa26a6031ce9. Installer terminates with error:

Error:
   0: Fluvio cluster error
   1: Failed to install Fluvio on Kubernetes
   2: Kubernetes client error
   3: error trying to connect: tls handshake: HandshakeError: the handshake failed: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1916:: unable to get issuer certificate
   4: error trying to connect: tls handshake: HandshakeError: the handshake failed: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1916:: unable to get issuer certificate
   5: tls handshake: HandshakeError: the handshake failed: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1916:: unable to get issuer certificate
   6: tls handshake: HandshakeError: the handshake failed: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1916:: unable to get issuer certificate

This is probably because K8 library doesn't add ca.crt in kubernetes.io/service-account-token.

kubectl get secret -n fluvio fluvio-token-5x5hb -o=jsonpath='{.data.ca.crt}' | base64 -D | openssl x509 -text                             *[master]
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            [xxx]
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=cluster-7133-ca
        Validity
            Not Before: Feb 14 09:37:46 2022 GMT
            Not After : Feb 13 10:38:16 2027 GMT
        Subject: CN=kubernetes-ca

[tarokkk]

Just to make clear that is 2 different error:

1. During the installation the rust Kubernetes client does not use/satisfied with the ca.crt in the KubeConfig it tries to search for the Issuer of that certificate. (curl --cacert ca.crt https://x.x.x.x:6443/ work as expected)
2. The fluvio service in-cluster config fails to authenticate probably mishandling kubernetes.io/service-account-token (described in the issue)

[wendyOn]

So OpenSSL - which is used via k8-client - has this X509_V_FLAG_PARTIAL_CHAIN thing - changed in OpenSSL 1.1.0
openssl/openssl#7871

a lot of - including node - people have hit this with intermediary self-signed certs
nodejs/node#36453 (comment)

curl has it enabled it by default
curl/curl#4655

This is why in curl it works.

[wendyOn]

Hi, I've repro'd and proposed a feature and added tests to it here at client level:

• [Merged by Bors] - feat: partial CA trust anchor future-aio#134 (comment) (openssl s_client)
• [Merged by Bors] - feat: partial CA trust anchor future-aio#134 (comment) (TlsClient level used by fluvio)

@tarokkk would it be possible to quickly see what kind of configuration so I can repro as-is using similar environment?

I think I've created pretty accurate repro but in case if you got that handy somewhere to emulate so I didn't miss any little thing-

Additional integration test could be always useful.

Thanks