x509: certificate signed by unknown authority

[joschi36]

What steps did you take and what happened:
Started octant on local machine with kubeconfig which points to internal domain.

unable to start runner: unable to start CRD watcher: crd watcher has failed: check access to watch CacheKey[APIVersion='apiextensions.k8s.io/v1beta1', Kind='CustomResourceDefinition']: unable to get resource for group kind CustomResourceDefinition.apiextensions.k8s.io: Get "https://srf-test.cargo.stxt.media.int/api?timeout=32s": x509: certificate signed by unknown authority

What did you expect to happen:
Started ocant web ui without certificate error as ca certificate is imported in debian (/etc/ssl/certs/ca-certificates.crt) and working in every other server.

Anything else you would like to add:

$ curl https://internal.domain.redacted/api\?timeout\=32s                                                                            
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "forbidden: User \"system:anonymous\" cannot get path \"/api\"",
  "reason": "Forbidden",
  "details": {
    
  },
  "code": 403
}%         

Environment:

• Octant version (use octant version): 0.13.0
• Kubernetes version (use kubectl version): v1.16.3
• OS (macOS 10.15, Windows 10, Ubuntu 19.10 etc): Debian Bullseye

[wwitzel3]

@joschi36 I've not seen this before. I'll take a look at how kubectl is dealing with unknown authority for certificates and model that behavior in Octant, probably as a flag like --allow-unknown-authority so it is explict.

[wwitzel3]

Just to confirm, kubectl is working on the same machine using the same kubeconfig that octant is failing with?

[joschi36]

Yes exactly, same computer and even same shell session.
Thanks for looking into this.

[aartij17]

I am facing a similar issue.

2020-07-16T14:11:45.232-0700	ERROR	api/navigation_manager.go:97	load namespaces	{"component": "websocket-client", "client-id": 
"f37b280c-c7a8-11ea-bd80-88e9fe868be2", "poller-name": "navigation", "poller-instance": "2952c2e6-b93d-45c0-82ff-ea631324c7bd", "err": "unable to generate 
navigation for module overview: generate entries for Custom Resources: listing custom resources for \"clusters.cluster.x-k8s.io\": check access to list 
CacheKey[Namespace='default',APIVersion='cluster.x-k8s.io/v1alpha2', Kind='Cluster']: unable to get resource for group kind Cluster.cluster.x-k8s.io: 
Get \"https://127.0.0.1:6443/api?timeout=32s\": x509: certificate signed by unknown authority (possibly because of \"crypto/rsa: verification error\" while trying 
to verify candidate authority certificate \"kubernetes\")"}

This happens when I copied over the kubeconfig from a remote machine running Linux to my local machine and ran octant -v

kubectl is working on the same machine using the same kubeconfig that octant is failing with.

[wwitzel3]

Thank you for adding info, more cases and logging helps us out here. I'll be sure to update as soon as I get a fix for this.

[wwitzel3]

Determine what the standard practice is here ignore vs. set custom CA

[bryanl]

If the CA cert is not in the kubeconfig, I could see this happening. To test this theory out, we could create a way to set up verification skips for insecure certs.

[tvvignesh]

Yup. I am facing the same issue as well. It used to work before. I upgraded to kubectl v1.19.2 added the proxy-url config (kubernetes/client-go#351) and removed an unwanted context from kubeconfig.

Now, kubectl works great but octant fails with this error:

UPDATE: Interestingly, using helm also fails if I don't prefix with the HTTPS_PROXY flag. Ideally it should be picked up from the kubeconfig. I get Error: Kubernetes cluster unreachable: Get "https://10.0.0.2/version?timeout=32s": dial tcp 10.0.0.2:443: i/o timeout