Kubernetes Troubleshooting

To troubleshoot connection issues with the Gateway:

• Log in to the Twingate Client and sync the Kubernetes configuration if Auto Sync is not enabled.
• Switch the kubectl current context to the context added by Twingate.
  • Run kubectl config view --minify --output yaml to print the configuration of the current context.
• Run kubectl auth whoami to verify the connection is working.
• Check the Gateway logs for errors.
  • If there are no logs, the kubectl command never reached the Gateway. See Connector fails to connect to the Gateway.
• Check the Kubernetes resource's network events in the Twingate Admin Console.

Connector fails to connect to the Gateway

Symptoms

• When running kubectl auth whoami, you get an error message:

$ kubectl auth whoami
E0728 11:28:58.794311   10416 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://my-k8s-cluster.int/api?timeout=32s\": read tcp 100.96.0.2:51459->100.109.236.181:443: read: connection reset by peer - error from a previous attempt: read tcp 100.96.0.2:51458->100.109.236.181:443: read: connection reset by peer"

Diagnosis

• In the Twingate Admin Console, check the Kubernetes resource for network events. If the Connector fails to connect to the Gateway, you should see a "DNS lookup failed" event.
• There are no logs in the Gateway pod.

Possible causes

• The Gateway is deployed with a ClusterIP service (the default configuration), but some Connectors are deployed outside the Kubernetes cluster.
  • With the default ClusterIP setup, the Gateway is only accessible to Connectors deployed inside the cluster. If you have a Connector (in the same Remote Network) deployed outside the cluster, you need to use a LoadBalancer or NodePort service to expose the Gateway.
• The resource's Gateway address is incorrect or not resolvable by the Connector.

TLS misconfiguration

Symptoms

• When running kubectl auth whoami, you get an error message:

$ kubectl auth whoami
E1120 18:17:28.472082   18859 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://my-k8s-cluster.int/api?timeout=32s\": read tcp 100.96.0.2:63678->100.96.88.64:443: read: connection reset by peer - error from a previous attempt: read tcp 100.96.0.2:63675->100.96.88.64:443: read: connection reset by peer"

• In the Twingate Admin Console, the network event shows success but the duration is less than 1 second.
• There are no logs in the Gateway pod.

Diagnosis

• Use curl to test the connection to the Kubernetes API server via the Gateway:

  • Test using kubernetes.default.svc.cluster.local (the default resource address). This should succeed if you installed the Gateway using the provided Helm chart or Twingate Operator:

    curl -k https://kubernetes.default.svc.cluster.local -vvv
    

  • If the resource has an alias, the Twingate Client syncs a kubeconfig that uses the resource alias. Test curl with the resource alias. If it fails, the Gateway's TLS certificate doesn't include the resource alias:

    curl -k https://my-k8s-cluster.int -vvv
    

Possible causes

• This failure indicates the Twingate Client failed to complete the TLS handshake with the Gateway. Possible reasons:

  • The CA certificate in the Kubernetes resource configuration doesn't match the Gateway's TLS certificate.
  • The Gateway's TLS certificate doesn't include the resource address or alias.

  If you're using the provided Helm chart, ensure the resource alias is included in the tls.dnsNames list in the values.yaml file.

Client configuration

~/.kube directory doesn't exist

If the ~/.kube directory doesn't exist on the machine, the Twingate Client configuration sync will not work.

Resolution: Create the ~/.kube directory. After this, the configuration sync should succeed.

~/.kube/config content is not synced on macOS

Check the Twingate Client logs for the following error:

[ERROR] [client] Sync kubernetes resources finished: Failed to create kubeconfig at /var/root/.kube/config: Permission denied (os error 13)

This error indicates that the Twingate Client is running as the root user whose home directory is /var/root on macOS. This could happen if Twingate is installed via an MDM like Kandji.

Resolution: Make sure the Twingate Client is running as the current user. You can check the user running the Twingate Client by running:

ps aux | grep '[T]wingate.app'