Cannot connect with Oracle Cloud OCI

[sambonbonne]

Describe the bug

I started a Kubernetes cluster on Oracle Cloud (OKE service) and installed the Telepresence traffic manager through the Helm chart with ArgoCD without any custom values in the telepresence namespace.

To use kubectl with OKE clusters, I have to use the oci cli, configure it in ~/.oci/ and i have the following user in the kubeconfig file:

users:
    - name: user-[REDACTED]
      user:
        exec:
            apiVersion: client.authentication.k8s.io/v1beta1
            command: oci
            args:
                - ce
                - cluster
                - generate-token
                - --cluster-id
                - [REDACTED]
                - --region
                - [REDACTED]
            env: []

When using kubetl get nodes to try the connection, it works perfectly.

When manully using the oci ce cluster generate-token … command, I see the response from Oracle Cloud with a generated token.

But when running telepresence connect --manager-namespace telepresence, it seems the oci command fails.

Here are the "direct" logs from the command:

telepresence connect: error: connector.Connect: initial cluster check failed: Get "https://89.168.59.197:6443/version": getting credentials: exec: executable oci failed with exit code 1

See logs for details (1 error found): "/home/sam/.cache/telepresence/logs/connector.log"
If you think you have encountered a bug, please run `telepresence gather-logs` and attach the telepresence_logs.zip to your github issue or create a new one: https://github.com/telepresenceio/telepresence/issues/new?template=Bug_report.md .

I ran telepresence gather-logs but it gives me an empty ZIP file because no logs are produced in ~/.cache/telepresence/logs/.

To Reproduce

Steps to reproduce the behavior:

1. Install the oci cli and configure it
2. Create a cluster in Oracle Cloud
3. Generate the kubeconfig file for this cluster
4. Try to use it with kubectl to ensure it works
5. Install Telepresence Helm chart in the cluster and try to use telepresence connect

Expected behavior

Telepresence creates the daemon and I can launch intercepts.

Current behavior

Telepresence fails to run.

Versions

$ telepresence version
OSS Client     : v2.18.0
OSS Root Daemon: v2.17.0
OSS User Daemon: v2.18.0
Traffic Manager: not connected

$ kubectl version
Client Version: v1.29.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.1

OCI CLI: 3.39.0

OS: Arch Linux

I am not using any VPN.

[cindymullins-dw]

if you need certain permissions to access your cluster then Telepresence also needs to see those permissions. So, for example, if you happen to connect Telepresence without the credentials needed to access the cluster this error can happen. So there might be a permissions issue with Oracle Cloud. But first could you try telepresence quit -s, and then try connecting again? If its an order of operations thing that would resolve it, so it'd be good to rule that out.

[sambonbonne]

@cindymullins-dw I just tried to run telepresence quit -s and run telepresence connect --manager-namespace telepresence again but I had the exact same issue.

As you see in the kubeconfig file part in my issue, the access to the cluster is managed by the oci CLI which is configured in the ~/.oci/config file. The command itself and kubectl works well, only Telepresence has an issue with oci.

Can I try anything else to help debugging this issue?

[sambonbonne]

So, I may have find a way to make it work. On another computer, I didn't want to install the OCI CLI so I changed the users section of the kubeconfig file to run the CLI in a container, like this:

users:
    - name: user-[REDACTED]
      user:
        exec:
            apiVersion: client.authentication.k8s.io/v1beta1
            command: podman
            args:
                - run
                - --rm
                - --tty
                - --volume
                - /home/my-user/.oci:/oracle/.oci:U
                - ghcr.io/oracle/oci-cli:latest
                - ce
                - cluster
                - generate-token
                - --cluster-id
                - [REDACTED]
                - --region
                - [REDACTED]
            env: []

And I don't have ay error when using telepresence connect.

This is a nice workaround but I don't understand why it does not work correctly when using the OCI CLI directly.

[cindymullins-dw]

Thanks very much, @sambonbonne for sharing that workaround. I'll mark this as a feature request in relation to the Oracle Cloud OCI since I don't see a mention (overt support) of it in our docs.

[thallgren]

But when running telepresence connect --manager-namespace telepresence, it seems the oci command fails.

One thing that comes to mind is that you're using oci without a path. That might explain why it works in a container.

[thallgren]

@sambonbonne Did you try with a full path to the oci binary? I'd like to close this one, because we have regression tests that verify the behavior of exec type identification, and they work as expected.

[thallgren]

Closing this, because no more info was provided.