Skip to content

🐛 Connect to EKS cluster via WARP/Cloudflared tunnel #669

@ghost

Description

Describe the bug
I am having some trouble using a cloudflared tunnel to connect to my kubernetes clusters. I have multiple existing k8s clusters hosted in AWS EKS with cloudflared running, and tunnels in each cluster that are currently routing to various http services, all of which are working as expected. However, when following the instructions from Cloudflare (Connect Through Cloudflare Access Using Kubectl), I am seeing the error: Unable to connect to the server: EOF when attempting to connect via a client machine.

As per another suggestion, I have tried to do this using the WARP client but the result is the same, leading me to believe that this could be an issue from the AWS side of things. Although I don't know what that would be.

To Reproduce
Steps to reproduce the behavior:
I have followed all of the steps in the document provided above. From the start, I have:

  • Created a zero trust policy for my application
  • Installed the cloudflared deployment on my cluster
  • Created and configured a tunnel using ingress rules
  • Created a DNS record to route traffic to the tunnel
  • Run the tunnel

The last step, attempting to connect via cloudflared access tcp, brings me to the error mentioned above. I have also attempted this using the WARP client, including:

  • Install and configure the warp client with my org
  • Set up HTTP filtering + gateway proxy
  • Turn on TCP proxy

My ingress looks like this:

tunnel: redacted
credentials-file: /etc/cloudflared/creds/cloudflared
warp-routing:
  enabled: true
metrics: 0.0.0.0:2000
no-autoupdate: true
ingress:
  - hostname: redacted
    service: redacted (ROUTES TO HTTP SERVICE, WORKING CORRECTLY)
  - hostname: redacted
    service: redacted (ROUTES TO HTTP SERVICE, WORKING CORRECTLY)
  - hostname: test-url.redacted.ai
    service: tcp://kubernetes.default.svc:443 (as opposed to kubernetes.docker.internal:6443 in CF documentation, as this is for EKS)
    originRequest:
      proxyType: socks
  # any traffic which didn't match a previous rule, and responds with HTTP 404.
  - service: http_status:404

Attempting to connect via another machine looks like this:

create connection to Cloudflare: cloudflared access tcp --hostname test-url.redacted.ai --url 127.0.0.1:8080
Then in another terminal window: env HTTPS_PROXY=socks5://127.0.0.1:8080 kubectl get po

Expected behavior
Expected behavior is to have the ability to run kubectl commands via this configuration.

Environment and versions
Local machine attempting to connect is MacOS Montery v12.1. Kubernetes clusters are hosted in AWS using EKS managed node groups.

Logs and errors
Running cloudflared access tcp in debug mode provides this log when attempting to use kubectl:

DBG tunnel to origin copy: readfrom tcp 127.0.0.1:1234->127.0.0.1:52078: websocket: close 1006 (abnormal closure): unexpected EOF
DBG origin to tunnel copy: read tcp 127.0.0.1:1234->127.0.0.1:52078: use of closed network connection

Additional context
Any help with this is greatly appreciated, as the Cloudflare docs are limited and there doesn’t seem to be much information about this particular use case online.

Metadata

Metadata

Assignees

Labels

Priority: NormalMinor issue impacting one or more usersType: BugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions