Deployment with Gitlab-Agent/KAS fails

[strowi]

Bug report

Using the Gitlab-Agent to deploy to K8s with Shopify/krane/ fails.

Expected behavior: Successful Deployment

Actual behavior: krane fails to kubectl get --raw /k8s-proxy

Version(s) affected: 2.4.7

Steps to Reproduce

1. setup + deploy gitlab-agent to a Rancher Kubernetes cluster.
2. try deploying with shopify/krane
3. alternatively try kubectl get --raw /k8s-proxy directly:

stages:
  - deploy

deploy:
  stage: deploy
  image:
    name: registry.gitlab.com/strowi/deploy:latest
  script:
    - kubectl config use-context "gitlab-agent:context"
    - kubectl create ns test
    - krane deploy test $KUBE_CONTEXT -f manifests/test.yml

Output:

...
# krane 2.4.7
# testing k8s-connection:
Kubernetes control plane is running at https://kas.gitlab.com/k8s-proxy
CoreDNS is running at https://kas.gitlab.com/k8s-proxy/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

NAME   STATUS   AGE
test   Active   3s
# deploying to test... 
[INFO][2022-05-25 13:58:36 +0000] 
[INFO][2022-05-25 13:58:36 +0000] ---------------------------------Phase 1: Initializing render task----------------------------------
[INFO][2022-05-25 13:58:36 +0000] Validating configuration
[INFO][2022-05-25 13:58:36 +0000] 
[INFO][2022-05-25 13:58:36 +0000] -----------------------------------Phase 2: Rendering template(s)-----------------------------------
[INFO][2022-05-25 13:58:36 +0000] Rendering nginx.yml...
[INFO][2022-05-25 13:58:36 +0000] Rendered nginx.yml
[INFO][2022-05-25 13:58:36 +0000] 
[INFO][2022-05-25 13:58:36 +0000] ------------------------------------------Result: SUCCESS-------------------------------------------
[INFO][2022-05-25 13:58:36 +0000] Successfully rendered 1 template(s)
[INFO][2022-05-25 13:58:36 +0000] 
[INFO][2022-05-25 13:58:37 +0000] 
[INFO][2022-05-25 13:58:37 +0000] ------------------------------------Phase 1: Initializing deploy------------------------------------
[INFO][2022-05-25 13:58:39 +0000] All required parameters and files are present
[INFO][2022-05-25 13:58:39 +0000] Discovering resources:
[WARN][2022-05-25 13:58:41 +0000] The following command failed and will be retried (attempt 1/5): kubectl get --raw /k8s-proxy --context\=gitlab-agent:context --request-timeout\=15
[WARN][2022-05-25 13:58:41 +0000] Error from server: the server responded with the status code 426 but did not return more information
[WARN][2022-05-25 13:58:41 +0000] The following command failed and will be retried (attempt 2/5): kubectl get --raw /k8s-proxy --context\=gitlab-agent:context --request-timeout\=15
[WARN][2022-05-25 13:58:41 +0000] Error from server: the server responded with the status code 426 but did not return more information
[WARN][2022-05-25 13:58:43 +0000] The following command failed and will be retried (attempt 3/5): kubectl get --raw /k8s-proxy --context\=gitlab-agent:context --request-timeout\=15
[WARN][2022-05-25 13:58:43 +0000] Error from server: the server responded with the status code 426 but did not return more information
[WARN][2022-05-25 13:58:47 +0000] The following command failed and will be retried (attempt 4/5): kubectl get --raw /k8s-proxy --context\=gitlab-agent:context --request-timeout\=15
[WARN][2022-05-25 13:58:47 +0000] Error from server: the server responded with the status code 426 but did not return more information
[WARN][2022-05-25 13:58:55 +0000] The following command failed (attempt 5/5): kubectl get --raw /k8s-proxy --context\=gitlab-agent:context --request-timeout\=15
[WARN][2022-05-25 13:58:55 +0000] Error from server: the server responded with the status code 426 but did not return more information
[INFO][2022-05-25 13:58:55 +0000] 
[INFO][2022-05-25 13:58:55 +0000] ------------------------------------------Result: FAILURE-------------------------------------------
[FATAL][2022-05-25 13:58:55 +0000]  Error retrieving raw path /: error from server: the server responded with the status code 426 but did not return more information
[FATAL][2022-05-25 13:58:55 +0000]  

Not sure yet if this is a problem with krane or gitlab restricting access. But since normal kubectl works i thought i'd start somewhere.
Any help debugging would be appreciated.;)

regards,
strowi