Kubelogin is not able to convert the kubeconfig to azurecli format for 1.24.0 AKS cluster

[gyuvaraj10]

Step Performed:

1. az login
2. az aks get-credentials -g "abc" -n "def"
3. kubelogin convert-kubeconfig -l azurecli

Actual config in the .kube/config file

- name: clusterUser_rc-a-aks-1
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - get-token
      - --environment
      - AzurePublicCloud
      - --server-id
      - xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      - --client-id
      - xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      - --tenant-id
      - xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      - --login
      - devicecode
      command: kubelogin
      env: null
      provideClusterInfo: false

Expected Config:

- name: clusterUser_rc-zaks-1
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - get-token
      - --server-id
      - xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      - --login
      - azurecli
      command: kubelogin
      env: null
      provideClusterInfo: false

Environment (please complete the following information):

• Kubectl Client CLI Version - 1.24.1
• Kubernetes version [e.g. 1.24.0]
• Kubelogin -
  git hash: v0.0.13/52e83a071f39e9e039e95aa9a6fbea04855eae13
  Go version: go1.17.9
  Build time: 2022-04-23T00:49:14Z

[gyuvaraj10]

This is a blocker for us to deploy the AKS cluster to 1.24.0

[slnowak]

+1, discovered same issue today after upgrading to 1.24.0

[weinong]

since 1.24.0, AKS default returns exec format kubeconfig so that it doesn't actually need kubelogin conversion. currently, kubelogin's convert-kubeconfig sub command doesn't do anything unless the kubeconfig is pre 1.24 format using azure auth. one can argue that convert-kubeconfig should probably convert the exec format as well to support different login method.

[weinong]

@gyuvaraj10, the kubeconfig you got from 1.24 cluster should be as good unless you require to use azurecli login?

[gyuvaraj10]

@weinong I like to connect to cluster and run some commands (example: 'kubectl get nodes') with in a CI pipeline. So, I want to connect to the cluster in non-interactive mode. Can you suggest the approach for the non-interactive communication to the cluster if kubelogin doesn't support.

[weinong]

@gyuvaraj10 how would azurecli mode help? do you use a service principal to login?

[slnowak]

Azurecli can be use in a non-interactive mode, as per https://github.com/Azure/kubelogin#azure-cli-token-login-non-interactive

[AshokPadarthi]

@weinong if we are performing az cli authentication using SPN and get the config using SPN then expectation would be that Kubectl commands execution happens non-interactively which is not the case. its asking us to sign in (SPM is meant for M2M)

Even though the config is downloaded in the context of authenticated SPN, the EXEC has property --login with value "devicecode" which is leading to interactive mode

[weinong]

we are working on the fix!

[anmathew]

Hi,
Please try with v0.0.14 release and let us know if this can be closed.

[davidbgonz]

Just recently updated kubelogin. On a 1.22 AKS cluster, convert-kubeconfig for azurecli is no longer working for me. The command runs fine, but any kubectl command I get the deprecation warning message as if I never ran the convert command.

[anmathew]

$ kubectl get nodes
W0708 10:02:09.364656   77597 azure.go:92] WARNING: the azure auth plugin is deprecated in v1.22+, unavailable in v1.25+; use https://github.com/Azure/kubelogin instead.
To learn more, consult https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code RE6TRRUJT to authenticate.
^C
$ kubelogin --version
kubelogin version 
git hash: v0.0.14/f345047a580aaaf133b009041963d50b98d8d2e2
Go version: go1.17.11
Build time: 2022-07-07T17:00:54Z
$ kubelogin convert-kubeconfig -l azurecli 
$ kubectl get nodes
NAME                                STATUS   ROLES   AGE   VERSION
aks-nodepool1-14804407-vmss000000   Ready    agent   53m   v1.22.6


$ kubectl version --short
Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
Client Version: v1.24.0
Kustomize Version: v4.5.4
Server Version: v1.22.6
WARNING: version difference between client (1.24) and server (1.22) exceeds the supported minor version skew of +/-1

@davidbgonz, unable to repro this on v1.22.6 - can you please provide details to repro and of your Environment?

[davidbgonz]

You are getting the same result as on my end.

❯ kubectl version --short
Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
W0708 15:12:34.922692  125926 azure.go:92] WARNING: the azure auth plugin is deprecated in v1.22+, unavailable in v1.25+; use https://github.com/Azure/kubelogin instead.
To learn more, consult https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins
Client Version: v1.24.2
Kustomize Version: v4.5.4
Server Version: v1.22.6
WARNING: version difference between client (1.24) and server (1.22) exceeds the supported minor version skew of +/-1

The behavior prior to updating was after running kubelogin convert-kubeconfig -l azurecli I no longer needed to authenticate through the browser and the warning message went away. I'm guessing this is due to the change in the kubeconfig.

[weinong]

@davidbgonz i'm not following. are you saying you are still being prompted to login after converting to azurecli using 0.0.14? Would you mind sharing the converted kubeconfig? you can omit the fqdn and CA to make it generic

[davidbgonz]

@weinong That was the case, but actually I just got it working again. There was some funny business going on with my config.

The steps I ran:

• deleted my config
• added back one my clusters using the az aks get-credentials command
• ran kubelogin convert-kubeconfig -l azurecli again

I am not sure why the config file got in such a state. I also had to remove a leftover config.lock that would not clear up. This was something else I noticed after the update but not sure if it's related since I was messing around with the az and kubelogin commands.

[weinong]

@davidbgonz no problem. If you encounter the issue consistently, please open a new issue with the kubeconfig before and after conversion.