AKS v1.16.7 dashboard login not working on Chrome/Firefox

[ITRON-kpalmberg]

What happened:
After upgrading from AKS v1.15.7 to v1.16.7 the Kubernetes dashboard login is not functioning on Chrome/Firefox web browsers. Note this is related to the new token authentication requirement in the latest dashboard version. After entering a token and attempting to sign in, users are not able to login to the dashboard and no errors are returned indicating login failure (i.e. Forbidden errors). When clicking the login button nothing seems to happen and the browser just stays at the login page. 200 status codes can be seen in browser debugging tools after clicking login button. When entering the same exact token on Microsoft Edge the dashboard logs in as expected!

What you expected to happen:
Kubernetes dashboard is available on Chrome/Firefox in the same manner as it's accessible on Microsoft Edge on AKS v1.16.7.

How to reproduce it (as minimally and precisely as possible):

• Use either az aks browse or kubectl proxy to access dashboard login page.
• Once on dashboard login screen, enter a valid authentication token or kube config file. (if you want to use token instead of kube config file, you can run az account get-access-token --query accessToken -o tsv)
• Click the login button on either Chrome/Firefox/Edge. Currently this only works on Edge for our cluster for user accounts. Chrome/Firefox browsers just stay on the login page.

Anything else we need to know?:

• The login process works flawlessly on Microsoft Edge.
• Issue is present on multiple separate AKS clusters (not isolated to 1 cluster).
• Issue is seen on multiple developer machines (not isolated to 1 client).
• Originally we thought there might be an RBAC issue until we discovered Edge worked just fine.
• Per the comment above, we have RBAC enabled clusters.
• The AKS admin account and service account tokens work properly when logging in via Chrome. But when using a user account token this only works on Edge.

Environment:

• Kubernetes version (use kubectl version): 1.15 (also occurs with latest version 1.18.0)
• Az CLI version: 2.5.1
• Size of cluster (how many worker nodes are in the cluster?): 5 nodes

[AlucardX7]

Is anyone looking at above issue?

[jluk]

We're investigating the other related dashboard issues, but I have not encountered this one.

I am able to login to the dashboard via chrome on a 1.16.7 cluster. Could you open a support request and share the SR# so we can diagnose the cluster a bit more? cc @robbiezhang

[omid-tansaz]

@jluk Hi Justin,

We have the exact same problem. The only addition to above use case is that we use PIM.
All PIM enabled users, even though they are activated as subscription owner and visible as owner in IAM section as well as member of aks admin group, cannot access the dashboard by providing their access token.

However, following the same procedure with any non-PIM activated admin/user, the provided token allows access to dashboard.

Another noticeable event, using the access token of non-PIM user that does not have access, will cause a 401 error in login screen of the dashboard, while PIM enabled users don't even get an error. It just doesn't do anything.

Other kubectl command work just fine for PIM activated user tokens.

[ITRON-kpalmberg]

@nexxbizdev @jluk: I believe this is closely related to this GitHub issue: Issue#2981

The problem is that the token used to authentication can often be too long. This results in a jweToken cookie size which is too large for most browsers to handle. When this occurs the dashboard simply does nothing when you enter a token and click the login button (no error messages). However, when using the browser debugger on FireFox I can see a warning in the console about the jweToken cookie being too large. For some reason I don't see this warning on Chrome but it seems like the same login issue behavior. Also, to some of my points in the original post I believe why the admin account and service accounts may work on Chrome/Firefox is because their auth tokens are much smaller, therefore resulting in a jweToken cookie which is of acceptable size.

Also @jluk, I can create a SR but this issue is present in multiple AKS clusters. All clusters which have upgraded to v1.16.7 have this issue (since that version comes with the new dashboard).

[omid-tansaz]

I can confirm that the tokens of PIM users we have tested (may have nothing to do with PIM) are indeed larger than 5KB!

[ITRON-kpalmberg]

I also created a similar post over on the Kubernetes dashboard GitHub after this issue here was stale for a while which can be located here. The suggested solution can be found in this comment by one of the members of the k8s dashboard project. I feel a number of Azure users are going to be running into this issue after upgrading their clusters to v1.16.7.

[ghost]

Action required from @Azure/aks-pm

[ghost]

Issue needing attention of @Azure/aks-leads

[palma21]

Closing as per answer from both AKS team and dashboard contributors on the dahsboard issue.

Also AKS just released: https://azure.microsoft.com/en-us/updates/kubernetes-resource-view-is-in-public-preview/