remove basic-auth from k8s-master

[kwmonroe]

Fixes: https://bugs.launchpad.net/charm-kubernetes-master/+bug/1841226

Quite a bit goes into this:

• deprecate the dashboard-auth config option
• s/setup_basic_auth/setup_tokens for the admin user
• ensure setup_tokens updates existing tokens instead of just appending new rows
• on upgrade, merge leader basic_auth.csv into known_tokens.csv and genericize basic_auth.csv
• ensure leader sets appropriate data so basic_auth.csv is genericized across the cluster
• generate kube_configs based on tokens instead of passwords

[kwmonroe]

Tested as follows:

• CK 1.19/edge deployed with custom basic_auth.csv entry (non-custom entries would be the same; just wanted to make sure those made it to known_tokens.csv):
  • fails to start apiserver due to invalid --basic-auth-file param (status is Stopped services: kube-apiserver,kube-controller-manager)
  • journalctl -xe shows kube-apiserver.daemon[17573]: Error: unknown flag: --basic-auth-file
  • state of *.csv and kube configs:

$ juju run --application kubernetes-master 'cat /root/cdk/basic_auth.csv && echo && cat /root/cdk/known_tokens.csv && echo && tail -5 /root/.kube/config && echo && tail -5 /home/ubuntu/config'
- Stderr: ""
  Stdout: |
    bMLJDSyGLzHIlknFEXmvy2njzyJ9peGl,admin,admin,system:masters
    password,kwm,kwmonroe,"system:masters,system:nodes"
    password,tes,test

    pfxD4F17xlrKZhGqpXQJFwBScDAy4w4T,system:kube-controller-manager,kube-controller-manager
    2hNQ6wmo0Z1YA3tyCXJczrDyTu2G4tsl,system:kube-proxy,kube-proxy
    HGUA510qr9aN4o3DOmF7AqQIcvUM1Cxx,admin,admin,"system:masters"
    V8LpmeAUHnDe50N2c3AWxOzJGhs3AeC2,system:monitoring,system:monitoring
    qcKjtoiMdygDDZ1KyCWyardeTTUdJPMF,system:node:ip-172-31-7-119,kubelet-0,"system:nodes"

    users:
    - name: admin
      user:
        password: bMLJDSyGLzHIlknFEXmvy2njzyJ9peGl
        username: admin

    users:
    - name: admin
      user:
        password: bMLJDSyGLzHIlknFEXmvy2njzyJ9peGl
        username: admin
  UnitId: kubernetes-master/0
- Stderr: ""
  Stdout: |
    bMLJDSyGLzHIlknFEXmvy2njzyJ9peGl,admin,admin,system:masters
    password,kwm,kwmonroe,"system:masters,system:nodes"
    password,tes,test


    pfxD4F17xlrKZhGqpXQJFwBScDAy4w4T,system:kube-controller-manager,kube-controller-manager
    2hNQ6wmo0Z1YA3tyCXJczrDyTu2G4tsl,system:kube-proxy,kube-proxy
    HGUA510qr9aN4o3DOmF7AqQIcvUM1Cxx,admin,admin,"system:masters"
    V8LpmeAUHnDe50N2c3AWxOzJGhs3AeC2,system:monitoring,system:monitoring
    qcKjtoiMdygDDZ1KyCWyardeTTUdJPMF,system:node:ip-172-31-7-119,kubelet-0,"system:nodes"

    users:
    - name: admin
      user:
        password: bMLJDSyGLzHIlknFEXmvy2njzyJ9peGl
        username: admin

    users:
    - name: admin
      user:
        password: bMLJDSyGLzHIlknFEXmvy2njzyJ9peGl
        username: admin
  UnitId: kubernetes-master/1

• Now upgrade-charm to a build with this PR
  • services are running with k-m in active/idle status
  • both *.csv have been updated, with known_tokens.csv and kubeconfigs now containing the admin token from previous basic_auth.csv:

$ juju run --application kubernetes-master 'cat /root/cdk/basic_auth.csv && echo && cat /root/cdk/known_tokens.csv && echo && tail -5 /root/.kube/config && echo && tail -5 /home/ubuntu/config'
- Stderr: ""
  Stdout: |
    # Basic auth entries have moved to known_tokens.csv

    NM2NrfwxGhvStj3h5IQyw7fRfOj3cSaX,system:kube-controller-manager,kube-controller-manager,
    D23FwsrqZRTBcm8C9PWYmHQQdpFhBZBV,system:kube-proxy,kube-proxy,
    kAinMNEZK1elIcO451JkObBy3LBCiDCd,admin,admin,system:masters
    X1vqKAq9CEEYR9ZCbQTeIClsiwE84Njv,system:monitoring,system:monitoring,
    B0MDGMWQ8Kz81lwhKbTUOaBkhrRcVsEO,system:node:ip-172-31-16-86,kubelet-0,system:nodes
    1t1hgoE5pYEFZwNudG4UeqcRC4Vf4sxU,system:node:ip-172-31-5-224,kubelet-1,system:nodes
    qo2o1GJZneFmajXtkSORLmAwwO3PWm2o,system:node:ip-172-31-6-7,kubelet-2,system:nodes
    password,kwm,kwmonroe,"system:masters,system:nodes"
    password,tes,test,

    preferences: {}
    users:
    - name: admin
      user:
        token: kAinMNEZK1elIcO451JkObBy3LBCiDCd

    preferences: {}
    users:
    - name: admin
      user:
        token: kAinMNEZK1elIcO451JkObBy3LBCiDCd
  UnitId: kubernetes-master/0
- Stderr: ""
  Stdout: |
    # Basic auth entries have moved to known_tokens.csv

    NM2NrfwxGhvStj3h5IQyw7fRfOj3cSaX,system:kube-controller-manager,kube-controller-manager,
    D23FwsrqZRTBcm8C9PWYmHQQdpFhBZBV,system:kube-proxy,kube-proxy,
    kAinMNEZK1elIcO451JkObBy3LBCiDCd,admin,admin,system:masters
    X1vqKAq9CEEYR9ZCbQTeIClsiwE84Njv,system:monitoring,system:monitoring,
    B0MDGMWQ8Kz81lwhKbTUOaBkhrRcVsEO,system:node:ip-172-31-16-86,kubelet-0,system:nodes
    1t1hgoE5pYEFZwNudG4UeqcRC4Vf4sxU,system:node:ip-172-31-5-224,kubelet-1,system:nodes
    qo2o1GJZneFmajXtkSORLmAwwO3PWm2o,system:node:ip-172-31-6-7,kubelet-2,system:nodes
    password,kwm,kwmonroe,"system:masters,system:nodes"
    password,tes,test,

    preferences: {}
    users:
    - name: admin
      user:
        token: kAinMNEZK1elIcO451JkObBy3LBCiDCd

    preferences: {}
    users:
    - name: admin
      user:
        token: kAinMNEZK1elIcO451JkObBy3LBCiDCd
  UnitId: kubernetes-master/1

[hyperbolic2346]

🍖