Help! k3s v0.7.0 generate userKey with ca, authenticate in failure #684

huapox · 2019-07-28T12:13:02Z

I generated a userKey by ca, with cluster-admin rbac permission.

but got this:

kc --kubeconfig=koper.kubeconfig get pod
error: You must be logged in to the server (Unauthorized)

k3s server's log:

server_1    | E0728 19:15:27.324625       8 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate signed by unknown authority

my userKey generate scripts:

ws=/opt/sec-rbac
day=3650

clus_name="t1.k3s"
clus_ns="default"
user="koper"
clus_url="https://10.200.100.183:7442"
ca_path=$ws

ctx=gen && mkdir -p $ws/$ctx/{kube,keys} && cd $ws/$ctx
#############
generate="keys/u-"$user
echo -e "\033[32m#>>GEN-KEY\033[0m"
openssl genrsa -out $generate.key 2048
#openssl ecparam -name prime256v1 -genkey -noout -out $generate.key
openssl req -new -key $generate.key -out $generate.csr -subj "/CN=${user}@${clus_name}/O=key-gen"
openssl x509 -req -in $generate.csr -CA $ca_path/ca.crt -CAkey $ca_path/ca.key -CAcreateserial -out $generate.crt -days $day

#-----------
ctx2="$user@$clus_name"
config="kube/$user.kubeconfig"
echo -e "\033[32m#>>KUBE-CONFIG\033[0m"	
kubectl --kubeconfig=$config config set-cluster $clus_name --embed-certs=true --server=$clus_url --certificate-authority=$ca_path/ca.crt
kubectl --kubeconfig=$config config set-credentials $user --embed-certs=true --client-certificate=$generate.crt  --client-key=$generate.key
kubectl --kubeconfig=$config config set-context $ctx2 --cluster=$clus_name --namespace=$clus_ns --user=$user
kubectl --kubeconfig=$config config set current-context $ctx2
kubectl --kubeconfig=$config --context=$ctx2 get pods

The text was updated successfully, but these errors were encountered:

huapox · 2019-07-28T12:21:35Z

the generated file:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJWekNCL3FBREFnRUNBZ0VBTUFvR0NDcUdTTTQ5QkFNQ01DTXhJVEFmQmdOVkJBTU1HR3N6Y3kxelpYSjIKWlhJdFkyRkFNVFUyTkRNd05qWXlNekFlRncweE9UQTNNamd3T1RNM01ETmFGdzB5T1RBM01qVXdPVE0zTUROYQpNQ014SVRBZkJnTlZCQU1NR0dzemN5MXpaWEoyWlhJdFkyRkFNVFUyTkRNd05qWXlNekJaTUJNR0J5cUdTTTQ5CkFnRUdDQ3FHU000OUF3RUhBMElBQkpkQkZ2a0dtR1dmYXJhZndJOGVRWDdneWRGdWxYaGMyQU1NbHdLYm5XOXIKZ0k5NFdPbmN2OUdKQzNzTitiK3dtWDJOTEJNWXFLNGZWNjUvam1aZkZRK2pJekFoTUE0R0ExVWREd0VCL3dRRQpBd0lDcERBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUFvR0NDcUdTTTQ5QkFNQ0EwZ0FNRVVDSVFDZjlSaFJ4cXhHCnZVdGpsOHhIcDFyQ1MrbS96WW5HNnlnQVZ2ZzZpU2djVHdJZ1BUVzhGY0ZKUWdpdU9kN0c4N1I3Zi9HTmdZV24KQWhGdDlTVUZ5OEdHVmFFPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://10.200.100.183:7442
  name: t1.k3s
contexts:
- context:
    cluster: t1.k3s
    namespace: default
    user: koper
  name: koper@t1.k3s
current-context: koper@t1.k3s
kind: Config
preferences: {}
users:
- name: koper
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNCakNDQWEwQ0NRRDl4QVMrUVJZbERUQUtCZ2dxaGtqT1BRUURBakFqTVNFd0h3WURWUVFEREJock0zTXQKYzJWeWRtVnlMV05oUURFMU5qUXpNRFkyTWpNd0hoY05NVGt3TnpJNE1USXdNRE01V2hjTk1qa3dOekkxTVRJdwpNRE01V2pBcE1SVXdFd1lEVlFRRERBeHJiM0JsY2tCME1TNXJNM014RURBT0JnTlZCQW9NQjJ0bGVTMW5aVzR3CmdnRWlNQTBHQ1NxR1NJYjNEUUVCQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUUQ3UmdhMmpDUjBrRUlIS21jN1pNMGYKVUlBYnNaVURsTTZob3ROQkNQK29YWFFHeFBwYXNGaWgrTzhLc0JnZHFrTkFUL2s4bUkrTHM3SEhBelVkSnFFbgpKeFAvdFR0WG5LZjQvNno5OFlJa0JsK2k1OUdQS0VuNCtNK1RIYy92eGRnT21PVCtaZ05qWW4zMWMwN21VOVdtCkVhS0xlOHM2RlB5UzZiOCtDVTMrUWsyNHVPMlZ3dTA5SUJJTnc1L3o3V2daSHplM1RJS3Jkd0hKVHVZQThZNnoKL2VnVWRKb01JcFJkMGZxTDVCRSsxa0JqT2t2d2thVFhoY0lCMmV6bk5RLy82K2JsbUMwbjdBYm1tSTJJblRHQwprV25naEZjakE2d29lUWw4bUZwTml0b3R1aWMvaGhGRnI0LzNKbWZFSUpJaG80MEl2bWZ0ak1Sdk45aEo3ZUhUCkFnTUJBQUV3Q2dZSUtvWkl6ajBFQXdJRFJ3QXdSQUlnRnMreHFXYzNzVjg1dmVpSlhpTjJmaTNmRm9xQ3QyUm4KbngwNzlKNmhWRTBDSUhFU2M3dS8yeHlGZzhqc0VIWGwyMlhGTmdWUmx5bW5STW1wM0tLK1lUaDcKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcFFJQkFBS0NBUUVBKzBZR3Rvd2tkSkJDQnlwbk8yVE5IMUNBRzdHVkE1VE9vYUxUUVFqL3FGMTBCc1Q2CldyQllvZmp2Q3JBWUhhcERRRS81UEppUGk3T3h4d00xSFNhaEp5Y1QvN1U3VjV5bitQK3MvZkdDSkFaZm91ZlIKanloSitQalBreDNQNzhYWURwamsvbVlEWTJKOTlYTk81bFBWcGhHaWkzdkxPaFQ4a3VtL1BnbE4va0pOdUxqdApsY0x0UFNBU0RjT2Y4KzFvR1I4M3QweUNxM2NCeVU3bUFQR09zLzNvRkhTYURDS1VYZEg2aStRUlB0WkFZenBMCjhKR2sxNFhDQWRuczV6VVAvK3ZtNVpndEord0c1cGlOaUoweGdwRnA0SVJYSXdPc0tIa0pmSmhhVFlyYUxib24KUDRZUlJhK1A5eVpueENDU0lhT05DTDVuN1l6RWJ6ZllTZTNoMHdJREFRQUJBb0lCQVFEVUFrR254SmI5d3JuegpVZFBJU1VUSkp5THdPdVdBSUE0NFV5bnJ0YXdBWXRtQzNMQmYxR3IwUHhWeDd5SnA1VDdaQktGR2YzS2ViUCtTCjZ5SGxkcktDVm5hSlNtREhpMll1c1l0RXVJRVY1RXJOS011bi9sWnJ1NE5vbmI3VWtCbThOMFQvWVJONng1OS8KZWNzWWk2TzRleWlxaDhqeE9NUGpNVllyQWE3TTEzaTlkTi9OT1BSa2F0aG5rL2txeGNMaFYxK3diODVGNVNPbAp5dVJ5WU03OGtYakxHUGkxY3dKcHhScU9KcWNxMDAzQjFrN3U5VG9uSjdKOFR6MjJyc2tUdFRSN3o1eGR2VDRvCkFLSWVCNnpmMko0K0o5N3piN3lYUVFxV0ZrZk9EaTJSSlY2SXVZbFFQSkFUZVcwaVJ2RjR1dnJhTGtYZllnU1cKbmtRaGRFQUJBb0dCQVA1MzgyVTB6YUx4MVcxT2xhbWVod3dqV2JES3k4ZEN6VEVwR0ZmZGlhUFVOaUhqVXBOMgpGaExndS9LS05SWmNPZlVBbnhMSTNjWUg1UWQ1ZUlDZnh5UGtlek1RdzQ3TnRwbUgvN3VXenhtbXFodHBCdVN1ClVLQ1QwOGV0SlJEY3JET3lNMHJkQXM5U1htcW9mVGZqT2VwQnJSMnBCWEtkbi9iQWJGcTl1V1RIQW9HQkFQekoKSnl6UVFEN21IQ1BGeWZWelZLcVlCVG5adzdOMDNxamZHOWRXREZzSTBMWmJIKzhjcVVqRFJnTGhZYkkyTVJITwpTbzJUUFNsWXFBUndxenA1VENWSDVJUXljUE41eXpsWVlWMm5pWmtvVXBNYUlONHVhTCt5U1QraEUzSGRnam5rCm45TklVOFJHakkwN3AyZ0w2MDRaUFFMVTNZeno0M09ranpmcldmYVZBb0dCQUtEVWxVdjQ5S013NzdDM1ExWkMKTUo2V1ZTQ3MrK0NEc3dhSUw2K1JBR1pBUUxwb1g0OTl5ZlBDZ0dlSnZJWFdZbmNjSG00VDhEOHlUQ25PTjBBcwpQQVBPYTZOWnpBK2NxdlVjaEtBK2I4U0psdWZlR0pJK0xnMWZnVEdwbUV5dy9GRnNKb2tCYUw0NkZCeWJReEVvCmx6a2NxMXFjc2ltL3dCT0hpTFJOUnppUEFvR0JBUHJ6SXYzOUc5cVZqSmdDMGZUbThzV010NXR2MFRXRnIwb00KZTlJeHJZQnVadXl4MkNrRDFoYlRMTnpOTExURDBjRHdmOWkrdERnb3VGdjRFalN4bUdObVZMamNibjkzaU1XOApOS1RLSHZLNk1nZXhKN0lLZHBqZ0FKRzNjZHRYWU9IaVVyeG9rQ2hKTlYwOFBId3hZUDhlVlJCTGpFcFRFSm1NClkxWExRbnRsQW9HQVhGNU5ianJZNklCMmc2cDJvNkNjNVNmbnN2S2JrZm9XNDFQaUFlZkxPUjhmU2E5TkZ1UnEKcnNHT3FpWGdzYnBPTHQ0b1VqbmNmYXRJUXBnSG5tQ1QvZmZLUEdObEE4NUt5VVBhOGtpVDEzbXZjT28xT0JaRApQL0JELzcvNGY3allGK3RqUExYZXA0NzNvLzlSYVQyWGMzUXZZdTBlZExTQm5qb2szV3UrSTk4PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=

huapox · 2019-07-28T12:23:13Z

then i change the user credential to admin with username and password, this works fine:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJWekNCL3FBREFnRUNBZ0VBTUFvR0NDcUdTTTQ5QkFNQ01DTXhJVEFmQmdOVkJBTU1HR3N6Y3kxelpYSjIKWlhJdFkyRkFNVFUyTkRNd05qWXlNekFlRncweE9UQTNNamd3T1RNM01ETmFGdzB5T1RBM01qVXdPVE0zTUROYQpNQ014SVRBZkJnTlZCQU1NR0dzemN5MXpaWEoyWlhJdFkyRkFNVFUyTkRNd05qWXlNekJaTUJNR0J5cUdTTTQ5CkFnRUdDQ3FHU000OUF3RUhBMElBQkpkQkZ2a0dtR1dmYXJhZndJOGVRWDdneWRGdWxYaGMyQU1NbHdLYm5XOXIKZ0k5NFdPbmN2OUdKQzNzTitiK3dtWDJOTEJNWXFLNGZWNjUvam1aZkZRK2pJekFoTUE0R0ExVWREd0VCL3dRRQpBd0lDcERBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUFvR0NDcUdTTTQ5QkFNQ0EwZ0FNRVVDSVFDZjlSaFJ4cXhHCnZVdGpsOHhIcDFyQ1MrbS96WW5HNnlnQVZ2ZzZpU2djVHdJZ1BUVzhGY0ZKUWdpdU9kN0c4N1I3Zi9HTmdZV24KQWhGdDlTVUZ5OEdHVmFFPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://10.200.100.183:7442
  name: t1.k3s
contexts:
- context:
    cluster: t1.k3s
    namespace: default
    user: koper
  name: koper@t1.k3s
current-context: koper@t1.k3s
kind: Config
preferences: {}
users:
- name: koper
  user:
    password: 3f43d55928f3d7fc7277d28d519b4ec8
    username: admin

yamt · 2019-07-29T06:39:06Z

can you confirm that your ca.crt is consistent with apiserver's --client-ca-file?

erikwilson · 2019-07-31T21:41:54Z

We are using a separate server-ca and client-ca, it may be that the cluster certificate-authority-data needs to be the server-ca, while user client-certificate-data/key needs to be the generated cert/key signed by the client-ca cert/key.

huapox · 2019-08-02T00:13:08Z

We are using a separate server-ca and client-ca, it may be that the cluster certificate-authority-data needs to be the server-ca, while user client-certificate-data/key needs to be the generated cert/key signed by the client-ca cert/key.

thx. it works for me now:

[root@(⎈ |default:default) kube]$ kc --kubeconfig=koper.kubeconfig get pod -A
NAMESPACE     NAME                            READY   STATUS    RESTARTS   AGE
kube-system   coredns-b7464766c-g27sh         1/1     Running   2          9m57s
kube-system   rbac-manager-79bdb8757d-9p2f6   1/1     Running   0          100s

[root@(⎈ |default:default) sec-rbac]$ rbac-lookup k3s
SUBJECT         SCOPE          ROLE
koper@t1.k3s    cluster-wide   ClusterRole/cluster-admin

[root@(⎈ |default:default) kube]$ cat koper.kubeconfig 
apiVersion: v1
clusters:
- cluster:
    certificate-authority: /opt/sec-rbac/server/tls/server-ca.crt
    server: https://server:6443
  name: t1.k3s
contexts:
- context:
    cluster: t1.k3s
    namespace: default
    user: koper
  name: koper@t1.k3s
current-context: koper@t1.k3s
kind: Config
preferences: {}
users:
- name: koper
  user:
    client-certificate: /opt/sec-rbac/gen/keys/u-koper.crt
    client-key: /opt/sec-rbac/gen/keys/u-koper.key

huapox · 2019-08-02T00:14:52Z

The scripts and env current i use:

[root@(⎈ |default:default) sec-rbac]$ tree
.
├── gen
│   ├── keys
│   │   ├── u-koper.crt
│   │   ├── u-koper.csr
│   │   └── u-koper.key
│   └── kube
│       └── koper.kubeconfig
├── server
│   ├── cred
│   │   ├── admin.kubeconfig
│   │   ├── api-server.kubeconfig
│   │   ├── controller.kubeconfig
│   │   ├── node-passwd
│   │   ├── passwd
│   │   └── scheduler.kubeconfig
│   ├── db
│   │   ├── state.db
│   │   ├── state.db-shm
│   │   └── state.db-wal
│   ├── manifests
│   │   ├── coredns.yaml
│   │   └── rolebindings.yaml
│   ├── node-token
│   ├── static
│   │   └── charts
│   │       └── traefik-1.64.0.tgz
│   └── tls
│       ├── client-admin.crt
│       ├── client-admin.key
│       ├── client-auth-proxy.crt
│       ├── client-auth-proxy.key
│       ├── client-ca.crt
│       ├── client-ca.key
│       ├── client-ca.srl
│       ├── client-controller.crt
│       ├── client-controller.key
│       ├── client-kube-apiserver.crt
│       ├── client-kube-apiserver.key
│       ├── client-kube-proxy.crt
│       ├── client-kube-proxy.key
│       ├── client-kubelet.key
│       ├── client-scheduler.crt
│       ├── client-scheduler.key
│       ├── request-header-ca.crt
│       ├── request-header-ca.key
│       ├── server-ca.crt
│       ├── server-ca.key
│       ├── service.key
│       ├── serving-kube-apiserver.crt
│       ├── serving-kube-apiserver.key
│       ├── serving-kubelet.key
│       └── temporary-certs
│           ├── apiserver-loopback-client__.crt
│           └── apiserver-loopback-client__.key
└── t2.sh

11 directories, 44 files

[root@(⎈ |default:default) sec-rbac]$ cat t2.sh 
ws=/opt/sec-rbac
day=3650

clus_name="t1.k3s"
clus_ns="default"
user="koper"
#clus_url="https://10.200.100.183:7442"
clus_url="https://server:6443"  ##
ca_path=$ws/server/tls
rm -f $ca_path/*-ca.srl

ctx=gen && mkdir -p $ws/$ctx/{kube,keys} && cd $ws/$ctx
#############
ca1=client-ca
generate="keys/u-"$user
echo -e "\033[32m#>>GEN-KEY\033[0m"
#openssl genrsa -out $generate.key 2048
openssl ecparam -name prime256v1 -genkey -noout -out $generate.key
openssl req -new -key $generate.key -out $generate.csr -subj "/CN=${user}@${clus_name}/O=key-gen"
openssl x509 -req -in $generate.csr -CA $ca_path/$ca1.crt -CAkey $ca_path/$ca1.key -CAcreateserial -out $generate.crt -days $day

#-----------
#generate=$ca_path/client-admin  ##test
ca2=server-ca
embed=false
ctx2="$user@$clus_name"
config="kube/$user.kubeconfig"
echo -e "\033[32m#>>KUBE-CONFIG\033[0m"	
kubectl --kubeconfig=$config config set-cluster $clus_name --embed-certs=$embed --server=$clus_url --certificate-authority=$ca_path/$ca2.crt
kubectl --kubeconfig=$config config set-credentials $user --embed-certs=$embed --client-certificate=$generate.crt  --client-key=$generate.key
kubectl --kubeconfig=$config config set-context $ctx2 --cluster=$clus_name --namespace=$clus_ns --user=$user
kubectl --kubeconfig=$config config set current-context $ctx2
kubectl --kubeconfig=$config --context=$ctx2 get pods

ca1=client-ca
ca2=server-ca

huapox · 2019-08-02T00:20:47Z

can you confirm that your ca.crt is consistent with apiserver's --client-ca-file?

thx, resolved.

wenerme · 2020-07-12T06:13:18Z

Same problem, create a script to help create new client cert.

ooichman · 2020-09-26T09:28:06Z

The problem is that the CA that should sign the certificates for the user is not the same as the server uses for it's API
what needs to be done is copy 2 files from the server/tls directoy and use them to sign :

 # mkdir ~/tls && ~/tls
 # cp /var/lib/rancher/k3s/server/tls/client-ca.{crt,key} ~/tls

Generate the key + certificate request

 # openssl genrsa -out user.key 4096
 #  openssl req -new -key user.key -out user.csr -subj "/CN=user@default/O=admins"

sign the certificate
# openssl x509 -req -in user.csr -CA client-ca.crt -CAkey client-ca.key -CAcreateserial -out user.crt -days 3650
Now your user01.kubeconfig should look like this :

 apiVersion: v1
 clusters:     
 - cluster:
     certificate-authority: /root/server-ca.crt
     server: https://server:6443
   name: default
 contexts:
 - context:
     cluster: default
     namespace: ns01
     user: user
   name: user@default
 current-context: user@default
 kind: Config
 preferences: {}
 users:
 - name: user
   user:
     client-certificate: /root/tls/user.crt
     client-key: /root/tls/user.key

sraillard · 2020-10-05T20:33:35Z

Thank you for the explanation! Two questions:

The subject of the certificate should contains some specific values or I can be anything?
How you map the client with API server role?

erikwilson added the status/more-info label Jul 31, 2019

huapox closed this as completed Aug 2, 2019

huapox mentioned this issue Aug 3, 2019

[Question] Certificate authentication and cert issuing via Certificate API. Have these features been removed? #245

Closed

saksmt mentioned this issue May 6, 2020

CertificateSigningRequest / user creation #1764

Closed

sraillard mentioned this issue Oct 5, 2020

Certificate and access management for edge computing #2342

Closed

prometherion mentioned this issue May 12, 2023

"failed to verify certificate: x509: certificate signed by unknown authority" on k3s installation projectcapsule/capsule-proxy#291

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Help! k3s v0.7.0 generate userKey with ca, authenticate in failure #684

Help! k3s v0.7.0 generate userKey with ca, authenticate in failure #684

huapox commented Jul 28, 2019 •

edited

Loading

huapox commented Jul 28, 2019

huapox commented Jul 28, 2019

yamt commented Jul 29, 2019

erikwilson commented Jul 31, 2019

huapox commented Aug 2, 2019 •

edited

Loading

huapox commented Aug 2, 2019 •

edited

Loading

huapox commented Aug 2, 2019

wenerme commented Jul 12, 2020

ooichman commented Sep 26, 2020 •

edited

Loading

sraillard commented Oct 5, 2020

Help! k3s v0.7.0 generate userKey with ca, authenticate in failure #684

Help! k3s v0.7.0 generate userKey with ca, authenticate in failure #684

Comments

huapox commented Jul 28, 2019 • edited Loading

huapox commented Jul 28, 2019

huapox commented Jul 28, 2019

yamt commented Jul 29, 2019

erikwilson commented Jul 31, 2019

huapox commented Aug 2, 2019 • edited Loading

huapox commented Aug 2, 2019 • edited Loading

huapox commented Aug 2, 2019

wenerme commented Jul 12, 2020

ooichman commented Sep 26, 2020 • edited Loading

sraillard commented Oct 5, 2020

huapox commented Jul 28, 2019 •

edited

Loading

huapox commented Aug 2, 2019 •

edited

Loading

huapox commented Aug 2, 2019 •

edited

Loading

ooichman commented Sep 26, 2020 •

edited

Loading