rke up --custom-certs not working

[Aisuko]

RKE version:

INFO[0000] Running RKE version: v1.1.2

Docker version: (docker version,docker info preferred)

Operating system and kernel: (cat /etc/os-release, uname -r preferred)

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)

cluster.yml file:

nodes:
  - address: 10.57.4.18
    port: "22"
    internal_address: ""
    role:
      - controlplane
      - etcd
    hostname_override: master
    user: rancher
    docker_socket: /var/run/docker.sock
    ssh_key: ""
    ssh_key_path: ~/.ssh/id_rsa
    ssh_cert: ""
    ssh_cert_path: ""
    labels: 
    taints:

  - address: 10.57.4.19
    port: "22"
    internal_address: ""
    role:
      - worker
    hostname_override: worker1
    user: rancher
    docker_socket: /var/run/docker.sock
    ssh_key: ""
    ssh_key_path: ~/.ssh/id_rsa
    ssh_cert: ""
    ssh_cert_path: ""
    labels: 
    taints:

  - address: 10.57.4.20
    port: "22"
    internal_address: ""
    role:
      - worker
    hostname_override: worker2
    user: rancher
    docker_socket: /var/run/docker.sock
    ssh_key: ""
    ssh_key_path: ~/.ssh/id_rsa
    ssh_cert: ""
    ssh_cert_path: ""
    labels: 
    taints:

  - address: 10.57.4.21
    port: "22"
    internal_address: ""
    role:
      - worker
    hostname_override: worker3
    user: rancher
    docker_socket: /var/run/docker.sock
    ssh_key: ""
    ssh_key_path: ~/.ssh/id_rsa
    ssh_cert: ""
    ssh_cert_path: ""
    labels: 
    taints:

services:
  etcd:
    image: ""
    extra_args: 
    extra_binds: 
    extra_env: 
    external_urls: 
    ca_cert: ""
    cert: ""
    key: ""
    path: ""
    uid: 0
    gid: 0
    snapshot: 
    retention: ""
    creation: ""
    backup_config: 
      enabled: true
      interval_hours: 24
      retention: 3
  kube-api:
    image: ""
    extra_args: 
    extra_binds: 
    extra_env: 
    service_cluster_ip_range: 10.43.0.0/16
    service_node_port_range: ""
    pod_security_policy: false
    always_pull_images: false
    secrets_encryption_config: 
    audit_log: 
    admission_configuration: 
    event_rate_limit: 
  kube-controller:
    image: ""
    extra_args:
    extra_binds: 
    extra_env: 
    cluster_cidr: 10.42.0.0/16
    service_cluster_ip_range: 10.43.0.0/16
  scheduler:
    image: ""
    extra_args: 
    extra_binds: 
    extra_env: 
  kubelet:
    image: ""
    extra_args: 
    extra_binds: 
    extra_env: 
    cluster_domain: rancher.local
    infra_container_image: ""
    cluster_dns_server: 10.43.0.10
    fail_swap_on: false
    generate_serving_certificate: false
  kubeproxy:
    image: ""
    extra_args:
      proxy-mode: "ipvs" 
    extra_binds: 
    extra_env: 
network:
  plugin: canal
  options:
    flannel_backend_type: "vxlan"
  mtu: 1450
  node_selector: 
  update_strategy: 
authentication:
  strategy: x509
  sans: 
  webhook: 
addons: ""
addons_include: 
system_images:
  etcd: rancher/coreos-etcd:v3.4.3-rancher1
  alpine: rancher/rke-tools:v0.1.56
  nginx_proxy: rancher/rke-tools:v0.1.56
  cert_downloader: rancher/rke-tools:v0.1.56
  kubernetes_services_sidecar: rancher/rke-tools:v0.1.56
  kubedns: rancher/k8s-dns-kube-dns:1.15.0
  dnsmasq: rancher/k8s-dns-dnsmasq-nanny:1.15.0
  kubedns_sidecar: rancher/k8s-dns-sidecar:1.15.0
  kubedns_autoscaler: rancher/cluster-proportional-autoscaler:1.7.1
  coredns: rancher/coredns-coredns:1.6.5
  coredns_autoscaler: rancher/cluster-proportional-autoscaler:1.7.1
  nodelocal: rancher/k8s-dns-node-cache:1.15.7
  kubernetes: rancher/hyperkube:v1.17.6-rancher2
  flannel: rancher/coreos-flannel:v0.11.0-rancher1
  flannel_cni: rancher/flannel-cni:v0.3.0-rancher6
  calico_node: rancher/calico-node:v3.13.4
  calico_cni: rancher/calico-cni:v3.13.4
  calico_controllers: rancher/calico-kube-controllers:v3.13.4
  calico_ctl: rancher/calico-ctl:v3.13.4
  calico_flexvol: rancher/calico-pod2daemon-flexvol:v3.13.4
  canal_node: rancher/calico-node:v3.13.4
  canal_cni: rancher/calico-cni:v3.13.4
  canal_flannel: rancher/coreos-flannel:v0.11.0
  canal_flexvol: rancher/calico-pod2daemon-flexvol:v3.13.4
  weave_node: weaveworks/weave-kube:2.6.4
  weave_cni: weaveworks/weave-npc:2.6.4
  pod_infra_container: rancher/pause:3.1
  ingress: rancher/nginx-ingress-controller:nginx-0.32.0-rancher1
  ingress_backend: rancher/nginx-ingress-controller-defaultbackend:1.5-rancher1
  metrics_server: rancher/metrics-server:v0.3.6
  windows_pod_infra_container: rancher/kubelet-pause:v0.1.3
ssh_key_path: ~/.ssh/id_rsa
ssh_cert_path: ""
ssh_agent_auth: false
authorization:
  mode: rbac
  options: 
ignore_docker_version: false
kubernetes_version: ""
private_registries: 
ingress:
  provider: "nginx"
  options: 
    use-forwarded-headers: "true"
  node_selector: 
  extra_args: 
  dns_policy: ""
  extra_envs: 
  extra_volumes: 
  extra_volume_mounts: 
  update_strategy: 
cluster_name: ""
cloud_provider:
  name: ""
prefix_path: ""
addon_job_timeout: 0
bastion_host:
  address: ""
  port: ""
  user: ""
  ssh_key: ""
  ssh_key_path: ""
  ssh_cert: ""
  ssh_cert_path: ""
monitoring:
  provider: ""
  options: 
  node_selector: 
  update_strategy: 
  replicas: 
restore:
  restore: false
  snapshot_name: ""
dns: 
  provider: coredns

Steps to Reproduce:

Results:

➜  kubesphere git:(master) ✗ ls
ceph  cluster_certs  cluster.yml  kubesphere-complete-setup.yaml  pem.sh  rke  tiller


➜  kubesphere git:(master) ✗ ls cluster_certs 
kube-admin-csr.pem      kube-apiserver-proxy-client-csr.pem      kube-ca-key.pem                  kube-etcd-10-57-4-18-csr.pem  kube-proxy-csr.pem      kube-service-account-token-key.pem
kube-admin-key.pem      kube-apiserver-proxy-client-key.pem      kube-ca.pem                      kube-etcd-10-57-4-18-key.pem  kube-proxy-key.pem      kube-service-account-token.pem
kube-apiserver-csr.pem  kube-apiserver-requestheader-ca-key.pem  kube-controller-manager-csr.pem  kube-node-csr.pem             kube-scheduler-csr.pem
kube-apiserver-key.pem  kube-apiserver-requestheader-ca.pem      kube-controller-manager-key.pem  kube-node-key.pem             kube-scheduler-key.pem


➜  kubesphere git:(master) ✗ ./rke up --custom-certs /root/code/infrastructure-configurations/RKE/kubesphere/cluster_certs 
INFO[0000] Running RKE version: v1.1.2                  
INFO[0000] Initiating Kubernetes cluster                
INFO[0000] [dialer] Setup tunnel for host [10.57.4.21]  
INFO[0000] [dialer] Setup tunnel for host [10.57.4.19]  
INFO[0000] [dialer] Setup tunnel for host [10.57.4.20]  
INFO[0000] [dialer] Setup tunnel for host [10.57.4.18]  
INFO[0000] Checking if container [cluster-state-deployer] is running on host [10.57.4.18], try #1  
INFO[0000] Checking if container [cluster-state-deployer] is running on host [10.57.4.19], try #1  
INFO[0000] Checking if container [cluster-state-deployer] is running on host [10.57.4.20], try #1  
INFO[0000] Checking if container [cluster-state-deployer] is running on host [10.57.4.21], try rancher/rke#1 
FATA[0000] Failed to validates certificates from dir [./cluster_certs]: Failed to find master CA certificate 

[dje4om]

Hi,

It seems you did not signed your certificates and you also need to provide the CA used to sign them.
In documentation : https://rancher.com/docs/rke/latest/en/installation/certs/

You can use them to sign the certificates by a real CA. After the certificates are signed, those certificates can be used by RKE as custom certificates.

[Aisuko]

Hi,

It seems you did not signed your certificates and you also need to provide the CA used to sign them.
In documentation : https://rancher.com/docs/rke/latest/en/installation/certs/

You can use them to sign the certificates by a real CA. After the certificates are signed, those certificates can be used by RKE as custom certificates.

Thanks for your comment but it still not working well for me even though I use the certificates which create by rke.

[superseb]

Please share the steps/commands you used to sign the certificates.

[pengmingming]

@superseb
Hi, My operation is as follows.

1.rke cert generate-csr
2.openssl genrsa -out kube-ca-key.pem 2048
3.openssl req -x509 -new -nodes -key kube-ca-key.pem -days 10000 -out kube-ca.pem -subj "/CN=kube-ca"
4.openssl x509 -req -days 1825 -sha256 -CA ./cluster_certs/kube-ca.pem -CAkey ./cluster_certs/kube-ca-key.pem -CAcreateserial -in ./cluster_certs/kube-apiserver-csr.pem -out ./cluster_certs/kube-apiserver.pem

5.rke up --custom-certs

kube-ca.zip

[superseb]

@pengmingming This is a different error and doesn't seem related to this issue. Can you open a new issue with the full log and the output of docker ps -a and docker logs kube-apiserver from node 172.16.4.145?

[stale]

This issue/PR has been automatically marked as stale because it has not had activity (commit/comment/label) for 60 days. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.