Tool to deploy a kubernetes cluster on openSUSE Kubic using kubeadm and salt
kubic-control consists of two binaries:
- kubicd, a daemon which communicates via gRPC with clients. It's setting up kubernetes on openSUSE Kubic, including pod network, kured, transactional-update, ...
- kubicctl, a cli interface
The communication is encrypted, the kubicctl command can run on any machine. The user authenticates with his certificate, using RBAC to determine if the user is allowed to call this function. kubiccd will use kubeadm and kubectl to deploy and manage the cluster. So the admin can at everytime modify the cluster with this commands, too, there is no hidden state-database.
kubicctl are using salt and certstrap to manage nodes and certificates, additional kubeadm, kubectl, kubelet and crio have to be installed.
Kubicd has to run on the future kubernetes master node,
kubicctl can run anywhere on the network. This requires only that
kubicd is configured to listen on all interfaces, not only
localhost. The salt minions have to be already accepted on the salt master. Before
kubicd can be started, certificates have to be generated:
kubicctl certificates initialize
This will create a CA and several certificates in
- Kubic-Control-CA.key - the private CA key
- Kubic-Control-CA.crt - the public CA key. This one is needed by kubicctl, too
- KubicD.key - the private key for kubicd
- kubicD.crt - the signed public key for kubicd
- admin.key - private key, allows kubicctl to connect to kubicd as admin
- admin.crt - public key, allows kubicctl to connect to kubcd as admin
kubicctl, you need to create a directory
user.crt. For the admin
role, this need to be a copy of admin.key and admin.crt. For other users,
you need to create corresponding certificates and sign them with
Kubic-Control-CA.crt. If you call
kubicctl as root and there is no
~/.config/kubicctl, the admin certificates from
/etc/kubicd/pki are used if they exist.
Certificates for additional users can be created with
kubicctl certificates create <account>.
Please take care of this certificates and store them secure, this are the passwords to access kubicd!
To deploy the control-plane on the master with flannel as POD network and
kured to manage the reboot of nodes:
For cilium instead of flannel you have to use
kubicctl init --pod-network cilium.
To add additional nodes:
kubicctl node add node1,...
In the same way, you can remove nodes:
kubicctl node remove or reboot
kubicctl node reboot.
To access with cluster with
kubectl, you can get the kubeconfig with:
The kubernetes cluster can be upgraded with:
Kubicd reads two configuration files:
first one is optional and contains the paths to the certificates and the server
name with port
kubicd should listen to. The default file can be found in
/usr/share/defaults/kubicd/kubicd.conf. The variables can be overriden with
/etc/kubicd/conf, which only contains the changed entries.
The second file,
rbac.conf, is mandatory, else nobody can access
all requests will be rejected. The default file can be found in
/usr/share/defaults/kubicd/rbac.conf. Changed entries should be written
rbac.conf contains the roles as key and the users, who are allowed to use
this functionality as comma seperated list.
kubicctl rbac list will print
out a list of current configured roles and the corresponding users.
kubicctl rbac add <role> <user> will add the user to the role.
- certificates - Manage certificates for kubicd/kubicctl communication
- create - Create certificate for an user. The certificate will be stored in the local directory where you did call kubicctl.
- initialize - Create CA, KubicD and admin certificates. This certificates will be stored in
- help - Help about any command
- init - Initialize Kubernetes Master Node
- --pod-network=<flannel|cilium> Pod network
- --adv-addr= IP address the API Server will advertise on
- kubeconfig - Download kubeconfig
- --output= - Where the kubeconfig file should be stored
- node - Manage kubernetes nodes
- add ,... - Add new nodes to cluster. Node names must be the name used by salt for that node. A comma seperated list or '' syntax are allowed to specify more than one new node.
- list - List all reacheable worker nodes
- reboot - Reboot node from cluster. Node will be drained first. Node name must be the name used by salt for that node.
- remove - Remove node form cluster
- rbac - Manage RBAC rules
- add - Add user account to a role
- list - List roles and accounts
- upgrade - Upgrade Kubernetes Cluster to the version of the installed kubelet command
- destroy-cluster - Remove all worker and master nodes
- version - Print version information
Kubicd does not store any informations about the state of the kubernetes
cluster. This allows to manage the cluster with
kubicctl. There is only one important thing: a grain
has to be set on new worker nodes:
kubicd=kubic-worker-node. If nodes
gets manual removed, this grain has to be deleted, too.