Cybozu Kubernetes Engine

CKE (Cybozu Kubernetes Engine) is a distributed service that automates Kubernetes cluster management.

Project Status: GA

Requirements

CKE requirements

• etcd
• Vault

Node OS Requirements

• Docker: etcd data is stored in Docker volumes.
• A user who belongs to docker group
• SSH access for the user

Features

• Bootstrapping and life-cycle management.

  CKE can bootstrap a Kubernetes and etcd cluster from scratch. CKE can also add or remove nodes to/from the Kubernetes and etcd cluster.

• In-place and fast upgrade of Kubernetes

  A version of CKE corresponds strictly to a single version of Kubernetes. Therefore, upgrading CKE will upgrade the managed Kubernetes.

  Unlike kubeadm or similar tools, CKE can automatically upgrade its managed Kubernetes without draining nodes. The time taken for the upgrade is not proportional to the number of nodes, so it is very fast.

• Graceful rebooting of nodes

  CKE can reboot specified nodes gracefully using the Kubernetes eviction API.

• Managed etcd cluster

  CKE manages an etcd cluster for Kubernetes. Other applications may also store their data in the same etcd cluster.

  Details are described in docs/etcd.md.

• CRI runtimes

  In addition to Docker, CRI runtimes such as containerd or cri-o can be used to run Kubernetes Pods.

• Certificate for admission webhooks

  Admission webhooks are Kubernetes extension to validate or mutate API resources. Installing them requires some sort of self-signed X509 certificates.

  CKE can become a certificate authority (CA) and issue certificates for these webhooks.

• Kubernetes features:

  • HA control plane.
  • RBAC is enabled.
  • Ready for PodSecurityPolicy
  • Ready for API aggregation.
  • Secret data are encrypted at rest.
  • CNI network plugins.
  • CoreDNS add-on.
  • Node-local DNS cache services.
  • Nodes can be registered with Taints.
  • Preparation of Scheduler extender.

• User-defined resources:

  CKE automatically creates or updates Kubernetes API resources such as Deployments, Namespaces, or CronJobs that are defined by users. This feature helps users to automate Kubernetes cluster maintenance.

• Sabakan integration

  CKE can be integrated with sabakan, a service that automates physical server management, to generate cluster configuration automatically.

  Sabakan is not a requirement; cluster configuration can be supplied externally by a YAML file.

• High availability

  CKE stores its configurations in etcd to share them among multiple instances. Etcd is also used to elect a leader instance that exclusively controls the Kubernetes cluster.

• Operation logs

  To track problems and life-cycle events, CKE keeps operation logs in etcd.

Programs

This repository contains these programs:

• cke: the service.
• ckecli: CLI tool for cke.

To see their usage, run them with -h option.

Getting started

A demonstration of CKE running on docker is available at example directory.

Documentation

docs directory contains tutorials and specifications.

Usage

Run CKE with docker

$ docker run -d --read-only \
    --network host --name cke \
    quay.io/cybozu/cke:1.18 [options...]

Install ckecli to host file system

$ docker run --rm -u root:root \
    --entrypoint /usr/local/cke/install-tools \
    --mount type=bind,src=DIR,target=/host \
    quay.io/cybozu/cke:1.18

Docker images

Docker images are available on Quay.io

Feedback

Please report bugs / issues to GitHub issues.

Feel free to send your pull requests!

License

CKE is licensed under MIT license.