Skip to content

Latest commit

 

History

History
102 lines (60 loc) · 6.45 KB

deploy.md

File metadata and controls

102 lines (60 loc) · 6.45 KB
Raw

Deploy Coredump Detector via Script

deploy.sh is a shell script which could deploy a test coredump detector in k8s cluster.

How to use

To deploy all the components in namespace default with service name being coredump, user could do like this:

sh deploy.sh default coredump /tmp/coredump-yaml
kubectl create -f /tmp/coredump-yaml/coredump.yaml

Then you could observe something like this:

$ kubectl get pod -n default
NAME                       READY   STATUS    RESTARTS   AGE
coredump-fcf796fd9-768c9   2/2     Running   0          3m31s
etcd-0                     1/1     Running   0          3m31s
kcdt-ds-gxqs5              1/1     Running   0          3m31s

Structure

The yaml file generated in last section contains following parts:

  • Service Account

    We defined two service accconts in the yaml file, coredump-api used by coredump api-server component, coredump-gc used by coredump garbage collection compoent.

  • APIService

    APIService is to register apis to core k8s server as well as which service should recieve corresponding requests. Any requests against this self-defined api, which in our case is coredump.fujitsu.com, will be proxied to that service.

  • Service

    We defined two services in yaml file, one is to handle api request, another one is etcd-svc which let coredump apiserver use etcd storage. Something should be noted:

    • The name of the service to handle api request should be decided by admins, the certificates used by Secret is related to it. So if you change the name of that service, you should also regenerate the certificates.
    • The etcd storage which coredump apiserver uses is NOT the one used by core apiserver, it exists as a statefulset inside k8s cluster.

  • WorkLoads

    • Etcd StatefulSet

      This workload is responsible for storing CoredumpEndpoint Object data. Coredump api-server will access it via etcd-svc service. It should be noted that etcd statefulset needs storage to store data, in our yaml file, we just use /tmp folder being that storage. But in production environment, user should provide other stable, reliable storage instead of /tmp foleder. Some persistent volume is a good choice.

    • Apiserver Deployment

      This workload handles all the user requests like downloading core files. It is a bridge between core file storage and users. When user launches a request to coredump.fujitsu.com api, the request will proxied to apiserver deployment. The proxy is set by APIService and user-named service.

    • Garbage Collection Deployment

      Garbage Collection Deploymentis simply do clean job in backend storge which used to store core files.

    • Kcdt DaemonSet

      Kcdt DaemonSet is responsible for managing coredump behaviors on node. It makes sure core files will be generated and stored correctly as we expected when jobs inside Pods crashed.

  • Secret

    Servercrt secret stores https certificates, deploy.sh can generated them automatically. But of course, you could generate them by hand, then replace __SERVICE_TLS_CRT__ && __SERVICE_TLS_KEY__ in yaml file.

  • RBAC Related

    RBAC is a mechanism to manage users' authorities. There are three pairs in our yaml file, one for Garbage Collection Deployment, two for Apiserver Deployment, this is the least authorities which enable it work properly. You should not modify it.

Following picture shows the main flow when a user launch a request to coredump.fujitsu.com api

For Production Use

The yaml file generated by deploy.sh is only for test purpose, to deploy in production enviornment, you should do some modifications.

  • Image

    There are four images defined in yaml file, feel free to complie and built your own images to replace them.

    WorkLoads Image Function Source
    coredump-api deployment fenggw/fujitsu-coredump:v1.0-alpha handle user requests complied and built from this repository
    coredump-gc deployment fenggw/fujitsu-coredump-gc:v1.0-alpha do garbage job against core files complied and built from this repository
    kcdt-ds daemonset fenggw/kcdt:v1.0 collect core files from node complied and built from kcdt repository
    etcd deployment quay.io/coreos/etcd:latest store CoredumpEndpoints Object from official etcd image source

  • Storage

    Storage is a big problem under production environment. It will be used for storing CoredumpEndpoint Object as well as core files. The deploy.sh simply use /tmp folder to store this data. Obviously it is unacceptable for production environment. For etcd storage, you could use any forms of persistent-volumes. Replace etcd-data-dir volume in generated yaml file with your own stable, reliable storage.

    When it comes to core file storage, coredump-detector only supports filesystem storage currently. Something like NFS, Ceph, GlusterFS are all good choice. Replace all the coredump-backend volumes in generated yaml file with your own volume.

  • Certificates

    To secure the communication between k8s core apiserver and coredump apiserver, it's necessary to use https. Our script could generate https certificates automatically. Of course you could use your own certificates, and replace __SERVICE_TLS_CRT__ && __SERVICE_TLS_KEY__. For detailed information, please see k8s official documents.