hgvarByKub

RKubernetes

To submit a variant annotation job to Kubernetes

• Create 4 node cluster "variants" on google kubernetes engine

gcloud container clusters create variants --machine-type n1-standard-1 --num-nodes 4

• Get credentials to point kubectl to our cluster

gcloud container clusters get-credentials kubepar

• In R, source runJob.R
• Point to the directory structure shown
• Create jobs for the .yaml files

kubectl create -f job1.yaml or kubectl create -f . (to create jobs for all the .yaml file present in the directory)

General procedure to create, run, monitor jobs on Kubernetes

Docker:

• Create a Docker image with the required Rscript file.

Kubernetes YAML:

• Create a job.yaml file that contains instructions for the controller.

Deploy job to kubernetes:

• Install Google Cloud SDK (https://cloud.google.com/sdk/)

• Install kubectl (https://kubernetes.io/docs/tasks/tools/install-kubectl/)

• Follow these steps (change cluster name and job file as required)

• Create 4 node cluster "kubepar" on google kubernetes engine

gcloud container clusters create kubepar --machine-type n1-standard-1 --num-nodes 4

• Get credentials to point kubectl to our cluster

gcloud container clusters get-credentials kubepar

• Create job

kubectl create -f job.yaml

Monitor pods, jobs, nods with kubectl

Cheatsheet for some kubectl commands

kubectl describe jobs

kubectl describe pods

kubectl get nodes

kubectl delete job "jobname"

kubectl logs "podname" (can be got from "Events" when we run kubectl describe jobs)

Kubernetes clusters can be deleted on the Kubernetes Engine Google Cloud console.