- Create 4 node cluster "variants" on google kubernetes engine
gcloud container clusters create variants --machine-type n1-standard-1 --num-nodes 4
- Get credentials to point kubectl to our cluster
gcloud container clusters get-credentials kubepar
- In R, source runJob.R
- Point to the directory structure shown
- Create jobs for the .yaml files
kubectl create -f job1.yaml or kubectl create -f . (to create jobs for all the .yaml file present in the directory)
General procedure to create, run, monitor jobs on Kubernetes
- Create a Docker image with the required Rscript file.
- Create a job.yaml file that contains instructions for the controller.
-
Install Google Cloud SDK (https://cloud.google.com/sdk/)
-
Install kubectl (https://kubernetes.io/docs/tasks/tools/install-kubectl/)
-
Follow these steps (change cluster name and job file as required)
-
Create 4 node cluster "kubepar" on google kubernetes engine
gcloud container clusters create kubepar --machine-type n1-standard-1 --num-nodes 4
- Get credentials to point kubectl to our cluster
gcloud container clusters get-credentials kubepar
- Create job
kubectl create -f job.yaml
kubectl describe jobs
kubectl describe pods
kubectl get nodes
kubectl delete job "jobname"
kubectl logs "podname" (can be got from "Events" when we run kubectl describe jobs)
Kubernetes clusters can be deleted on the Kubernetes Engine Google Cloud console.