List of resources and notes for passing the Certified Kubernetes Application Developer (CKAD) exam. Official links below.
I'm giving a 90 minute presentation on passing the CKAD and CKA in Kansas City. See link below. Once complete, I'll add my slides and other material here.
- Passing the CKAD and CKA Exams
- CNCF Official CKAD Main
- CNCF Kubernetes Curriculum Repo
- CNCF Official CKAD Exam Tips
- CNCF Official CKAD Candidate Handbook
- VIM Cheatsheet - You should know VIM pretty well
- Excellent CKAD Exercises to complement this guide
- TMUX Cheat Sheet - TMUX is useful, especially for CKA
Current Kubernetes Version
Important vi Tips
- 'dG' - Deletes contents from cursor to end of file. This is very useful when editing YAML files.
- 'ZZ' - Save and exit quickly.
To set nano as your editor for 'kubectl edit'
Right now there are five primary sections to this document.
- General Exam Tips
- A Checklist of Curriculum Progress
- Where to Practice?
- Detailed Review (with Tips) Ordered by Curriculum
- List of Resources Ordered by Curriculum (mostly K8s.io) for Study
Okay, this section is new and contains some general pointers to help pass the exam.
First, as discussed later, the exam is primarily about speed. With that in mind, the best way to approach the moderate to complex questions is to generate the initial YAML via the dry run flag. Then, edit the file with either vi or nano, and then create the required resource. The steps are outlined below.
$ kubectl run nginx --image=nginx --restart=Never --dry-run -o yaml > mypod.yaml $ nano mypod.yaml $ kubectl create -f mypod.yaml pod "nginx" created
There you go. If you're not satisfied with the results. Delete the resource, re-edit the declaritive yaml file, and redo.
$ kubectl delete -f mypod.yaml pod "nginx" deleted $ nano mypod.yaml $ kubectl create -f mypod.yaml pod "nginx" created
The exam is 100% hands on using the innovative exams (www.examslocal.com) product. The CKAD exam requires an excellent understanding of K8s along with how to efficiently use kubectl to accomplish various tasks on Kubernetes. I'm sure they use this exam approach as it pretty much precludes any form of cheating. You either know the material and can very quickly implement it or not.
You will be given a list of 'tasks' to accomplish on one of four kubernetes clusters (these are described in the official exam tips above). The exam is 'open book' but only with the content available at kubernetes.io. You will have one tab for the exam content and one additional tab for kubernetes.io. However, don't expect that you can just research questions during the exam, as there will be very little time for 'learning' a specific k8s concept at exam time. It's there to help with YAML syntax detail only, IMO.
The items in this particular repo / page describe and follow the official curriculum and point back to the various documents at Kubernetes.io. There is a lot of content on k8s and a lot of it does not pertain to the CKAD exam, so I've pulled out the sections that are pertinent based on the curriculum. There is a nice checklist below that you can update once you think you have mastered a particular topic.
I think the best approach is to fork this repo as a starting point for your studies, and then use the markdown checklist to ensure you cover all of the expected material, etc.
The list below is based on the curriculum v1.0. Once you have mastered a section, check it off and move on to the next. You need to understand them ALL very well. The Core Concepts piece is kind of vague, but the others are defined well enough that it is easy to prepare for with a hands-on work through the tasks offered at kubernetes.io. The rest of this document follows this same outline of curriculum.
- Core Concepts - 13%
- API Primitives
- Create and Configure Basic Pods
- Configuration - 18%
- Understand ConfigMaps
- Understand SecurityContexts
- Define App Resource Requirements
- Create and Consume Secrets
- Understand Service Accounts
- Multi-Container Pods - 10%
- Design Patterns: Ambassador, Adapter, Sidecar
- - Sidecar Pattern
- - Init Containers
- Design Patterns: Ambassador, Adapter, Sidecar
- Pod Design - 20%
- Using Labels, Selectors, and Annotations
- Understand Deployments and Rolling Updates
- Understand Deployment Rollbacks
- Understand Jobs and CronJobs
- - State Persistence - 8%
- - Understand PVCs for Storage
- Observability - 18%
- Liveness and Readiness Probes
- Understand Container Logging
- Understand Monitoring Application in Kubernetes
- Understand Debugging in Kubernetes
- Services and Networking - 13%
- Understand Services
- Basic Network Policies
Where to Practice
This particular items was difficult for me as I didn't have a (current) k8s cluster to use at work. As I was initially studying for the CKA which requires more cluster-level work, I tried many, many different approaches for an inexpensive k8s environment. built many clusters using K8s The Hard way on gcloud (and AWS), built a raspberry pi cluster I could carry to work, and tried using kubeadm / kops on gcloud and aws.
In my opinion, and all that is required to pass this test, is to just setup a gcloud account, and use a two-node GKE cluster for studying. Heck, you can even use the very nice google cloud shell and not even leave your browser.
Here are commands used to create a two-node cluster for studying. I keep these here just so I can fire up and destroy a cluster for a few hours each day for study. Notice that you can tailor the cluster version to match the k8s version for the exam.
gcloud config set compute/zone us-central1-a gcloud config set compute/region us-central1 gcloud container clusters create my-cluster --cluster-version=1.10.7-gke.6 \ --image-type=ubuntu --num-nodes=2
NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS my-cluster us-central1-a 1.10.7-gke.6 220.127.116.11 n1-standard-1 1.10.7-gke.6 2 RUNNING cloudshell:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION gke-my-cluster-default-pool-5f731fab-9d6n Ready <none> 44s 1.10.7-gke.6 gke-my-cluster-default-pool-5f731fab-llrb Ready <none> 41s 1.10.7-gke.6
Setting kubectl Credentials
If using the cloud shell, you'll sometimes need to authorize kubectl to connect to your cluster instance.
gcloud container clusters get-credentials my-cluster
Deleting Your Cluster
No need to keep the cluster around when not studying, so:
gcloud container clusters delete my-cluster
To Get Current GKE Kubernetes Versions
gcloud container get-server-config
The exam is about speed and efficiency. If you spend very much time looking at documentation, you will have zero chance of completing the many questions. With that said, the following will help with time management. I've aligned the tips to follow the curriculum. This section is best used to provide a quick overview of the curriculum along with the needed kubectl commands for a hands-on exam.
The core concepts section covers the core K8s API and its primitives and resources. It also covers the important concept of a POD. This is the basic unit of deployment for app developers and so this 'POD' concept is important to understand as well as how they are managed with kubectl. To me, this is embodied in the kubectl RUN command.
Using the RUN command for Pods, Deployments, etc.
The RUN command allows quick creation of the various high-level execution resources in k8s, and provides speed, which we need for the exam.
The specific, underlying resource created from a particular RUN command is based on its 'generator'. For example, by default RUN creates a deployment. However, it can also create a POD, or JOB, or CRONJOB, based on various flags, in particular the ---restart flag. This is handy.
$ kubectl run nginx --image=nginx (deployment) $ kubectl run nginx --image=nginx --restart=Never (pod) $ kubectl run busybox --image=busybox --restart=OnFailure (job) $ kubectl run busybox --image=busybox --schedule="* * * * *" --restart=OnFailure (cronJob)
The --schedule flag creates a Cron Job, and --restart=OnFailure creates a Job resource.
The above is helpful in the exam as speed is important. If the question indicates to 'create a pod', use the quick syntax to get a pod going.
$ kubectl run nginx --image=nginx --restart=Never pod "nginx" created
If we leave off the --restart flag, we get a deployment with replica set and all.
$ kubectl run nginx --image=nginx deployment.apps "nginx" created $ kubectl get deployment nginx NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx 1 1 1 1 8s
Oh, you just want a job to run, change the restart flag:
$ kubectl run busybox --image=busybox --restart=OnFailure job.batch "busybox" created $ kubectl get jobs NAME DESIRED SUCCESSFUL AGE busybox 1 1 9s
And finally, we want a CRON-based job, add the schedule flag:
$ kubectl run busybox --image=busybox --schedule="* * * * *" --restart=OnFailure cronjob.batch "busybox" created $ kubectl get cronjobs NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE busybox * * * * * False 0 <none> 20s
The point here is that one or two flags will create a different resource type, and quickly. Very useful when speed is necessary.
Configuration covers how run-time 'config' data is provided to your applications running in k8s. This includes environment variables, config maps, secrets, etc. Other items that are pertinent to config are the service account and security contexts used to execute the containers. The below items are covered by this part of the curriculum.
Config Maps / Environment Variables
The exam is about application development and its support within Kubernetes. With that said, high on the list of objectives is setting up config options and secrets for your applications. To create the most basic config map with a key value pair, see below.
$ kubectl create configmap app-config --from-literal=key123=value123 configmap "app-config" created
There are many ways to map config map items to environment variables within a container process. One quick, but tricky (syntax) option is shown below. This would be for a simple nginx container.
spec: containers: - image: nginx name: nginx envFrom: - configMapRef: name: app-config
Here is another way to map a specific value to a specific environment variable value.
containers: - image: nginx name: nginx env: - name: SPECIAL_APP_KEY valueFrom: configMapKeyRef: name: app-config key: key123
Now to verify it worked.
$ kubectl exec -it nginx /bin/bash root@nginx:/# env HOSTNAME=nginx SPECIAL_APP_KEY=value123 KUBERNETES_PORT_443_TCP_PROTO=tcp ...
Security contexts can be applied at either the pod or container level. Of course pod-level contexts apply to all containers within that pod. There are several ways of defining privileges and access controls with these contexts.
These concepts are covered well in the tasks section below, but here is a basic RunAs example from the doc that shows both pod and container contexts being used.
apiVersion: v1 kind: Pod spec: securityContext: runAsUser: 1000 containers: - name: sec-ctx-demo image: gcr.io/google-samples/node-hello:1.0 securityContext: runAsUser: 2000 allowPrivilegeEscalation: false
App Resource Requirements
Defining the memory and cpu requirements for your containers is something that should always be done. It allows for more efficient scheduling and better overall hygiene for your application environment. Again, covered well in the tasks section below, but here is a brief snippet for the standard mem/cpu specification.
apiVersion: v1 kind: Pod spec: containers: - name: demo image: polinux/stress resources: limits: memory: 200Mi cpu: 200m requests: memory: 100Mi cpu: 100m
Now to verify:
$ kubectl describe po stress Name: stress Namespace: default Labels: run=stress IP: 10.36.1.17 Containers: stress: Image: polinux/stress Limits: cpu: 200m memory: 200Mi Requests: cpu: 100m memory: 100Mi
To quickly generate secrets, use the --from-literal flag like this:
$ kubectl create secret generic my-secret --from-literal=foo=bar -o yaml --dry-run > my-secret.yaml
This produces the following secret that can then be consumed by your containers. The value is encoded.
apiVersion: v1 data: foo: YmFy kind: Secret metadata: creationTimestamp: null name: my-secret
Now create the secret:
$ kubectl create -f my-secret.yaml secret "my-secret" created
Now have a look at it:
kubectl get secret my-secret -o yaml apiVersion: v1 data: foo: YmFy kind: Secret metadata: name: my-secret namespace: default uid: bae5b8d8-d01a-11e8-8972-42010a800002 type: Opaque
$ echo "YmFy" | base64 --decode bar
Secrets can be mounted as data volumes or be exposed as environment variables to be used by a container in a pod. Here we'll mount our above secret as a volume.
apiVersion: v1 kind: Pod metadata: name: secrets-test-pod spec: containers: - image: nginx name: test-container volumeMounts: - mountPath: /etc/secret/volume name: secret-volume volumes: - name: secret-volume secret: secretName: my-secret
When pods are created by K8s they are provided an identify via the service account. In most cases, pods use the default service account, but it can be specified directly.
apiVersion: v1 kind: Pod metadata: name: my-pod spec: serviceAccountName: build-robot ...
This particular section needs additional detail as these concepts are not covered that well via the tasks provided at kubernetes.io. Actually, the best coverage (for sidecars) is in the concepts section under logging architecture.
One or more containers running within a pod for enhancing the main container functionality (logger container, git synchronizer container); These are sidecar container
One or more containers running within a pod for accessing external applications/servers (Redis cluster, memcache cluster); These are called ambassador container
One or more containers running within a pod to allow access to application running within the container (Monitoring container); These are called as adapter containers-
The Pod design section mostly covers deployments, rolling updates, and rollbacks (and jobs). These are all covered well in the tasks section later in this document. The primary trick here is to really understand the basic commands for updating a deployment which causes a new replicaSet to be created for the rollout. Both replica sets exist as the rollout continues and completes.
Below is a quick example of creating a deployment and then updating its image. This will force a rolling deployment to start. You can then roll it back.
$ kubectl run nginx --image=nginx --replicas=3 deployment.apps "nginx" created
Okay, now force a rolling update by updating its image.
$ kubectl set image deploy/nginx nginx=nginx:1.9.1 deployment.apps "nginx" image updated
Now you can check the status of the roll out.
$ kubectl rollout status deploy/nginx Waiting for rollout to finish: 2 out of 3 new replicas have been updated... Waiting for rollout to finish: 1 old replicas are pending termination... Waiting for rollout to finish: 2 of 3 updated replicas are available... deployment "nginx" successfully rolled out
Now, if you want to roll it back:
$ kubectl rollout undo deploy/nginx $ kubectl rollout status deploy/nginx Waiting for rollout to finish: 1 old replicas are pending termination... Waiting for rollout to finish: 2 of 3 updated replicas are available... deployment "nginx" successfully rolled out
This is all describe well on kubernetes.io by searching for 'deployment' and reading the overview there. Kubernetes Deployments
Jobs and CronJobs
Job vs CronJob -> A job runs a pod to a number of successful completions. Cron jobs manage jobs that run at specified intervals and/or repeatedly at a specific point in time, thus they have the 'schedule' aspect. The below demonstrates quickly creating a cronjob and then a quick edit to add a command, etc.
$ kubectl run crontest --image=busybox --schedule="*/1 * * * *" --restart=OnFailure --dry-run -o yaml apiVersion: batch/v1beta1 kind: CronJob metadata: labels: run: crontest name: crontest spec: schedule: '*/1 * * * *' jobTemplate: spec: template: spec: containers: - image: busybox name: crontest command: ["date; echo Hello"] restartPolicy: OnFailure
You can also redirect (> cron.yaml) the above to a file, edit it to add the container command, and then create the cronjob with the kubectl create:
$ kubectl create -f cron.yaml
This is still one of my weaknesses and the whole PV creation is high dependent on the underlying cloud or file storage technique used. For now, the links provided later in the persistence tasks are best for studying this.
This part of the curriculum covers the logging, debugging, and metrics of your running applications.
Container metrics require that heapster be running, and it is pretty standard on clusters now.
$ kubectl top pod -n my-namespace $ kubectl top node -n my-namespace
SERVICES and NETWORKING
Services are pretty straight forward, but there are lots of networking details in a k8s cluster. The curriculum only mentions network policies so you should understand that particular aspect of networking in good detail.
Services provide a persistent endpoint for a logical set of pods. This endpoint is typically used to expose a pods services externally. They are quite straight forward and quick to build and configure, so the concepts are more important than the 'speed' factor for the exam, IMO. There are several ways to expose a service as well as the underlying detail of using selectors to ultimately select the target pods of the service.
'Exposing' Ports for PODS
By default pods can all inter-communicate via their internal IP address and port. Services are needed to expose services OUTSIDE of the cluster. So, it's important to understand the basic container spec for specifying the port a container will use. The example below declares the port as well as an environment variable describing same.
spec: containers: image: nginx imagePullPolicy: Always name: busybox env: - name: PORT value: "80" ports: - containerPort: 80 protocol: TCP
Resources use labels to select pods and define rules which specify what traffic is allowed to the selected pods. So, the pods themselves require certain labels / selectors to enable network policies.
By default, pods are non-isolated; they accept traffic from any source.
Pods become isolated by having a NetworkPolicy that selects them. Once there is any NetworkPolicy in a namespace selecting a particular pod, that pod will reject any connections that are not allowed by any NetworkPolicy. (Other pods in the namespace that are not selected by any NetworkPolicy will continue to accept all traffic.)
MISCELANEOUS TIPS and TRICKS
Extracting yaml from running resource
Use the --export and -o yaml flags to export the basic yaml from an existing resource:
kubectl get deploy busybox --export -o yaml > exported.yaml
The --dry-run flag
The --dry-run flag can be used with the kubectl run and create commands. It provides a nice template to start your declarative yaml config files. Below is an example for creating a basic secret yaml.
kubectl create secret generic my-secret --from-literal=foo=bar -o yaml --dry-run > my-secret.yaml
The --from-literal flag
As shown above, the --from-literal flag is useful for things like config maps and secrets for the basic cases.
apiVersion: v1 data: foo: YmFy kind: Secret metadata: creationTimestamp: null name: my-secret
Tasks from Kubernetes Doc
The following are primarily links to either the 'concepts' or 'tasks' section of the kubernetes.io documentation. The 'task' items are very useful to use as labs. I've tied them directly to the curriculum to ensure they are appropriate study material for the exam.
Core Concepts and Kubectl
- Tasks -> Accessing Multiple Clusters
- Tasks -> Accessing Cluster with API
- Tasks -> Port Forwarding
- Tasks -> Shell to Running Container (exec)
- Task -> Config Maps
- Task -> Security Contexts
- Tasks -> Assigning Memory Resources to Pods
- Tasks -> Assigning CPU Resources to Pods
- Tasks -> Pod QOS
- Tasks -> Credentials using Secrets
- Tasks -> Project Volume w/Secrets
- Tasks -> Setting Service Account
- Concepts -> Assign Pods to Nodes - Selectors
- Concepts -> Labels and Selectors
- Tasks -> ReplicaSet Rolling Updates
- Concepts -> Deployments, Rollouts, and Rollbacks
- Tasks -> Automated Tasks with Cron Jobs
- Tasks -> Parallel Jobs with Expansions
- Tasks -> Course Parallel Processing with a Work Queue
- Tasks -> Fine Parallel Processsing with a Work Queue
- Tasks -> App Introspection and Debugging
- Tasks - Liveness and Readiness Probes
- Tasks -> Debugging Pods
- Tasks -> Troubleshooting Applications
- Tasks -> Debugging Services
- Tasks -> Debugging Services Locally
- Tasks -> Core Metrics Pipeline