diff --git a/docs/cloud_operations/docker/_category_.json b/docs/cloud_operations/docker/_category_.json new file mode 100644 index 0000000..6e43e99 --- /dev/null +++ b/docs/cloud_operations/docker/_category_.json @@ -0,0 +1,9 @@ +{ + "label": "Docker", + "position": 2, + "link": { + "type": "generated-index", + "description": "Simple Guide to Docker" + } + } + \ No newline at end of file diff --git a/docs/cloud_operations/docker/docker-arch.png b/docs/cloud_operations/docker/docker-arch.png new file mode 100644 index 0000000..cb01d50 Binary files /dev/null and b/docs/cloud_operations/docker/docker-arch.png differ diff --git a/docs/cloud_operations/docker/docker-basics.md b/docs/cloud_operations/docker/docker-basics.md new file mode 100644 index 0000000..793e372 --- /dev/null +++ b/docs/cloud_operations/docker/docker-basics.md @@ -0,0 +1,909 @@ +--- +id: docker-basics +title: Docker Basics +description: Essential Docker CLI commands +slug: /docker/basics +sidebar_position: 2 +--- + +## Starting a container + +:::warning + +Make the distinction between a **docker image** and a **docker container**. We can see the docker +image as the template, containing a set of instructions, used for creating and running a container. +A docker container is the running instance of an image. This is similar to the distinction between +a program and a process (i.e. a process is a running instance of a program). You can read more +about this difference [here](https://aws.amazon.com/compare/the-difference-between-docker-images-and-containers/#:~:text=A%20Docker%20container%20is%20a%20self%2Dcontained%2C%20runnable%20software%20application,containers%20over%20an%20application's%20lifecycle.). + +::: + +In order to start a Docker container we use the following command: + +```bash +cristian@cristianson:~/Desktop/ipw-docker$ docker run -it ubuntu:22.04 bash +Unable to find image 'ubuntu:22.04' locally +22.04: Pulling from library/ubuntu +3713021b0277: Already exists +Digest: sha256:340d9b015b194dc6e2a13938944e0d016e57b9679963fdeb9ce021daac430221 +Status: Downloaded newer image for ubuntu:22.04 +root@78f701a0d391:/# ls +bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var +root@78f701a0d391:/# +``` + +:::info + +If the above command requires superuser privileges, (i.e. run with **sudo**), then follow these +[steps](https://docs.docker.com/engine/install/linux-postinstall/) to avoid prefixing every command +with **sudo**. + +::: + +Let's break down the arguments of the `docker` command: + +- `run`, starts the container +- `-i`, the container is started in **interactive** mode, which means that it can accept keyboard +input +- `-t`, associates a terminal to the run command +- `ubuntu:22.04` is the name of the **image** : **version** we want to use. Keep in mind that if we +do not explicitly specify the version, than the latest image will be pulled from +[Dockerhub](https://hub.docker.com/) +- `bash`, the command we want to run in the container + +:::info + +Dockerhub is a public image repository that contains prebuilt images that we can download. + +::: + +If we want to see the local images we have downloaded from Dockerhub or created locally, we can do +`docker image ls`. + +```bash + +cristian@cristianson:~/Desktop/ipw-docker$ docker image ls +REPOSITORY TAG IMAGE ID CREATED SIZE +ubuntu 22.04 8a3cdc4d1ad3 4 weeks ago 77.9MB +ubuntu latest 35a88802559d 7 weeks ago 78.1MB + +``` + +:::tip + +If you do not know what an argument does or what is the purpose of a command, use `man docker` or + `docker help`. + +::: + +We can also run non-interactive commands in containers: + +```bash +cristian@cristianson:~/Desktop/ipw-docker$ docker run ubuntu:22.04 ls +bin +boot +dev +etc +home +lib +lib32 +lib64 +libx32 +media +mnt +opt +proc +root +run +sbin +srv +sys +tmp +usr +var +``` + +:::note + +This time, the command just shows us the output of **ls** and the container exits immediately. This +is because we have run this command in the **foreground**. + +::: + +:::tip + +Try to also run the `sleep 5` command and see what happens! + +::: + +Sometimes, however, running commands in the foreground is not ideal, especially if the command +takes a long time to run/output something. During that time, our terminal input is basically +blocked and we have to open another terminal tab if we want to do something else. This is why, when +we are required to run a command or a script that takes a long time, it is better to run the +command in the background. + +In order to start a container in the background, we use the `-d` option for the `docker run` +command as follows: + +```bash +cristian@cristianson:~/Desktop/ipw-docker$ docker run -d ubuntu:22.04 sleep 100 +8b3d484ae9ad92f669d2780faaa1b1dc850922029391bf13a12de84014610758 +cristian@cristianson:~/Desktop/ipw-docker$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +8b3d484ae9ad ubuntu:22.04 "sleep 100" 2 seconds ago Up 2 seconds distracted_sammet +``` + +The breakdown of the columns in the `docker ps` output are: + +- `CONTAINER ID` - a unique id assigned by docker to each container. +- `IMAGE` - the name of the image that served as a template for this container +- `COMMAND` - the command we have issued when starting the container +- `PORTS` - ports the container exposes for communication with the outside world +- `NAMES` - a name which is randomly assigned by Docker + +:::tip + +You can change the name of the container when you are starting it. Do `docker run --help`, find the +option and then restart the ubuntu container with a new name! Do `docker ps` to see if the name +changed. Also, whenever you are in doubt about what a command is supposed to do or what options it +takes, the general form is `docker --help` to list all of the available options. + +::: + +Observe the fact that this time the container did not exit, and is running in the background. The +container will stop after the provided command, in our case, `sleep 100`, finishes its execution. +Running `docker ps` after 100 seconds confirms this: + +```bash + +cristian@cristianson:~/Desktop/ipw-docker$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + +``` + +:::tip + +Run the `docker ps` command after starting a container in the foreground! You need to open another +terminal tab in order to do this. + +::: + +After starting a container in the background using the `-d` option, we can also connect to it +interactively with the `docker exec` command. + +```bash + +cristian@cristianson:~/Desktop/ipw-docker$ docker run -d ubuntu:22.04 sleep 1000 +48d58d5ab0a17c69dadcf5e3c6cfd8be519845cae3c67f41da19fe5ffc1f6382 +cristian@cristianson:~/Desktop/ipw-docker$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +48d58d5ab0a1 ubuntu:22.04 "sleep 1000" 11 seconds ago Up 10 seconds zen_hodgkin +cristian@cristianson:~/Desktop/ipw-docker$ docker exec -it 48d58d5ab0a1 /bin/bash +root@48d58d5ab0a1:/# ls +bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var +root@48d58d5ab0a1:/# + +``` + +The format of the `docker exec` command is similar to that of `docker run`. We have used the `-it` +flags to start an interactive session with an attached terminal and we have chosen to run the +`/bin/bash` command. It is important to note that the container is uniquely identified via its +**ID** or assigned name in the **NAMES** column. + +Now, we want to stop the running container because we its no fun to wait 1000 seconds to exit +automatically. In order to do this, we use the `docker stop` command with the container's **ID** or +**NAME**. + +```bash + +cristian@cristianson:~/Desktop/ipw-docker$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +48d58d5ab0a1 ubuntu:22.04 "sleep 1000" 5 minutes ago Up 5 minutes zen_hodgkin +cristian@cristianson:~/Desktop/ipw-docker$ docker stop 48d58d5ab0a1 +48d58d5ab0a1 +cristian@cristianson:~/Desktop/ipw-docker$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +cristian@cristianson:~/Desktop/ipw-docker$ docker ps -a +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +48d58d5ab0a1 ubuntu:22.04 "sleep 1000" 5 minutes ago Exited (137) 3 seconds ago zen_hodgkin +8b3d484ae9ad ubuntu:22.04 "sleep 100" 24 hours ago Exited (0) 24 hours ago distracted_sammet +a236cc7b0efa ubuntu:22.04 "sleep 5" 24 hours ago Exited (0) 24 hours ago hardcore_ritchie +94ef886a0e61 ubuntu:22.04 "sleep 1" 24 hours ago Exited (0) 24 hours ago serene_keller +c7591793567d ubuntu:22.04 "ls" 24 hours ago Exited (0) 24 hours ago adoring_jang +d5cd0c63b9bb ubuntu:22.04 "ps aux" 24 hours ago Exited (0) 24 hours ago condescending_mcclintock +f81e1edf1b36 ubuntu:22.04 "lsdir" 24 hours ago Created condescending_wu +77fa7ff22c40 ubuntu:22.04 "ls" 24 hours ago Exited (0) 24 hours ago pedantic_lewin +707ae3470fe6 ubuntu:22.04 "ps -ef" 24 hours ago Exited (0) 24 hours ago exciting_heisenberg +cf3998b22236 ubuntu:22.04 "cat" 24 hours ago Exited (0) 24 hours ago bold_ritchie +78f701a0d391 ubuntu:22.04 "bash" 25 hours ago Exited (130) 24 hours ago unruffled_feistel +081fcd62be22 ubuntu "bash" 25 hours ago Exited (130) 25 hours ago interesting_swanson +f65bb2661f94 ubuntu "bash" 25 hours ago Exited (130) 25 hours ago friendly_liskov +5b7f19201652 alpine "shell" 25 hours ago Created youthful_roentgen +eb2c9ced368b alpine "bash" 25 hours ago Created magical_satoshi +5b27ae6a1c47 alpine "bash" 25 hours ago Created epic_volhard +cristian@cristianson:~/Desktop/ipw-docker$ + +``` + +We can see that the container is no longer running. Sometimes the stop command takes a while, so +do not abort it. Also, if we pass the `-a` argument to the `docker stop` command, it will also list +the containers that were stopped. We can see that the first container, **zen_hodgkin** is the one +we stopped earlier. + +## Exercise 1 + +- Start a container of your choice in background. Name it 'IPW-ROCKS'. +- Once started, connect to the container and install the `fzf` tool. +- Disconnect from the container. +- **NEW!** Try to pause and unpause the container. After each command, do a `docker ps`. +- Stop the container. +- **NEW** Completely remove the stopped container. + +:::tip + +You must start your container with a long running command or script, otherwise it will exit +immediately. + +Also, **you are not allowed** to use Google to search how to do the pause/unpause/container removal. +💀 Use `docker help` and `grep` in order to find what you need. 😉 +::: + +## Let's create our own docker image + +### Why would we want to create multiple images for multiple containers? + +So far, we have used the containers interactively. Most of the times, however, this is not the case. +A container is a separate unit of computing with a well defined purpose. That is, it should do one +single thing, and do it well. + +For example, we might have a web application with multiple components, and we have decided to split +encapsulate each component in its own docker container. That is: + +- a database container +- a backend container +- a frontend container + +Each of the above containers does one thing, and in the case of a backend or frontend change, the +rest of the containers remain unaffected and running. Even if one container crashes, we can easily +restart it without affecting the rest of the components. + +### Building an image + +The flow of building an image and deploying a container looks like this: + +![Docker image build](overview.drawio.png "Build process") + +In order to create our custom container, we need to create a custom template, that is, a custom +docker image. To accomplish this, we will create a `Dockerfile`. + +```text + +FROM ubuntu:22.04 + +ARG DEBIAN_FRONTEND=noninteractive +ARG DEBCONF_NONINTERACTIVE_SEEN=true + +ENV HELLO="hello" + +RUN apt-get update +RUN apt-get install -y firefox + +``` + +Let's break down each line of the above document: + +- `FROM` - the first instruction in each Dockerfile, specifies the base container image, which means +that subsequent modifications will add/remove from this image. +- `ARG` - represents a variable that is available only when the container is built and can be +referenced throughout the Dockerfile. +- `ENV` - sets an environment variable that will be available in the resulting container at +runtime. +- `RUN` - runs a command when building the image. In this case, the resulting image will have +`firefox` pre-installed. + +:::info + +You can read more about the differences between **ARG** and **ENV** +[here](https://vsupalov.com/docker-arg-vs-env/). + +::: + +Once we have created the `Dockerfile`, we can build our image using the following command: +`docker build -t my-container .` + +```bash + +cristian@cristianson:~/Desktop/ipw-docker$ docker build -t my-container . +[+] Building 30.7s (7/7) FINISHED +[...] + => exporting to image 1.0s + => => exporting layers 0.9s + => => writing image sha256:7493be1166b06d3521599a21c1ece1c5b4e2d438c3dacef0935e74d927aa875e 0.0s + => => naming to docker.io/library/my-container + +``` + +Let's break down the arguments to the `docker build` command: + +- `-t` - specifies the tag of the image. +- **my-container** is the assigned tag. +- **.** - specifies that the Dockerfile is located in the current directory + +:::note + +In larger projects, we may have multiple Dockerfiles, each specifying the recipe for another image. +It is useful, then, to name them differently. However, by default, Docker recognizes only files +named `Dockerfile`. In order to have files named `Dockerfile.backend` or `Dockerfile.frontend` or +any other name we may come up with, we need to specify this to the `docker build` command via the +`-f` parameter. See `docker build --help` for more info. + +::: + +Now that we have built our image, let's run `docker image ls`: + +```bash +cristian@cristianson:~/Desktop/ipw-docker$ docker image ls +REPOSITORY TAG IMAGE ID CREATED SIZE +my-container latest 7493be1166b0 13 minutes ago 369MB + +``` + +This is the confirmation that the build was successful. Let's create a brand new container from this +image and verify if the environment variable has been correctly set up: + +```bash + +cristian@cristianson:~/Desktop/ipw-docker$ docker run -it my-container bash +root@40b9e8dae8f1:/# echo $HELLO +hello +root@40b9e8dae8f1:/# + +``` + +Nice! We did it. We could have also checked that the image had the **HELLO** environment variable +set by using the `docker image inspect` command. + +```bash + +cristian@cristianson:~/Desktop/ipw-docker$ docker image inspect my-container +[ + { + "Id": "sha256:7493be1166b06d3521599a21c1ece1c5b4e2d438c3dacef0935e74d927aa875e", + "RepoTags": [ + "my-container:latest" + ], + "RepoDigests": [], + "Parent": "", + "Comment": "buildkit.dockerfile.v0", + "Created": "2024-08-01T13:51:50.003474082+03:00", + "Container": "", + "ContainerConfig": { + "Hostname": "", + "Domainname": "", + "User": "", + "AttachStdin": false, + "AttachStdout": false, + "AttachStderr": false, + "Tty": false, + "OpenStdin": false, + "StdinOnce": false, + "Env": null, + "Cmd": null, + "Image": "", + "Volumes": null, + "WorkingDir": "", + "Entrypoint": null, + "OnBuild": null, + "Labels": null + }, + "DockerVersion": "", + "Author": "", + "Config": { + "Hostname": "", + "Domainname": "", + "User": "", + "AttachStdin": false, + "AttachStdout": false, + "AttachStderr": false, + "Tty": false, + "OpenStdin": false, + "StdinOnce": false, + "Env": [ + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", + "HELLO=hello" + ], + "Cmd": [ + "/bin/bash" + ], + "Image": "", + "Volumes": null, + "WorkingDir": "", + "Entrypoint": null, + "OnBuild": null, + "Labels": { + "org.opencontainers.image.ref.name": "ubuntu", + "org.opencontainers.image.version": "22.04" + } + }, + "Architecture": "amd64", + "Os": "linux", + "Size": 369369133, + "GraphDriver": { + "Data": { + "LowerDir": "/var/lib/docker/overlay2/p323vqywxcogfgl6sadeqwrsc/diff:/var/lib/docker/overlay2/372a31c779498a88a96829322ca93f496d0cf79a3a23f4c46b276f6670199ccc/diff", + "MergedDir": "/var/lib/docker/overlay2/aakuflif4l5nqvh24azlltsw4/merged", + "UpperDir": "/var/lib/docker/overlay2/aakuflif4l5nqvh24azlltsw4/diff", + "WorkDir": "/var/lib/docker/overlay2/aakuflif4l5nqvh24azlltsw4/work" + }, + "Name": "overlay2" + }, + "RootFS": { + "Type": "layers", + "Layers": [ + "sha256:931b7ff0cb6f494b27d31a4cbec3efe62ac54676add9c7469560302f1541ecaf", + "sha256:7b75401998b8840828c675a5956ab91e405aec86d363e76b4a0d645bb1a8414e", + "sha256:c7e0e739bfbbcc8f59a777e8bb57b845ece9598a8a1df45833dc215104ad7dd1" + ] + }, + "Metadata": { + "LastTagTime": "2024-08-01T13:51:50.961377181+03:00" + } + } +] + +``` + +We can see that in the `Env` section we have our **HELLO** env variable. + +:::info + +Each Docker image is comprised of layers. Each command in the Dockerfile basically adds a new layer +that can be cached and later be used in other builds. Talking about the very inner workings of +Docker is beyond the scope of this workshop, but you can read more information here: + +- [Docker storage driver](https://docs.docker.com/storage/storagedriver/) +- [Docker image optimization](https://cloudyuga.guru/blogs/understanding-docker-image-optimization-techniques-for-effective-deployment/#:~:text=Minimize%20The%20Number%20Of%20Layers,-In%20this%20technique&text=Each%20instruction%20like%20FROM%2C%20COPY,size%20of%20the%20resulting%20image.) +- [Number of docker layers](https://stackoverflow.com/questions/47079114/should-i-minimize-the-number-of-docker-layers) + +::: + +:::tip + +With time, a system can accumulate lots of local images, containers and build caches. That means +that a user may end up with 0 space left on its laptop/PC. So, it is useful to see how much storage +Docker occupies. In order to do this, run the `docker system df` command. Ask one of the course +instructors for more details about the output and how you can free up disk space. + +::: + +## Exercise 2 + +- Write a `Dockerfile.image` file containing the instructions for generating a container image +based on `ubuntu`. The image should have the `24.04` version. +- **NEW** Create a file called `test.txt` in the same folder with `Dockerfile.image`. Copy this file +inside the container with some content inside. +- Set an environment variable called **MESSAGE** to whatever message you want. +- **NEW** Using `echo`, append the output of the environment variable to the copied file. +- Using a specific command, create the image such as, when running it non-interactively, it +outputs the contents of the file. Basically, add a default for executing the container. + +:::tip + +Have a look on the [Dockerfile reference](https://docs.docker.com/reference/dockerfile/) for the +required commands. + +::: + +## Docker networking + +The Docker networking subsystem is plugable and uses a variety of drivers in order to offer implicit +behavior for network components. Why do we care about the networking subsystem? Because in order to +build useful apps, we need to make the containers communicate with each other. Moreover, we may +even want to isolate the traffic between certain containers and create sub-networks. + +:::info + +You can read for about docker networking [here](https://docs.docker.com/network/). + +::: + +Containers residing in the same network can communicate with each other using **named DNS**. This +means that we can access a container using its name, and not necessarily its IP. In order to +communicate with the outside world (the host machine, containers which are outside the network), +you must expose [ports](https://www.mend.io/blog/how-to-expose-ports-in-docker/). + +Moving forward, we are going to demonstrate how the `bridge` networks work in Docker. You can read +more about them [here](https://docs.docker.com/network/drivers/bridge/). We are going to start two +containers and try to send pings from one another to see if anything happens. In order to do this, +it is easier if you open two separate terminal tabs. + +```bash + +cristian@cristianson:~$ docker container run --name first -it alpine ash +/ # + + +``` + +```bash + +cristian@cristianson:~$ docker container run --name second -it alpine ash +/ # + +``` + +This time, we have started two alpine containers, because they are more lightweight than the ubuntu +ones. `ash` is the default shell for the `alpine` containers. Now, if we try to ping from the `first` +container the `second` container, we see that this does not work. Same story if we try the same +thing from the `second` container. + +```bash + +cristian@cristianson:~$ docker container run --name first -it alpine ash +/ # ping second +ping: bad address 'second' +/ # + +``` + +```bash + +cristian@cristianson:~$ docker container run --name second -it alpine ash +/ # ping first +ping: bad address 'first' +/ # + +``` + +If we do an `ifconfig` inside one of the containers, we see that there are only two networks +available to us right now: + +- lo +- eth0 + +You can ask the course instructors about more information about these two networks. + +```bash + +/ # ifconfig +eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:02 + inet addr:172.17.0.2 Bcast:172.17.255.255 Mask:255.255.0.0 + UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 + RX packets:67 errors:0 dropped:0 overruns:0 frame:0 + TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 + collisions:0 txqueuelen:0 + RX bytes:8933 (8.7 KiB) TX bytes:216 (216.0 B) + +lo Link encap:Local Loopback + inet addr:127.0.0.1 Mask:255.0.0.0 + UP LOOPBACK RUNNING MTU:65536 Metric:1 + RX packets:0 errors:0 dropped:0 overruns:0 frame:0 + TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 + collisions:0 txqueuelen:1000 + RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) + +/ # + + +``` + +Let's make these containers communicate! First, create a docker network object: + +```bash + +cristian@cristianson:~$ docker network create -d bridge my-bridge +8508d7585c6b8145da8afac4bf159de14293c4fd11ebcf2662e3367fc46d92c9 +cristian@cristianson:~$ docker network ls +NETWORK ID NAME DRIVER SCOPE +9ccf3f0b6346 bridge bridge local +1e22e9263c46 host host local +8508d7585c6b my-bridge bridge local +294f9f02c5c1 none null local + +``` + +:::tip + +Use `docker network --help` to find out more about the command. Ask one of the course instructors +for more information if necessary. + +::: + +Listing the available networks with `docker network ls` shows the newly created `my-bridge` network +of type `bridge`. Now, let's connect the two containers to the network. Keep in mind we are adding +the containers to the network while they are still running. We could have also added them at creation. + +```bash + +cristian@cristianson:~$ docker network connect my-bridge first +cristian@cristianson:~$ docker network connect my-bridge second + +``` + +It was that easy! Running an `ifconfig` now yields: + +```bash + +/ # ifconfig +eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:02 + inet addr:172.17.0.2 Bcast:172.17.255.255 Mask:255.255.0.0 + UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 + RX packets:129 errors:0 dropped:0 overruns:0 frame:0 + TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 + collisions:0 txqueuelen:0 + RX bytes:18254 (17.8 KiB) TX bytes:216 (216.0 B) + +eth1 Link encap:Ethernet HWaddr 02:42:AC:14:00:02 + inet addr:172.20.0.2 Bcast:172.20.255.255 Mask:255.255.0.0 + UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 + RX packets:66 errors:0 dropped:0 overruns:0 frame:0 + TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 + collisions:0 txqueuelen:0 + RX bytes:9143 (8.9 KiB) TX bytes:0 (0.0 B) + +lo Link encap:Local Loopback + inet addr:127.0.0.1 Mask:255.0.0.0 + UP LOOPBACK RUNNING MTU:65536 Metric:1 + RX packets:0 errors:0 dropped:0 overruns:0 frame:0 + TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 + collisions:0 txqueuelen:1000 + RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) + + +``` + +We have one extra network interface, `eth1`. Let's ping again the `second` container from `first`. + +```bash + +/ # ping -c2 second +PING second (172.20.0.3): 56 data bytes +64 bytes from 172.20.0.3: seq=0 ttl=64 time=0.328 ms +64 bytes from 172.20.0.3: seq=1 ttl=64 time=0.181 ms + +--- second ping statistics --- +2 packets transmitted, 2 packets received, 0% packet loss +round-trip min/avg/max = 0.181/0.254/0.328 ms +/ # + +``` + +Nice! This time everything works as expected. Observe the fact that we have used the container's +name and not the IP. You can stop and remove the containers now. + +## Exercise 3 + +- Create a network called `ipw` of type `bridge`. +- **NEW** Create two containers and assign them to the `ipw` network at creation. +- Check if the containers can communicate. +- **NEW** In another terminal tab, do `docker network inspect ipw` and comment on the output with +one of the course instructors. +- **NEW** Do a `cat /etc/hosts` in each container and comment on the output with one of the course +instructors. +- Stop the containers, remove them and also remove the newly created network. + +:::warning + +You are not allowed to use Google! Use `docker --help` whenever you can to get more +information, or ask one of the course instructors. + +::: + +## Docker persistence + +In Docker, data we create or edit inside a container is not persisted in the outside world. This is +due to the way Docker works and the particularities of its filesystem. Let's illustrate this: + +:::info + +You can read more about this here: + +- [MobyLab](https://mobylab.docs.crescdi.pub.ro/docs/softwareDevelopment/laboratory1/persistence) +- [OverlaysFs](https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt) +- [AuFs](https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt) + +::: + +### Volumes + +In order to persist data from a container, Docker uses a mechanism called **volumes**. These volumes +represent a mapping between files in the container and files on the host system. The major advantage +of Docker volume is the fact that they are not tied to the lifetime of the container they are +attached to. This means that even if a container crashes, stops or is deleted, its data will still +persist in the outside world, because volumes are an outside abstraction that are just linked to +container, but have a standalone lifetime. Other advantages of volumes include: + +- easy migration between containers and machines +- can be configured via the CLI or Docker API +- can be shared between multiple container, which means that volumes represent a way of +*communication* via storage +- by employing different storage drivers, volumes can be used to persist data on remote machines, +cloud environments, network drives etc. + +Volumes managed by the Docker engine are also called **named volumes**. There are multiple ways of +defining volumes: + +- by using the **VOLUME** command inside the Dockerfile when creating the image, see the +[Docker reference](https://docs.docker.com/reference/dockerfile/#volume) +- at runtime, when creating a volume +- with a docker compose file (more on that later) and the docker volume API: `docker volume create`, +`docker volume ls`, etc. + +Let's see how we can create a volume a runtime with the following command: + +```bash + +cristian@cristianson:~$ docker container run --name ipw -d -v /test alpine sh -c 'ping 8.8.8.8 > /test/ping.txt' +3e6beddbd15e43365be7f863023a43cffcdbab86916d78c553ec0822b58f9b6a4 +cristian@cristianson:~$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +3e6beddbd15e alpine "sh -c 'ping 8.8.8.8…" 2 seconds ago Up 1 second ipw + +``` + +The `-v` argument followed by the volume name defines a volume at the `/test` path inside the +container. Every file we create or modify in that folder will basically alter the volume. Also note +that we are using a long running command, `sh -c 'ping 8.8.8.8 > /test/ping.txt'`, in order to +continuously append data to the file. + +Now, if we do a `docker volume ls` we should see: + +```bash + +cristian@cristianson:~$ docker volume ls +DRIVER VOLUME NAME +local a5ec5808eb58a6cc5551bd5f979f038f99015668f79314bec28ada192880d065 + +``` + +Let's see if we can get more information about our newly created volume: + +```bash + +cristian@cristianson:~$ docker volume inspect a5ec5808eb58a6cc5551bd5f979f038f99015668f79314bec28ada192880d065 +[ + { + "CreatedAt": "2024-08-02T12:20:22+03:00", + "Driver": "local", + "Labels": { + "com.docker.volume.anonymous": "" + }, + "Mountpoint": "/var/lib/docker/volumes/a5ec5808eb58a6cc5551bd5f979f038f99015668f79314bec28ada192880d065/_data", + "Name": "a5ec5808eb58a6cc5551bd5f979f038f99015668f79314bec28ada192880d065", + "Options": null, + "Scope": "local" + } +] + +``` + +The `Mountpoint` label specifies the location on the host machine were the volume data is stored. If +we list the contents of that folder than we would get the following output: + +```bash + +cristian@cristianson:~$ sudo ls /var/lib/docker/volumes/a5ec5808eb58a6cc5551bd5f979f038f99015668f79314bec28ada192880d065/_data +[sudo] password for cristian: +ping.txt + +``` + +Doing a `cat` inside the file we get: + +```bash + +cristian@cristianson:~$ sudo cat /var/lib/docker/volumes/a5ec5808eb58a6cc5551bd5f979f038f99015668f79314bec28ada192880d065/_data/ping.txt +PING 8.8.8.8 (8.8.8.8): 56 data bytes +64 bytes from 8.8.8.8: seq=0 ttl=57 time=20.006 ms +64 bytes from 8.8.8.8: seq=1 ttl=57 time=20.352 ms +64 bytes from 8.8.8.8: seq=2 ttl=57 time=18.195 ms +64 bytes from 8.8.8.8: seq=3 ttl=57 time=18.668 ms + +``` + +Now, if we stop and remove the container, with + +- `docker container stop ipw` +- `docker container rm ipw` + +we see that the volume data is still intact, event though the container was destroyed: + +```bash + +cristian@cristianson:~$ sudo cat /var/lib/docker/volumes/a5ec5808eb58a6cc5551bd5f979f038f99015668f79314bec28ada192880d065/_data/ping.txt +PING 8.8.8.8 (8.8.8.8): 56 data bytes +64 bytes from 8.8.8.8: seq=0 ttl=57 time=20.006 ms +64 bytes from 8.8.8.8: seq=1 ttl=57 time=20.352 ms +64 bytes from 8.8.8.8: seq=2 ttl=57 time=18.195 ms +64 bytes from 8.8.8.8: seq=3 ttl=57 time=18.668 ms + +``` + +This proves the fact that the volume and container have separate lifetimes. + +### Bind mounts + +Besides volumes, we also have the concept of a **bind mount**. These are somewhat similar, the main +difference being that bind mounts are not managed by Docker, but by the file system of the host +machine and can be accessed by any external process which does not belong to Docker. A bind mount is, +in its purest form, a path to a location in the host machine, while a volume is a Docker abstraction +that behind the scenes uses bind mounts. All in all, bind mounts allow us to **import** and access +folders, files and paths from the host machine in our Docker container and persist any modification. + +:::info + +You can read more about volumes and bind mounts [here](https://docs.docker.com/storage/bind-mounts/). + +::: + +We can add a bind mount to a container in a similar fashion, when we are creating it. + +```bash + +cristian@cristianson:~/Desktop/ipw-docker$ docker container run --name first -d --mount type=bind,source=/home/cristian/Desktop/ipw-docker/test.txt,target=/root/test.txt alpine sh -c 'ping 8.8.8.8 > /root/test.txt' +8438bfb2f16d940770ed3e4ba48cb67428e78ff530ec73de859a5f168d36e8ab +cristian@cristianson:~/Desktop/ipw-docker$ cat test.txt +PING 8.8.8.8 (8.8.8.8): 56 data bytes +64 bytes from 8.8.8.8: seq=0 ttl=57 time=21.369 ms +64 bytes from 8.8.8.8: seq=1 ttl=57 time=20.204 ms +64 bytes from 8.8.8.8: seq=2 ttl=57 time=20.705 ms +64 bytes from 8.8.8.8: seq=3 ttl=57 time=18.625 ms +64 bytes from 8.8.8.8: seq=4 ttl=57 time=20.626 ms +64 bytes from 8.8.8.8: seq=5 ttl=57 time=20.248 ms +64 bytes from 8.8.8.8: seq=6 ttl=57 time=18.777 ms +cristian@cristianson:~/Desktop/ipw-docker$ docker container stop first +first +cristian@cristianson:~/Desktop/ipw-docker$ docker container rm first +first +cristian@cristianson:~/Desktop/ipw-docker$ cat test.txt +PING 8.8.8.8 (8.8.8.8): 56 data bytes +64 bytes from 8.8.8.8: seq=0 ttl=57 time=21.369 ms +64 bytes from 8.8.8.8: seq=1 ttl=57 time=20.204 ms +64 bytes from 8.8.8.8: seq=2 ttl=57 time=20.705 ms +64 bytes from 8.8.8.8: seq=3 ttl=57 time=18.625 ms +64 bytes from 8.8.8.8: seq=4 ttl=57 time=20.626 ms +64 bytes from 8.8.8.8: seq=5 ttl=57 time=20.248 ms +64 bytes from 8.8.8.8: seq=6 ttl=57 time=18.777 ms + +``` + +This is a lot to take in, so let's break it down. We are creating a mount by specifying the `--mount` +argument, of `type=bind`, we specify the source file from the host system that we want to share with +our container, and the target file in the container, which does not necessarily need to exist. + +We see that the running ping command outputs into the file on our local system, and even if we delete +the container and remove it, the data persists. + +### Exercise 4 + +- Using a container image of your choice, create a container which has a volume that will contain +the output of the `ps -aux` command inside. +- **NEW** Mount a read-only bind mount into the container which contains an image of your choice. + +## Exercise 5 (wrapping things up) + +- Inspect the source code in [this repository](https://github.com/IPW-CloudOps/simple-node-app) and +create a Dockerfile that builds a container image for that application. +- Run the newly created container image to make sure everything works. + +:::info + +This task is intentionally written ambiguous in order to make you search the official documentation, +ask the course instructors questions and familiarize yourself with what a DevOps engineer has to do +on a day-to-day basis. So do not feel bad if, at first, the task seems hard. Do your best, solve it +at your own pace, collaborate with your colleagues, and, most importantly, have fun while learning +new things! + +::: + +:::note + +This course borrows many things, as well as its structure from: + +- [SCGC Pages UPB](https://scgc.pages.upb.ro/cloud-courses/docs/security/containers) +- [Mobylab Pages UPB](https://mobylab.docs.crescdi.pub.ro/docs/softwareDevelopment/laboratory1/) + +This note is here then to give credits to the teams that created the above resources. For more +information on Docker and other things, feel free to check them out! + +::: diff --git a/docs/cloud_operations/docker/docker-compose.md b/docs/cloud_operations/docker/docker-compose.md new file mode 100644 index 0000000..8b51c1d --- /dev/null +++ b/docs/cloud_operations/docker/docker-compose.md @@ -0,0 +1,217 @@ +--- +id: docker-compose +title: Docker Compose +description: Working with Docker compose +slug: /docker/compose +sidebar_position: 3 +--- + +## Why Docker Compose? + +As we have previously seen in the [Docker Basics](./docker-basics.md) tutorial, we can create and +run containers from the CLI. But this method gets increasingly tedious as soon as we try to create +multiple containers that need to interact with one another. Would it not be nice if we also had +some kind of universal format that could allow us to specify which containers we want, how they +communicate over the network or share data? + +This is where Docker Compose comes into play. It allows a user to write a specification file for +an environment. That specification file is then run by using the `docker compose` command and +Docker takes care of creating all of the required resources. + +## Installation + +On Windows and MacOS, Docker Compose is part of the installation bundle for Docker Desktop. On, +Linux, you can check if you have Docker Compose installed by using the command: + +```bash + +cristian@cristianson:~$ docker compose version +Docker Compose version v2.24.5 + +``` + +If the command fails, you can follow [this guide](https://docs.docker.com/compose/install/) to +install Docker Compose. + +## YAML file format + +The file format we are going to use write Docker Compose files is called +[YAML](https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html) (Yet Another +Markup Language). The format assumes the following concepts: + +- each entry is of type `key:value` +- indentations in YAML are important as indented paragraphs are children of previous paragraphs +- list items are marked with `-` + +## Docker Compose example file + +```yaml + +version: "3.8" +services: + api: + build: . # use a Dockerfile located in the current directory to build the image + environment: # set some environment variables that will be available in the container at runtime + NODE_ENV: development + MY_ENV_VAR: my_custom_value + ports: + - "HOST_PORT:CONTAINER_PORT" + - "5000:80" # bind the 5000 port on the host machine to the port 80 inside the container + networks: + - my-network # the container will be assigned to this network and will be able to + # communicate only with containers which are part of the same network + + postgres: + image: postgres:12 # use the official postgres image, version 12 + secrets: # the secret which is created at the bottom of the file is referenced here + - my_ultra_secret_password + environment: + PGPASSWORD_FILE: /run/secrets/my_ultra_secret_password + volumes: + - my-volume:/var/lib/postgresql/data + - ./init-script/init-db.sql:/docker-entrypoint-init.d/init-db.sql + networks: + - my-network + +volumes: + my-volume: + +networks: + my-network: + +secrets: + my_ultra_secret_password: + file: './who-stores-passwords-in-plain-text.txt' + +``` + +The inspiration for this Docker Compose file is from [here](https://mobylab.docs.crescdi.pub.ro/docs/softwareDevelopment/laboratory2/compose#exemplu-de-fi%C8%99ier-docker-compose). + +Let's break it down line by line: + +- `version`: the set of Docker Compose functionalities that will be used. This is the first line of +each compose file and is **mandatory**. Omitting this line will result in an error. + +- `services`: the containers that will run after the configuration is loaded by the Compose agent. +Each service is basically a container. In the provided file, we have two services/containers, called +**api** and **postgres** + - `build`: the directory where the Dockerfile used for creating the container image is located + - `image`: the image used for running the container + - `ports`: a list of `HOST_MACHINE_PORT: CONTAINER_PORT` entries + - `volumes`: a list of `HOST_VOLUME: PATH_IN_CONTAINER` entries, where `HOST_VOLUME` can be either +a Docker managed volume or a bind mount + - `networks`: a list of assigned networks for the container. + - `secrets`: a list of secrets used inside the container + - `environment`: object with multiple fields of type `ENV_VARIABLE_NAME: ENV_VARIABLE_VALUE` + +:::warning + +The `build` and `image` properties are mutually exclusive! + +::: + +### Volumes + +Top level property (written of the same level as `services`). These are Docker managed objects, and +can have multiple properties, such as the storage driver that should be used or if the volume already +exists on the host machine. More information on volumes [here](https://docs.docker.com/compose/compose-file/07-volumes/). + +```yaml + +volumes: + db-data: + driver: foobar + external: false + +``` + +### Networks + +Top level property (written of the same level as `services`). These are Docker managed objects, and +can have multiple properties, such as the driver that should be used of if the network already exists +on the host machine. More information on networks [here](https://docs.docker.com/compose/compose-file/06-networks/). + +```yaml + +networks: + db-data: + driver: bridge + external: false + +``` + +### Secrets + +Top level property (written of the same level as `services`). These are Docker managed objects, and +can have multiple properties. These are a flavor of [Configs](https://docs.docker.com/compose/compose-file/08-configs/), +focusing on hiding sensitive data. More information on secrets [here](https://docs.docker.com/compose/compose-file/09-secrets/). + +```yaml + +secrets: + server-certificate: + file: ./server.cert + +``` + +`server-certificate` secret is created as `_server-certificate` when the application +is deployed, by registering content of the `server.cert` as a platform secret. + +## List of Docker Compose commands + +A list with all of the important Docker Compose commands is [here](https://docs.docker.com/compose/reference/). +The instructor will perform a demo to showcase the most important ones, but when you do not know +the meaning of a command, do `docker compose --help`. + +```bash + +docker compose up -d # services run in the background, detached from the terminal that initialized them +docker compose up --build # creates images before starting +docker compose start # starts the containers +docker compose pause # pauses the containers of a service (SIGPAUSE is sent) +docker compose unpause # unpauses the containers +docker compose ps # lists active containers +docker compose ls # lists all container stacks +docker compose -p my-project -f my-docker-compose.yml up # uses the specified Compose file instead of the default one and with a project name +docker compose down # stops the containers and deletes them, along with networks, volumes, and images created at up +docker compose rm # deletes all stopped containers (you can specify the name of the container to be deleted at the end) +docker compose rm -s -v # with -s it stops all containers and with -v it also deletes the attached anonymous volumes + +``` + +## Exercise (final boss) + +- Inspect the source code in [this repository](https://github.com/IPW-CloudOps/simple-node-app) and +create a Docker Compose manifest file for that application. +- Check that the required services are up and running. +- Do some requests to test the service. +- Delete the stack + +:::tip + +If you have solved the last exercise in the Docker Basics section, then you should already have the +Dockerfile created. Just reference it in the Docker Compose manifest! + +::: + +:::info + +This task is intentionally written ambiguous in order to make you search the official documentation, +ask the course instructors questions and familiarize yourself with what a DevOps engineer has to do +on a day-to-day basis. So do not feel bad if, at first, the task seems hard. Do your best, solve it +at your own pace, collaborate with your colleagues, and, most importantly, have fun while learning +new things! + +::: + +:::note + +This course borrows many things, as well as its structure from: + +- [SCGC Pages UPB](https://scgc.pages.upb.ro/cloud-courses/docs/security/containers) +- [Mobylab Pages UPB](https://mobylab.docs.crescdi.pub.ro/docs/softwareDevelopment/laboratory1/) + +This note is here then to give credits to the teams that created the above resources. For more +information on Docker and other things, feel free to check them out! + +::: diff --git a/docs/cloud_operations/docker/docker-overview.md b/docs/cloud_operations/docker/docker-overview.md new file mode 100644 index 0000000..64a57b7 --- /dev/null +++ b/docs/cloud_operations/docker/docker-overview.md @@ -0,0 +1,75 @@ +--- +id: docker-overview +title: Introduction and Setup +description: General Overview of Docker and Setup +slug: /docker/overview +sidebar_position: 1 +--- + +## What is Docker? + +Docker is a software containerization platform used to package and run applications both locally +and on cloud systems, eliminating "it works on my machine" problems. It provides an environment for +running containers on any platform, based on containerd. + +Key benefits of Docker include: + +- Faster building, testing, deployment, updating, and error recovery compared to standard +application deployment methods. +- A uniform development and production environment, eliminating compatibility issues with operating +systems and conflicts between library/package versions on the host system. +- Ephemeral containers, meaning the failure or shutdown of one container doesn't crash the entire +system. +- Consistency between development and production environments. +- Maximum flexibility for large-scale projects, allowing easy integration of new software tools as +requirements change. +- Easy replication of infrastructure in different environments using saved Docker images from a +registry. +- Simple updates of components by rewriting images, ensuring the latest versions are always +deployed as containers. + +Docker thus provides a versatile and efficient solution for modern software development and +deployment challenges. + +## Docker architecture + +![Docker architecture](docker-arch.png) + +Above picture taken from [Medium](https://medium.com/@basecs101/understanding-docker-architecture-latest-c7a165571d89). + +## Docker Setup + +- [Linux setup](https://docs.docker.com/engine/install/debian/) +- [Windows setup](https://docs.docker.com/desktop/install/windows-install/) +- [Mac setup](https://docs.docker.com/desktop/install/mac-install/) + +If you get stuck or do not know what steps need to be taken, please ask the course instructors to +help you. It is very important that you have your Docker engine properly set-up as this is the basis +for the Kubernetes tutorial, also. + +## Overview of docker concepts that will be discussed in this course + +- Containers +- Images +- Dockerfile +- Docker Hub +- Docker Compose +- Volumes +- Networks +- Docker Registry +- Container orchestration (basic concept) +- Multi-stage builds +- Docker security best practices +- Docker CLI commands +- Container lifecycle management +- Image layering and caching +- Environment variables in Docker +- Port mapping and exposure +- Docker logging and monitoring +- Container resource management (CPU, memory limits) +- Docker healthchecks +- Docker build context +- Docker image tagging strategies +- Docker networking modes +- Docker storage drivers +- Containerization of different application types (web, database, etc.) \ No newline at end of file diff --git a/docs/cloud_operations/docker/overview.drawio.png b/docs/cloud_operations/docker/overview.drawio.png new file mode 100644 index 0000000..d9cc896 Binary files /dev/null and b/docs/cloud_operations/docker/overview.drawio.png differ diff --git a/docs/cloud_operations/final/_category_.json b/docs/cloud_operations/final/_category_.json new file mode 100644 index 0000000..5a79737 --- /dev/null +++ b/docs/cloud_operations/final/_category_.json @@ -0,0 +1,9 @@ +{ + "label": "Grand Finale", + "position": 4, + "link": { + "type": "generated-index", + "description": "Deplyoment of Your App" + } + } + \ No newline at end of file diff --git a/docs/cloud_operations/final/development-strategy.md b/docs/cloud_operations/final/development-strategy.md new file mode 100644 index 0000000..e02b2f1 --- /dev/null +++ b/docs/cloud_operations/final/development-strategy.md @@ -0,0 +1,15 @@ +--- +id: deployment-strategy +title: Deployment Strategy +description: Best practices for deploying applications on Kubernetes +slug: /deployment/strategy +sidebar_position: 1 +--- + +# Deployment Strategy + +### Deployment Considerations + +### CI/CD Overview + +### Best Practices for Kubernetes Deployments \ No newline at end of file diff --git a/docs/cloud_operations/final/minikube-deployment.md b/docs/cloud_operations/final/minikube-deployment.md new file mode 100644 index 0000000..6a81e54 --- /dev/null +++ b/docs/cloud_operations/final/minikube-deployment.md @@ -0,0 +1,15 @@ +--- +id: minikube-deployment +title: Deploying to Minikube +description: Practical guide to deploying an application on Minikube +slug: /deployment/minikube +sidebar_position: 2 +--- + +# Deploying to Minikube + +### Deploying the Node.js App to Minikube + +### Exposing the Application + +### Scaling and Updating the Deployment \ No newline at end of file diff --git a/docs/cloud_operations/final/next-steps.md b/docs/cloud_operations/final/next-steps.md new file mode 100644 index 0000000..63ff20d --- /dev/null +++ b/docs/cloud_operations/final/next-steps.md @@ -0,0 +1,15 @@ +--- +id: next-steps +title: Next Steps +description: Further learning resources and advanced topics +slug: /next-steps +sidebar_position: 3 +--- + +# Next Steps + +### Further Learning Resources + +### Advanced Topics (Helm, Operators, etc.) + +### Cloud-native Landscape Overview \ No newline at end of file diff --git a/docs/cloud_operations/index.md b/docs/cloud_operations/index.md deleted file mode 100644 index b382a27..0000000 --- a/docs/cloud_operations/index.md +++ /dev/null @@ -1,3 +0,0 @@ -# Intorduction - - diff --git a/docs/cloud_operations/kubernetes/_category_.json b/docs/cloud_operations/kubernetes/_category_.json new file mode 100644 index 0000000..d1c6bd8 --- /dev/null +++ b/docs/cloud_operations/kubernetes/_category_.json @@ -0,0 +1,8 @@ +{ + "label": "Kubernetes", + "position": 3, + "link": { + "type": "generated-index", + "description": "Simple Guide to Kubernetes" + } +} \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/kubernetes-overview.md b/docs/cloud_operations/kubernetes/kubernetes-overview.md new file mode 100644 index 0000000..8f26d03 --- /dev/null +++ b/docs/cloud_operations/kubernetes/kubernetes-overview.md @@ -0,0 +1,79 @@ +--- +id: kubernetes-overview +title: Kubernetes Overview +description: Introduction to Kubernetes +slug: /kubernetes/overview +sidebar_position: 1 +--- + +# Kubernetes Overview + +## What is Kubernetes? + +Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. Key points include: + +- Developed originally by Google, now maintained by the Cloud Native Computing Foundation +- Designed to run distributed systems resiliently +- Provides features like service discovery, load balancing, storage orchestration, and self-healing +- Enables declarative configuration and automation + +## Kubernetes Architecture + +Kubernetes uses a master-worker architecture: + +1. Master Node (Control Plane): + - API Server: Central management point for the cluster (this is the frontend for the cluster) + - Scheduler: Assigns work to nodes + - Controller Manager: Regulates the state of the system + - Cloud Controller Manager: Regulates the state of the system, in case we are using a cloud provider + - etcd: Distributed key-value store for cluster data (mini database) + +2. Worker Nodes: + - Kubelet: Ensures containers are running in a pod + - Container Runtime: Software for running containers (e.g., Docker) + - Kube-proxy: Manages network rules on nodes + +![cluster](/img/docs/kubernetes/1.svg) + +## Other architectures + +### Orchestrated architecture + +![orchestrated-architecture](/img/docs/kubernetes/2.png) + + +## Key Kubernetes Concepts + +1. Clusters: + - A set of nodes that run containerized applications + - Provides high availability and scalability (you can have replicas in different datacenters) + +1. Nodes: + - Physical or virtual machines in the Kubernetes cluster + - Can be master nodes (part of the control plane) or worker nodes + +1. Namespaces: + - Virtual clusters within a physical cluster + - Used for organizing resources and multi-tenancy + +1. [Pods](./resources/pods.md): + - Smallest deployable units in Kubernetes + - Can contain one or more containers + - Share network namespace and storage + +1. [Replica Sets](./resources/replicasets.md): + - Describe the desired state for a set of pods + - Provide basic scaling and self-healing mechanisms + +1. [Services](./resources/services.md): + - An abstract way to expose applications running on pods + - Provide a stable network endpoint + +1. [Persistent Volumes (and Persistent Volume Claims)](./resources/persistent-volumes.md): + - Abstraction for storage resources in the cluster + +1. [ConfigMaps and Secrets](./resources/configmaps_and_secrets.md): + - Manage configuration data and sensitive information + +1. [Ingress](./resources/ingress.md): + - Manage external access to services in a cluster \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/kubernetes-setup.md b/docs/cloud_operations/kubernetes/kubernetes-setup.md new file mode 100644 index 0000000..96b8ffd --- /dev/null +++ b/docs/cloud_operations/kubernetes/kubernetes-setup.md @@ -0,0 +1,89 @@ +--- +id: kubernetes-setup +title: Kubernetes Setup +description: Setting up a local Kubernetes environment +slug: /kubernetes/setup +sidebar_position: 2 +--- + +# Kubernetes Setup + +## Installing Minikube + + https://minikube.sigs.k8s.io/docs/start + +## Installing kubectl + + https://kubernetes.io/docs/tasks/tools/ + + Don't forget autocompletion! + +## Setting up a Local Kubernetes Cluster + +1. Start Minikube: + + ```shell + > minikube start + ``` + + :::warning + + If you have `VirtualBox` or `VMWare`(?), minikube defaults to using them over docker. This might be fine on your machine, but for some this operation will fail due to some configuration issues inside the BIOS. The simple solution is to tell minikube to use `docker` as the driver. + + ```shell + > minikube start --driver=docker + ``` + + ::: + +2. Verify the cluster status: + + ```shell + > minikube status + > kubectl cluster-info + ``` + +3. Enable necessary addons: + + ```shell + > minikube addons enable ingress + > minikube addons enable dashboard + ``` + +4. Access the Kubernetes dashboard: + + ```shell + > minikube dashboard + ``` + +5. Deploy a sample application: + + ```shell + > kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.10 + > kubectl expose deployment hello-minikube --type=NodePort --port=8080 + ``` + +6. Access the deployed application: + + ```shell + > minikube service hello-minikube + ``` + + A browser should have launched showing you some information about the sample application. Optionally, you can play with the app using `curl`: + + ```shell + > curl 127.0.0.1: -d "edit me" + ``` + +7. Clean up: + + ```shell + > kubectl delete service hello-minikube + > kubectl delete deployment hello-minikube + ``` + +8. Stop the Minikube cluster: + + ```shell + > minikube stop + ``` \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/_category_.json b/docs/cloud_operations/kubernetes/resources/_category_.json new file mode 100644 index 0000000..a220d77 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/_category_.json @@ -0,0 +1,8 @@ +{ + "position": 5, + "label": "Resources", + "link": { + "type": "generated-index", + "title": "Resources" + } +} \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/condition.md b/docs/cloud_operations/kubernetes/resources/condition.md new file mode 100644 index 0000000..82fc9ab --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/condition.md @@ -0,0 +1,18 @@ +--- +title: Condition +description: '' +unlisted: true +--- + +# [Condition](https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/object-meta/) + +Volume represents a named volume in a pod that may be accessed by any container in the Pod. + +- name: `string` - required +- persistentVolumeClaim: + - claimName: `string` - required + - readOnly: `boolean` +- configMap: + - name: `string` +- secret: + - secretName: `string` \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/configmaps_and_secrets.md b/docs/cloud_operations/kubernetes/resources/configmaps_and_secrets.md new file mode 100644 index 0000000..2801235 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/configmaps_and_secrets.md @@ -0,0 +1,286 @@ +--- +title: ConfigMaps and Secrets +description: '' +sidebar_position: 3 +--- + +# [ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/) and [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) + +### Goals +1. **Understand ConfigMaps and Secrets**: + - Learn what ConfigMaps and Secrets are, and their roles in Kubernetes. +1. **Basic Examples**: + - Provide basic examples of ConfigMaps and Secrets. + - Show how to define ConfigMaps and Secrets using YAML. +1. **Usage in Pods**: + - Demonstrate how to use ConfigMaps and Secrets to manage configuration and sensitive data within pods. +1. **Exercise**: + - Hands-on activities to create and use ConfigMaps and Secrets in a Kubernetes cluster. + +## Understanding ConfigMaps + +**ConfigMaps** are Kubernetes objects used to store non-confidential data in key-value pairs. They can be used to configure application settings and pass configuration data into pods. + +### Basic ConfigMap Example + +Here is a basic example of a Kubernetes ConfigMap specification in YAML: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: example-configmap +data: + config.json: | + { + "key1": "value1", + "key2": "value2" + } +``` + +Save this to a file named `configmap.yaml` and create the ConfigMap using: + +```bash +> kubectl apply -f configmap.yaml +``` + +You can view all config maps and read their contents using: + +``` +> kubectl get configmaps +NAME DATA AGE +example-configmap 1 54s +kube-root-ca.crt 1 46m + +> kubectl describe configmaps example-configmap +Name: example-configmap +Namespace: default +Labels: +Annotations: + +Data +==== +config.json: +---- +{ + "key1": "value1", + "key2": "value2" +} + + +BinaryData +==== + +Events: +``` + +### Using ConfigMaps in Pods + +To use a ConfigMap in a pod, you can reference it in the pod specification. For example: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: configmap-pod +spec: + containers: + - name: app-container + image: nginx:latest + volumeMounts: + - name: config-volume + mountPath: /etc/config + volumes: + - name: config-volume + configMap: + name: example-configmap +``` + +In this example, the ConfigMap `example-configmap` is mounted as a volume in the pod at `/etc/config`. + +After you `apply` the pod, you can see the config by `cat`-ing the file contents: + +```bash +> kubectl exec -it configmap-pod -- cat /etc/config/config.json +``` + +## Understanding Secrets + +**Secrets** are Kubernetes objects designed to hold sensitive data, such as passwords, OAuth tokens, and SSH keys. The data in Secrets is encoded as base64 strings. + +### Basic Secret Example + +Here is a basic example of a Kubernetes Secret specification in YAML: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: example-secret +data: + username: YWRtaW4= + password: MWYyZDFlMmU2N2Rm +``` + +The values for `username` and `password` are base64-encoded strings (`admin` and `1f2d1e2e67df`, respectively). + +Save this to a file named `secret.yaml` and create the Secret: + +```bash +> kubectl apply -f secret.yaml +``` + +### Using Secrets in Pods + +To use a Secret in a pod, you can reference it similarly to a ConfigMap: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: secret-pod +spec: + containers: + - name: app-container + image: nginx:latest + env: + - name: USERNAME + valueFrom: + secretKeyRef: + name: example-secret + key: username + - name: PASSWORD + valueFrom: + secretKeyRef: + name: example-secret + key: password +``` + +In this example, the Secret `example-secret` is used to set environment variables in the pod. + +## Exercises + +### Exercise 1: Creating and Using a ConfigMap (Easy) + +**Objective**: Create a ConfigMap and use it to configure a pod. + +**Task**: +- Create a ConfigMap with a configuration file. +- Use this ConfigMap in a pod. + +**Instructions**: +1. Create a YAML file named `my-configmap.yaml`: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: my-configmap +data: + app.properties: | + setting1=value1 + setting2=value2 +``` + +2. Apply the ConfigMap: + +```bash +> kubectl apply -f my-configmap.yaml +``` + +3. Create a YAML file named `configmap-pod.yaml`: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: configmap-pod +spec: + containers: + - name: app-container + image: busybox + command: ["sh", "-c", "cat /etc/config/app.properties && sleep 3600"] + volumeMounts: + - name: config-volume + mountPath: /etc/config + volumes: + - name: config-volume + configMap: + name: my-configmap +``` + +4. Apply the pod configuration: + +```bash +> kubectl apply -f configmap-pod.yaml +``` + +5. Verify the pod is running and inspect the logs to see the ConfigMap data: + +```bash +> kubectl logs configmap-pod +``` + +--- + +### Exercise 2: Creating and Using a Secret (Medium) + +**Objective**: Create a Secret and use it in a pod. + +**Task**: +- Create a Secret containing sensitive information. +- Use this Secret in a pod to set environment variables. + +**Instructions**: +1. Create a YAML file named `my-secret.yaml`: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: my-secret +data: + api-key: c2VjcmV0YXBpa2V5 +``` + +(The value `c2VjcmV0YXBpa2V5` is base64 for `secretapikey`.) + +2. Apply the Secret: + +```bash +> kubectl apply -f my-secret.yaml +``` + +3. Create a YAML file named `secret-pod.yaml`: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: secret-pod +spec: + containers: + - name: app-container + image: busybox + command: ["sh", "-c", "echo $API_KEY && sleep 3600"] + env: + - name: API_KEY + valueFrom: + secretKeyRef: + name: my-secret + key: api-key +``` + +4. Apply the pod configuration: + +```bash +> kubectl apply -f secret-pod.yaml +``` + +5. Verify the pod is running and inspect the logs to see the API key: + +```bash +> kubectl logs secret-pod +``` + +These exercises cover the basics of creating and using ConfigMaps and Secrets in Kubernetes, providing both practical examples and explanations. diff --git a/docs/cloud_operations/kubernetes/resources/ingress.md b/docs/cloud_operations/kubernetes/resources/ingress.md new file mode 100644 index 0000000..d4d7969 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/ingress.md @@ -0,0 +1,94 @@ +--- +title: Ingress +description: '' +sidebar_position: 5 +--- +# [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) + +## Goals +1. **Understand Kubernetes Ingress**: + - Learn what an Ingress is and its role in the Kubernetes architecture. +1. **Basic Example(s)**: + - Provide (a) basic example(s) of Ingress resources. + - Show how to define an Ingress using YAML. +1. **Ingress Spec Explanation**: + - Explain the components of the Ingress specification using the example(s) above. +1. **Ingress Controllers**: + - Describe different Ingress controllers and their use cases. +1. **Exercise**: + - Hands-on activity to create a simple Ingress. + +## Understanding Kubernetes Ingress + +An **Ingress** in Kubernetes is an API object that manages external access to the services in a cluster, typically HTTP. Ingress can provide load balancing, SSL termination, and name-based virtual hosting. + +## Basic Ingress Example + +### Simple Ingress + +Here is a basic example of a Kubernetes Ingress specification in YAML: + +``` +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: example-ingress +spec: + rules: + - host: example.com + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: example-service + port: + number: 80 +``` + +This Ingress routes traffic to the `example-service` service when accessing `example.com`. + +Save this to a file `ingress.yaml` and create the resource using `kubectl apply -f ingress.yaml`: + +``` +> kubectl apply -f ingress.yaml +ingress.networking.k8s.io/example-ingress created +``` + +## Ingress Spec Explanation + +1. apiVersion: networking.k8s.io/v1 (API group and version) +1. kind: Ingress (type of the resource) +1. metadata: [ObjectMeta](./object-meta.md) +1. spec: [IngressSpec](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#ingressspec-v1-networking-k8s-io) +1. status: [IngressStatus](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#ingressstatus-v1-networking-k8s-io) + +## Ingress Controllers + +- **Nginx Ingress Controller**: A popular choice that uses Nginx as the backend for managing Ingress resources. (Easiest to set up) +- **Traefik**: A dynamic and modern HTTP reverse proxy and load balancer. +- **Contour**: An open-source ingress controller for Kubernetes that provides advanced routing features. +- **Istio**: A service mesh that provides a powerful way to manage ingress and other traffic within a Kubernetes cluster. + +## Exercises + +### Exercise 1: Create a Simple Ingress + +**Objective**: Deploy an application and expose it externally using an Ingress resource. + +**Task**: +- Deploy a pod running a simple HTTP server (e.g., `nginx` or `httpd`). +- Create a Service to expose this HTTP server. +- Define an Ingress resource to route external traffic to the service. +- Verify that the service is accessible via the Ingress. + +:::tip + +Make sure your cluster has an Ingress controller installed. If using Minikube, you can enable the Nginx Ingress controller using: + +``` +minikube addons enable ingress +``` + +::: \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/object-meta.md b/docs/cloud_operations/kubernetes/resources/object-meta.md new file mode 100644 index 0000000..1b99bee --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/object-meta.md @@ -0,0 +1,34 @@ +--- +title: ObjectMeta +description: '' +unlisted: true +--- +# [ObjectMeta](https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/object-meta/) + +The `ObjectMeta` object is used for metadata. It includes fields that provide context and identity for resources. Basically, it allows for identification of resources. + +Some of the most important attributes of the `ObjectMeta` object are: + +1. **name**: + - type: `string` + - description: the name of the object (unique within the namespace) + +1. **namespace**: + - type: `string` + - description: the namespace in which the object resides + +1. **labels** + - type: `map` + - description: key-value pairs that can be used to organize and select objects - often used for grouping, searching and managing sets of objects (i.e pods, services, pvs belonging to a specific api version) + +1. **annotations**: + - type: `map` + - description: key-value pairs used to store arbitrary metadata - can be used to attach non-identifying metadata to objects (i.e description of what a Pod does, storing external IDs of a resource, storing configuration hints - but also [Ingress annotations](https://kubernetes.io/docs/concepts/services-networking/ingress/#the-ingress-resource)) + +1. **uid**: + - type: `string` + - description: unique identifier, set by Kubernetes (unique across the cluster) + +1. **creationTimestamp**: + - type: `string` + - decsription: time at which the object was created, set by Kubernetes \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/persistent-volumes.md b/docs/cloud_operations/kubernetes/resources/persistent-volumes.md new file mode 100644 index 0000000..b0c023a --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/persistent-volumes.md @@ -0,0 +1,100 @@ +--- +title: Persistent Volumes +description: '' +sidebar_position: 7 +--- + +# [Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + +## Goals + +1. **Understand Kubernetes Persistent Volumes**: + - Learn what a Persistent Volume (PV) is and its role in Kubernetes storage architecture. +1. **Basic Example(s)**: + - Provide (a) basic example(s) of Persistent Volume and Persistent Volume Claim (PVC) resources. + - Show how to define a PV and PVC using YAML. +1. **Persistent Volume Spec Explanation**: + - Explain the components of the Persistent Volume specification using the example(s) above. +1. **Persistent Volume Claims**: + - Describe the relationship between Persistent Volumes and Persistent Volume Claims. +1. **Exercise**: + - Hands-on activity to create and use a Persistent Volume. + +## Understanding Kubernetes Persistent Volumes + +A **Persistent Volume** (PV) in Kubernetes is a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using Storage Classes. PVs are independent of the lifecycle of a pod and provide a way to persist data beyond the lifespan of individual pods. + +## Basic Persistent Volume Example + +### Simple Persistent Volume and Persistent Volume Claim + +Here is a basic example of a Kubernetes Persistent Volume and Persistent Volume Claim in YAML: + +```yaml +apiVersion: v1 +kind: PersistentVolume +metadata: + name: example-pv +spec: + capacity: + storage: 1Gi + accessModes: + - ReadWriteOnce + persistentVolumeReclaimPolicy: Retain + storageClassName: manual + hostPath: + path: /mnt/data +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: example-pvc +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: manual +``` + +This example defines a 1Gi PV using a hostPath and a PVC requesting 1Gi of storage. + +Save this to a file `pv-pvc.yaml` and create the resources using: + +```sh +> kubectl apply -f pv-pvc.yaml +persistentvolume/example-pv created +persistentvolumeclaim/example-pvc created +``` + +## Persistent Volume Spec Explanation + +1. **apiVersion**: v1 (API group and version) +1. **kind**: PersistentVolume (type of the resource) +1. **metadata**: [ObjectMeta](./object-meta.md) +1. **spec**: [PersistentVolumeSpec](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#persistentvolumespec-v1-core) +1. **status**: [PersistentVolumeStatus](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#persistentvolumestatus-v1-core) + +## Persistent Volume Claims + +- A **Persistent Volume Claim (PVC)** is a request for storage by a user. It is similar to a pod in that pods consume node resources, and PVCs consume PV resources. +- PVCs can request specific size and access modes (e.g., ReadWriteOnce, ReadOnlyMany). + +## Exercises + +### Exercise 1: Create a Simple Persistent Volume + +**Objective**: Deploy an application that uses a Persistent Volume for storage. + +**Task**: +- Define a Persistent Volume with a suitable storage capacity. +- Create a Persistent Volume Claim to request storage. +- Deploy a pod that uses the PVC for persistent storage. +- Verify that the pod can write and read data from the Persistent Volume. + +:::tip + +Make sure your cluster has a storage provider configured. For local testing, using a hostPath as shown in the example above is a simple way to simulate PV functionality. + +::: diff --git a/docs/cloud_operations/kubernetes/resources/pod/_category_.json b/docs/cloud_operations/kubernetes/resources/pod/_category_.json new file mode 100644 index 0000000..65db197 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/pod/_category_.json @@ -0,0 +1,11 @@ +{ + "label": "Pods", + "position": 2, + "link": { + "type": "generated-index", + "title": "Pods" + }, + "customProps": { + "unlisted": true + } +} \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/pod/container.md b/docs/cloud_operations/kubernetes/resources/pod/container.md new file mode 100644 index 0000000..65a96d5 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/pod/container.md @@ -0,0 +1,49 @@ +--- +title: Container +unlisted: true +description: '' +--- + +# [Container](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#Container) + +- name: `string` - required +- image: `string` +- imagePullPolicy: `"Always" | "Never" | "IfNotPresent"`; Default: `"Always"` if :latest tag is specified, or "IfNotPresent" otherwise. (Can't be updated) +- command: `Array` (entrypoint) +- args: `Array` (entrypoint) +- workingDir: `string` (entrypoint) +- ports: `Array` + - containerPort: `int32` - required (port to expose on the Pod's IP address) + - hostIP: `string` (host IP to bind the external port to) + - hostPort: `int32` (port to expose on the host - most containers don't need this) + - name: `string` + - protocol: `"UDP" | "TCP" | "SCTP"`; Default: `"TCP"` +- env: `Array` + - name: `string` - required + - value: `string` + - valueFrom: + - configMapKeyRef: + - key: `string` - required + - name: `string` + - optional: `boolean` + - fieldRef: + - fieldPath: `metadata.name | metadata.namespace | metadata.labels['\'] | metadata.annotations['\'] | spec.nodeName | spec.serviceAccountName | status.hostIP | status.podIP | status.podIPs` - required - `string` + - apiVersion: `string` - Default: `"v1"` +- envFrom: `Array` + - configMapRef: + - name: `string` + - optional: `boolean` + - prefix: `string` +- volumeMounts: `Array` + - mountPath: `string` - required + - name: `string` - required + - mountPropagation: `string` + - readOnly: `boolean` + - subPath: `string` +- resources: `Array` + - claims: `Array` + - name: `string` - required + - limits: `map` + - requests: `map` +- livenessProbe: [`Probe`](./probe.md) +- readinessProbe: [`Probe`](./probe.md) \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/pod/pod-spec.md b/docs/cloud_operations/kubernetes/resources/pod/pod-spec.md new file mode 100644 index 0000000..c581628 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/pod/pod-spec.md @@ -0,0 +1,12 @@ +--- +title: Pod Spec +description: '' +unlisted: true +--- + +# Pod Spec + +Here's the most important attributes: + +1. containers - [`Array`](./container.md) +1. volumes - [`Array`](./volume.md) \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/pod/pod-status.md b/docs/cloud_operations/kubernetes/resources/pod/pod-status.md new file mode 100644 index 0000000..818a5ac --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/pod/pod-status.md @@ -0,0 +1,67 @@ +--- +title: PodStatus +description: '' +unlisted: true +--- + +# [PodStatus](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodStatus) + +The PodStatus represents the information about the status that a Pod is in. Status may TRAIL the actual state of a system, especially if the node that hosts the pod cannot contact the control plane. + +Here, we remind the most important: + +- phase: + - type: `string` - `"Pending" | "Running" | "Succeeded" | "Failed" | "Unknown"` + - description: summary of the pod lifecycle phase: + - `Pending`: pod has been accepted by kubernetes, but one or more containers have not been started + - `Running`: pod has been placed on a node, all containers have been created and at least one container is running, starting or restarting + - `Succeeded`: all containers in the pod have been terminated successfully and will not be restarted + :::info + + While our main focus is on applications (well, containers and pods) that run indefinitely, there can be pods that run once + + ::: + - `Failed`: all containers in a pod have terminated, and at least one container terminated in a failure + :::warning + + The `Failed` status appears, most of the time, when a pod has been restarted too frequently and has failed too frequently, as well + + ::: + - `Unknown`: the state of the pod could not be obtained. This usually happens because there is trouble communicated with the node where the pod should be running + +- conditions: + - type: [`Array`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/) + - description: list of current observed conditions of the pod: + - `Initialized`: all init containers have started successfully + - `Ready`: the pod is able to serve requests and all containers are ready + - `ContainersReady`: all containers in the pod are ready + - `PodScheduled`: the pod has been scheduled to a node + +- hostIP: + - type: `string` + - description: the IP address of the node where the pod is running + +- podIP: + - type: `string` + - description: the IP address assigned to the pod + +- startTime: + - type: `string (timestamp)` + - description: the time when the pod was acknowledged by the Kubernetes system + +- containerStatuses: + - type: [`Array`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/) + - description: detailed information about the status of each container in the pod, including: + - `name`: the name of the container + - `state`: the current state of the container (`waiting`, `running`, or `terminated`) + - `lastState`: details about the last terminated state of the container, if applicable + - `ready`: a boolean indicating if the container is ready to serve requests + - `restartCount`: the number of times the container has been restarted + +- message: + - type: `string` + - description: a human-readable message indicating details about why the pod is in its current condition + +- reason: + - type: `string` + - description: a brief CamelCase message indicating why the pod is in its current state diff --git a/docs/cloud_operations/kubernetes/resources/pod/probe.md b/docs/cloud_operations/kubernetes/resources/pod/probe.md new file mode 100644 index 0000000..45d9185 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/pod/probe.md @@ -0,0 +1,18 @@ +--- +title: Probe +description: '' +unlisted: true +--- + +# [Probe](ttps://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#Probe) + +Probes are used to describe health checks to be performed against a container to determine whether it is alive or ready to receive traffic. + +- exec: + - command: `Array` +- httpGet: + - port: `integer | string` - required + - host: `string` - defaults to Pod IP + - path: `string` + - scheme: `string` - defaults to 'HTTP' +- initialDelaySeconds: `int32` diff --git a/docs/cloud_operations/kubernetes/resources/pod/volume.md b/docs/cloud_operations/kubernetes/resources/pod/volume.md new file mode 100644 index 0000000..3beb325 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/pod/volume.md @@ -0,0 +1,18 @@ +--- +title: Volume +description: '' +unlisted: true +--- + +# [Volume](https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/object-meta/) + +Volume represents a named volume in a pod that may be accessed by any container in the Pod. + +- name: `string` - required +- persistentVolumeClaim: + - claimName: `string` - required + - readOnly: `boolean` +- configMap: + - name: `string` +- secret: + - secretName: `string` \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/pods.md b/docs/cloud_operations/kubernetes/resources/pods.md new file mode 100644 index 0000000..8eea7ee --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/pods.md @@ -0,0 +1,233 @@ +--- +title: Pods +description: '' +sidebar_position: 1 +--- +# [Pods](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/) + +## Goals +1. **Understand Kubernetes Pods**: + - Learn what a pod is and its role in the Kubernetes architecture. +1. **Basic Example(s)**: + - Provide (a) basic example(s) of Pods. + - Show how to define a Pod using YAML. +1. **Pod Spec Explanation**: + - Visit the Pod specification while following the example(s) above. +1. **Direct Pod Access**: + - Describe how to create a direct network tunnel to a Pod for testing and/or debugging purposes. +1. **Exercise**: + - Hands-on activity to create a simple Pod (using only the spec). + +## Understanding Kubernetes Pods: + +A **Pod** is the smallest, most basic deployable object in Kubernetes. It represents a single instance of a running process in your cluster. A pod encapsulates one or more containers, storage resources, a unique network IP, and options that govern how the containers should run. + +## Basic Pod Example + +Here is a basic example of a Kubernetes pod specification in YAML: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: nginx-pod +spec: + containers: + - name: nginx-container + image: nginx:1.21 + ports: + - containerPort: 80 +``` + +Save this to a file `pod.yaml` and create the resource using `kubectl apply -f pod.yaml`: + +```bash +> kubectl apply -f pod.yaml +pod/nginx-pod created +``` + +Next, let's take a look at the pods running in our cluster and see if we can find the newly created Pod using `kubectl get pods`: + +```bash +> kubectl get pods +NAME READY STATUS RESTARTS AGE +nginx-pod 1/1 Running 0 72s +``` + +Let's explain this a little: +1. `NAME`: As we can see on the left side we have the name of the Pod we've just created (remember the `metadata.name`?) +1. `READY`: To the right of it, we have the `READY` section: `1/1` means that `1` out of `1` Pods of this type are ready to serve traffic. But do we really need this? Well, yes, the maximum number of Pods, for any name, is `1`, but what if our Pod is not ready? We should know that when retrieving Pods, so we can know that the reason we are having trouble with our app is cause the Pod is literally not in a `Running` state. Usually, we won't launch just 1 Pod of this type, we'll be using a Set resource to help us deploy multiple and manage them, but then the `metadata.name` provided will be the prefix for the final name of the pod (i.e `nginx-pod-134af`) +1. `STATUS`: Next, we have the `STATUS` section, which lets us know what status our Pod is in [(`"Pending" | "Running" | "Succeeded" | "Failed" | "Unknown"`)](./pod/pod-status.md) +1. `RESTARTS`: `RESTARTS` show us how many times has the Pod restarted since it has been created, along with how much time has it passed since the last restart (if there has been any) +1. `AGE`: lastly, the `AGE`, which shows us how old the Pod is + +After creating the resource, we should also take a look at the state it is in, we can do this by making use of the `kubectl describe pods/nginx-pod`: + +```bash +> kubectl describe pods/nginx-pod +Name: nginx-pod +Namespace: default +Priority: 0 +Service Account: default +Node: minikube/192.168.49.2 +Start Time: Mon, 05 Aug 2024 12:07:28 +0300 +Labels: +Annotations: +Status: Running +IP: 10.244.0.45 +IPs: + IP: 10.244.0.45 +Containers: + nginx-container: + Container ID: docker://7b219a363f96823719c9f68f0e0e5106007808c6d4b840d1521ab1383606e99b + Image: nginx:1.21 + Image ID: docker-pullable://nginx@sha256:2bcabc23b45489fb0885d69a06ba1d648aeda973fae7bb981bafbb884165e514 + Port: 80/TCP + Host Port: 0/TCP + State: Running + Started: Mon, 05 Aug 2024 12:07:39 +0300 + Ready: True + Restart Count: 0 + Environment: + Mounts: + /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-kwl7s (ro) +Conditions: + Type Status + PodReadyToStartContainers True + Initialized True + Ready True + ContainersReady True + PodScheduled True +Volumes: + kube-api-access-kwl7s: + Type: Projected (a volume that contains injected data from multiple sources) + TokenExpirationSeconds: 3607 + ConfigMapName: kube-root-ca.crt + ConfigMapOptional: + DownwardAPI: true +QoS Class: BestEffort +Node-Selectors: +Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s + node.kubernetes.io/unreachable:NoExecute op=Exists for 300s +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Scheduled 25m default-scheduler Successfully assigned default/nginx-pod to minikube + Normal Pulling 25m kubelet Pulling image "nginx:1.21" + Normal Pulled 25m kubelet Successfully pulled image "nginx:1.21" in 9.687s (9.687s including waiting). Image size: 141526528 bytes. + Normal Created 25m kubelet Created container nginx-container + Normal Started 25m kubelet Started container nginx-container +``` + +A lot of information, I know, but bare with me, we can digest most of it, but we don't even need to. +The `describe` function, though, lets us see exactly why stuff went wrong, went it did, well that's what it's mostly used for, or making sure nothing did, in fact, went wrong, so we can go in search of the real culprit that is giving us headaches. + +What is of most interest are the next things: +- containers' ports +- containers' ready state +- conditions +- events + +If we know these things, we can see pretty quick (at least for our use cases) if the Pod is set up properly. While most of the data is pretty self explanatory, emphasis should be put on the `Events` section, which gives us a detailed trace of each step that happened. If a problem did happen, you'd be pretty sure it is found here. + +## Explanation of the [Pod Spec](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/): + +1. apiVersion: v1 (hardcoded) +1. kind: Pod (type of resource) +1. metadata: [ObjectMeta](./object-meta.md) +1. spec: [PodSpec](./pod/pod-spec.md) +1. status: [PodStatus](./pod/pod-status.md) + +## Direct Pod Access + +Great, we've deployed our first resource, a Pod! But what can we do with it? Can we access it? Well, usually, you'd want to put it behind a Service (so that it can be accessed from a hostname), that would also be behind an Ingress (so that outside traffic can enter the cluster). + +Since we haven't done any of those, we'll create a direct tunnel to the Pod: + +```bash +> kubectl port-forward nginx-pod 8000:80 +Forwarding from 127.0.0.1:8000 -> 80 +Forwarding from [::1]:8000 -> 80 +``` + +With this simple command (don't close the terminal!!!), our machine's port `8000` is made so it proxies traffic to the port `80` of our container + +:::warn + +If you are using WSL2, the things aren't that simple. WSL2 has a different network interface than Windows, which means you need to start your browser from WSL2 (usually install it in WSL2 again, cause it's basically another operating system) or forward traffic from Windows to WSL2. + +::: + +Now to test it and make sure it works, we can do a simple `curl http://localhost:8000` command in the terminal: + +```bash +> curl http://localhost:8000 + + + +Welcome to nginx! + + + +

Welcome to nginx!

+

If you see this page, the nginx web server is successfully installed and +working. Further configuration is required.

+ +

For online documentation and support please refer to +nginx.org.
+Commercial support is available at +nginx.com.

+ +

Thank you for using nginx.

+ + +``` + +And just like that we got to have our first Pod, create a tunnel to it and see that it works! + + +## Exercises: + +1. Write a YAML file named `multi-container-pod.yaml` that defines a Pod having 2 containers: a container named `nginx-container`, but on port `3000`, that is the same as the one provided in the example, and another container named `busybox-container` that uses the `busybox` image and that provides it the next command: `/bin/sh -c echo BusyBox Hello! && sleep 3600' + + :::tip + + It is usually recommended to split commands into multiple parts as follows: first being the exectuable run (in our case `/bin/sh`) and aftewards each argument by itself (`-c`, `echo BusyBox Hello! && sleep 3600`). + + So in the end it looks like: `['/bin/sh', '-c', 'echo BusyBox Hello! && sleep 3600']` when you pass it to the container + + ::: + + Create the resource using `kubectl apply`, create a tunnel to send a request and make sure it works when you request the webpage, and also check the logs of the Pod using the `kubectl logs` command + +1. Write a YAML file named `resource-limited-pod.yaml` to define a pod with one container, `resource-limited-container`, using the `busybox` image, the `['/bin/sh', '-c', 'while true; do echo Hello Kubernetes!; sleep 5; done']` command and with the next resources: + - Resource requests: + - CPU: `100m` + - Memory: `64Mi` + - Resource limits: + - CPU: `200m` + - Memory: `128Mi` + + Create the Pod and monitor its usage using the `kubectl top` command + + Try running a resource-intensive command `yes > /dev/null` and observe the behavior of the Pod under resource limits + + :::tip + + When using minikube, make sure you enable the `metrics-server`: `> minikube addons enable metrics-server` + + ::: + + :::tip + + To run commands inside Pods, you can make use of the `kubectl exec` command as follows: + + ```bash + > kubectl exec -it __POD_NAME__ -- /bin/sh -c "echo Hello" + ``` + + ::: \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/replicasets.md b/docs/cloud_operations/kubernetes/resources/replicasets.md new file mode 100644 index 0000000..9fca9b9 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/replicasets.md @@ -0,0 +1,112 @@ +--- +title: ReplicaSets +description: '' +sidebar_position: 6 +--- + +# [ReplicaSets](https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/) + +## Goals +1. **Understand Kubernetes ReplicaSets**: + - Learn what a ReplicaSet is and its role in the Kubernetes architecture. +1. **Advantages of Using ReplicaSets vs. Simple Pods** + - Figure out why you should be almost always be using a ReplicaSet in production instead of a simple Pod. +1. **Basic Example(s)**: + - Provide (a) basic example(s) of ReplicaSet resources. + - Show how to define a ReplicaSet using YAML. +1. **ReplicaSet Spec Explanation**: + - Explain the components of the ReplicaSet specification using the example(s) above. +1. **Exercise**: + - Hands-on activity to create a simple ReplicaSet. + +## Understanding Kubernetes ReplicaSets + +A **ReplicaSet** in Kubernetes is a controller that ensures a specified number of pod replicas are running at any given time. If a pod goes down, the ReplicaSet will automatically create a new one to maintain the desired number of replicas. + +## Advantages of Using ReplicaSets vs. Simple Pods + +1. **Automatic Scaling**: + - ReplicaSets automatically create new Pods to replace any that fail, ensuring the desired number of Pods are always running. + +2. **Declarative Management**: + - You can define the desired state (e.g., number of replicas) in a YAML file, and Kubernetes maintains that state automatically. + +3. **Rolling Updates**: + - ReplicaSets, when used with Deployments, allow you to update your application with zero downtime by gradually replacing old Pods with new ones. + +4. **Consistency and Reliability**: + - ReplicaSets ensure that your application consistently has the necessary resources to handle its workload by maintaining a fixed number of replicas. + +5. **Label Selection**: + - ReplicaSets use labels to manage Pods, allowing you to adjust which Pods are managed by the ReplicaSet through label changes. + +6. **Simplified Management**: + - Instead of managing individual Pods, ReplicaSets provide a single point of control for a group of Pods, making operations like scaling and updates easier. + +## Basic ReplicaSet Example + +### Simple ReplicaSet + +Here is a basic example of a Kubernetes ReplicaSet specification in YAML: + +```yaml +apiVersion: apps/v1 +kind: ReplicaSet +metadata: + name: example-replicaset +spec: + replicas: 3 + selector: + matchLabels: + app: example-app + template: + metadata: + labels: + app: example-app + spec: + containers: + - name: nginx + image: nginx:1.21.6 + ports: + - containerPort: 80 +``` + +This ReplicaSet ensures that three replicas of the `nginx` container are running at all times. + +Save this to a file `replicaset.yaml` and create the resource using `kubectl apply -f replicaset.yaml`: + +```sh +> kubectl apply -f replicaset.yaml +replicaset.apps/example-replicaset created +``` + +## ReplicaSet Spec Explanation + +1. **apiVersion**: apps/v1 (API group and version) +1. **kind**: ReplicaSet (type of the resource) +1. **metadata**: [ObjectMeta](./object-meta.md) +1. **spec**: [ReplicaSetSpec](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#replicasetspec-v1-apps) + - **replicas**: The desired number of pod replicas. + - **selector**: The label selector used to identify the pods managed by the ReplicaSet. + - **template**: The pod template that defines the pods to be created. + +## Exercises + +### Exercise 1: Create a Simple ReplicaSet + +**Objective**: Deploy a ReplicaSet that ensures a specific number of pods are running. + +**Task**: +- Define a ReplicaSet that runs three replicas of an `nginx` container. +- Verify that the desired number of replicas are running. +- Test scaling by increasing or decreasing the number of replicas in the ReplicaSet definition. + +:::tip + +If you want to scale the ReplicaSet, you can edit the `replicas` field in the YAML file and reapply it or use the following command: + +```sh +kubectl scale replicaset example-replicaset --replicas=5 +``` + +::: diff --git a/docs/cloud_operations/kubernetes/resources/service/_category_.json b/docs/cloud_operations/kubernetes/resources/service/_category_.json new file mode 100644 index 0000000..3c864f5 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/service/_category_.json @@ -0,0 +1,11 @@ +{ + "label": "Services", + "position": 4, + "link": { + "type": "generated-index", + "title": "Services" + }, + "customProps": { + "unlisted": true + } +} \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/service/service-port.md b/docs/cloud_operations/kubernetes/resources/service/service-port.md new file mode 100644 index 0000000..9861b32 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/service/service-port.md @@ -0,0 +1,28 @@ +--- +title: Service Port +description: "" +unlisted: true +--- + +# [Service Port](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#serviceport-v1-core) + +Here's the most important attributes: + +1. name - `string` + - The name of this port within the service, must be a `DNS_LABEL`. All ports within a `ServiceSpec` must have unique names. + - Optional if only one `ServicePort` is defined. + +1. nodePort - `integer` + - The port on each node on which this service is exposed when `type` is `NodePort` or `LoadBalancer`. Usually assigned by the system. + - If specified, the value must be in-range and not in use, otherwise the operation will fail. + - If not specified, a port will be allocated if required. + - This field is cleared when updating a Service to no longer need it (e.g., changing `type` from `NodePort` to `ClusterIP`). + +1. port - `integer` + - The port that will be exposed by this service. + +1. targetPort - `integer | string` + - The port number or name to access on the pods targeted by the service. Number must be in the range 1 to 65535. + - Name must be an `IANA_SVC_NAME`. If specified as a string, it will be looked up as a named port in the target Pod's container ports. + - If not specified, the value of the `port` field is used. + - This field is ignored for services with `clusterIP=None`, and should be omitted or set equal to the `port` field. \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/service/service-spec.md b/docs/cloud_operations/kubernetes/resources/service/service-spec.md new file mode 100644 index 0000000..568b976 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/service/service-spec.md @@ -0,0 +1,17 @@ +--- +title: Service Spec +description: "" +unlisted: true +--- + +# [Service Spec](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#servicespec-v1-core) + +Here's the most important attributes: + +1. ports - [`Array`](./service-port.md) +1. selector - `Object` + - Route service traffic to pods with label keys and values matching this selector +1. type - [`"ExternalName" | "ClusterIP" | "NodePort" | "LoadBalancer"` - `string`] + - determines how the Service is exposed + - defaults to `ClusterIP` - which allocates a cluster-internal IP address for load-balancing to endpoints + - endpoints are determined by the selector diff --git a/docs/cloud_operations/kubernetes/resources/service/service-status.md b/docs/cloud_operations/kubernetes/resources/service/service-status.md new file mode 100644 index 0000000..4b425b3 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/service/service-status.md @@ -0,0 +1,12 @@ +--- +title: Service Status +description: "" +unlisted: true +--- + +# [Service Status](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#servicestatus-v1-core) + +Here's the most important attributes: + +1. conditions - [`Array`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#condition-v1-meta) +1. loadBalancer - [`LoadBalancerStatus`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#loadbalancerstatus-v1-core) \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/services.md b/docs/cloud_operations/kubernetes/resources/services.md new file mode 100644 index 0000000..1dd9563 --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/services.md @@ -0,0 +1,164 @@ +--- +title: Services +description: '' +sidebar_position: 4 +--- +# [Services](https://kubernetes.io/docs/concepts/services-networking/service/) + +## Goals +1. **Understand Kubernetes Services**: + - Learn what a Service is and its role in the Kubernetes architecture. +1. **Basic Example(s)**: + - Provide (a) basic example(s) of Services. + - Show how to define a Service using YAML. +1. **Service Spec Explanation**: + - Explain the components of the Service specification using the example(s) above. +1. **Service Types**: + - Describe different types of Services and their use cases (ClusterIP, NodePort, LoadBalancer, ExternalName). +1. **Exercise**: + - Hands-on activity to create a simple Service. + +## Understanding Kubernetes Services + +A **Service** in Kubernetes is an abstraction that defines a logical set of pods and a policy by which to access them. Services enable networking and connectivity between different parts of a Kubernetes application or with external services. + +## Basic Service Example + + +### 1. ClusterIP Service +Here is a basic example of a Kubernetes Service specification in YAML: + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: nginx-service +spec: + selector: + app: nginx + ports: + - protocol: TCP + port: 80 + targetPort: 80 + type: ClusterIP +``` + +This Service provides a stable IP and DNS name to access the set of `nginx` pods. + +Save this to a file `service.yaml` and create the resource using `kubectl apply -f service.yaml`: + +```bash +> kubectl apply -f service.yaml +service/nginx-service created +``` + +### NodePort Service + +First we create the server Pod `service-simple-pod.yaml` + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: nginx-pod + labels: + app: nginx +spec: + containers: + - name: nginx-container + image: nginx:latest + ports: + - containerPort: 80 +``` + +We then create the corresponding service for the pod `service-simple.yaml` + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: nginx-nodeport +spec: + selector: + app: nginx + ports: + - protocol: TCP + port: 80 + targetPort: 80 + nodePort: 30001 + type: NodePort +``` + +We proceed to create both: + +```bash +> kubectl apply -f service-simple-pod.yaml +> kubectl apply -f service-simple.yaml +``` + +We then find the Node's IP: + +```bash +> minikube ip +``` + +And then we try to curl from the terminal: + +```bash +> curl http://$(minikube ip):30001 + + + +Welcome to nginx! + + + +

Welcome to nginx!

+

If you see this page, the nginx web server is successfully installed and +working. Further configuration is required.

+ +

For online documentation and support please refer to +nginx.org.
+Commercial support is available at +nginx.com.

+ +

Thank you for using nginx.

+ + +``` + +## Service Spec Explanation + +1. apiVersion: v1 (hardcoded) +1. kind: Service (type of the resource) +1. metadata: [ObjectMeta](./object-meta.md) +1. spec: [ServiceSpec](./service/service-spec.md) +1. status: [ServiceStatus](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#loadbalancerstatus-v1-core) + +## Service Types + +- **ClusterIP**: Exposes the Service on a cluster-internal IP. This is the default type. +- **NodePort**: Exposes the Service on each Node's IP at a static port. (We'll be using this for examples) +- **LoadBalancer**: Exposes the Service externally using a cloud provider's load balancer. +- **ExternalName**: Maps a Service to an external DNS name. + +## Exercises + +### Exercise 1: Expose an Application Internally Using a ClusterIP Service + +**Objective**: Deploy an application and expose it within the cluster using a ClusterIP Service. + +**Task**: +- Deploy a pod running a simple HTTP server (e.g., `nginx` or `httpd`). +- Create a ClusterIP Service to expose this HTTP server within the Kubernetes cluster. +- Verify that the service is accessible from other pods in the cluster. + +:::tip + +You can deploy a Pod running a simple linux image (`ubuntu:22.04`). Shelling into it was shown previously on the [`Pods`](./pods.md) page, do you remember how to do it without looking at it? + +::: \ No newline at end of file diff --git a/docs/cloud_operations/kubernetes/resources/statefulsets.md b/docs/cloud_operations/kubernetes/resources/statefulsets.md new file mode 100644 index 0000000..ba4aa5b --- /dev/null +++ b/docs/cloud_operations/kubernetes/resources/statefulsets.md @@ -0,0 +1,110 @@ +--- +title: StatefulSets +description: '' +sidebar_position: 8 +--- + +# [StatefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) + +## Goals + +1. **Understand Kubernetes StatefulSets**: + - Learn what a StatefulSet is and its role in Kubernetes for managing stateful applications. +1. **Basic Example(s)**: + - Provide (a) basic example(s) of StatefulSet resources. + - Show how to define a StatefulSet using YAML. +1. **StatefulSet Spec Explanation**: + - Explain the components of the StatefulSet specification using the example(s) above. +1. **StatefulSets vs. ReplicaSets**: + - Describe the advantages and differences between StatefulSets and ReplicaSets. +1. **Exercise**: + - Hands-on activity to create a simple StatefulSet. + +## Understanding Kubernetes StatefulSets + +A **StatefulSet** in Kubernetes is a controller that manages the deployment and scaling of a set of Pods, and provides guarantees about the ordering and uniqueness of these Pods. StatefulSets are useful for applications that require stable network identifiers, stable storage, and ordered deployment and scaling. + +## Basic StatefulSet Example + +### Simple StatefulSet + +Here is a basic example of a Kubernetes StatefulSet specification in YAML: + +```yaml +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: example-statefulset +spec: + serviceName: "example" + replicas: 3 + selector: + matchLabels: + app: example + template: + metadata: + labels: + app: example + spec: + containers: + - name: example-container + image: nginx + ports: + - containerPort: 80 + volumeClaimTemplates: + - metadata: + name: example-storage + spec: + accessModes: ["ReadWriteOnce"] + resources: + requests: + storage: 1Gi +``` + +This StatefulSet deploys three replicas of an nginx container, each with a unique identity and associated persistent storage. + +Save this to a file `statefulset.yaml` and create the resource using: + +```sh +> kubectl apply -f statefulset.yaml +statefulset.apps/example-statefulset created +``` + +## StatefulSet Spec Explanation + +1. **apiVersion**: apps/v1 (API group and version) +1. **kind**: StatefulSet (type of the resource) +1. **metadata**: [ObjectMeta](./object-meta.md) +1. **spec**: [StatefulSetSpec](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#statefulsetspec-v1-apps) +1. **status**: [StatefulSetStatus](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#statefulsetstatus-v1-apps) + +## StatefulSets vs. ReplicaSets + +- **StatefulSets**: + - Provide stable, unique network identifiers for Pods. + - Ensure that Pods are started and stopped in a specific order. + - Support stable storage using PersistentVolumeClaims. + - Ideal for stateful applications requiring consistent identities and storage. + +- **ReplicaSets**: + - Focus on maintaining a stable set of replicas for a given Pod specification. + - Do not provide stable network identities or persistent storage. + - Suitable for stateless applications where Pods can be replaced or rescheduled without losing data. + +## Exercises + +### Exercise 1: Create a Simple StatefulSet + +**Objective**: Deploy an application using a StatefulSet with persistent storage. + +**Task**: +- Define a StatefulSet with a suitable number of replicas and a container image. +- Create a Service to manage network access to the StatefulSet. +- Use volumeClaimTemplates to request persistent storage for each Pod. +- Verify that each Pod has a unique network identity and persistent storage. + +:::tip + +StatefulSets require a Headless Service to manage network identities. Ensure you define the `serviceName` in your StatefulSet spec and create a corresponding Headless Service. + +::: diff --git a/docs/cloud_operations/overview/_category_.json b/docs/cloud_operations/overview/_category_.json new file mode 100644 index 0000000..8070419 --- /dev/null +++ b/docs/cloud_operations/overview/_category_.json @@ -0,0 +1,9 @@ +{ + "label": "Overview", + "position": 1, + "link": { + "type": "generated-index", + "description": "Overview of Docker and Kubernetes" + } + } + \ No newline at end of file diff --git a/docs/cloud_operations/overview/container-basics.md b/docs/cloud_operations/overview/container-basics.md new file mode 100644 index 0000000..e438107 --- /dev/null +++ b/docs/cloud_operations/overview/container-basics.md @@ -0,0 +1,73 @@ +--- +id: container-basics +title: Container Basics +description: General Overview of the Course +slug: /overview/container-basics +sidebar_position: 2 +--- + +# Container Basics + +## Containerization in a nutshell + +To understand containerization let's first take a look at what *virtualization* is. In short, if you +ever wanted to run Linux inside your Windows computer, or Mac OS inside Linux, or any other +combination - you can through the use of **Virtual Machines**! As their name implies, you can fully +simulate any operating system that is isolated from your own PC's OS. Usually, a tool like +[VirtualBox](https://www.virtualbox.org/), or +[KVM](https://www.redhat.com/en/topics/virtualization/what-is-KVM) can run multiple virtual machines +at once, being limited only by how powerful your hardware is. + +
![img](./virtualization.drawio.png)
+ +Virtual machines are very useful if you want to accurately simulate an entire machine. This is often +useful in cybersecurity where you want to analyze some malware to see its behaviour, or in education +where you want all students to have the same setup. However, if all you want is to have a +**consistent** environment in which you can run your application, simulating an entire operating +system could be a bit overkill. This is where **containers** come in. + +**Containers**, compared to VMs, are friendlier with the host operating system, sharing resources in +a more streamlined manner. They don't require you to run an entire operating system, just the +underlying libraries necessary for your application to run, and your application code. The +industry-standard application to manage containers is [Docker](https://www.docker.com/). + +
![img](./containerization.drawio.png)
+ +Another neat advantage of containers, due to their lightweight nature they start extremely fast. +This is extremely helpful when managing widely popular applications due to their ability to create +more containers to respond to a sudden surge in new users. + +## Why would someone care about containerization? + +Imagine that you have built your billion-dollar application - the software which will revolutionize +the world as we know it 💰! Now, all you need to do is to put it on a server so it can be accessed +by everybody. You install all of your dependencies, `node`, `python`, etc., and your program +crashes. You tracked down the culprit and there was a slight change in `node` that introduced a +small bug. No worries, you change the version to one that works before. + +Then, after some weeks, you notice that your website keeps going down. After some investigation, you +notice that there is a small bug in a mini-game you introduced as an easter egg. That small bug +crashes the mini-game, but more importantly, it takes down the whole application with it. You fix it +and move on. + +Your website is now getting thousands of users, and your database cannot handle that many users at +once. Not to worry, you just rent out a bigger computer. Then, your app goes popular in America, and +Australia and now you are getting complaints that your app is slow due to the massive delay between +your server in Europe and their location. Now you have to rent out multiple servers, in multiple +locations, but then you forgot that you had to specify a `node` version, and you have to shut down +your service to re-engineer everything. + +All of these headaches could have been avoided by using containers. They offer an **isolated and +reproducible environment**. This means that you have to specify your dependencies once, and each +time a new container for your application is created it is guaranteed to work. More so, you can +split up your application into multiple containers - for example, a container handles the database, +one handles the user interface, and another handles that buggy mini-game. If one fails, the others +will continue running. And, after you learn about **Kubernetes**, you will also understand how to +start multiple instances of the same application to better handle bigger workloads. + +However, as a word of caution, don't start creating your application with the idea that it will have +millions of users on the first day. Even some popular applications with millions of users [used +Google Sheets as a placeholder for their +database](https://youtu.be/zcu9EaLP-aM?si=Ach8bAEg3xnR3Zsa&t=41) and got away with it. It's best to +use a few containers at the start and design your software in such a way that if it does go popular +you don't have to work too hard. diff --git a/docs/cloud_operations/overview/containerization.drawio.png b/docs/cloud_operations/overview/containerization.drawio.png new file mode 100644 index 0000000..77278af Binary files /dev/null and b/docs/cloud_operations/overview/containerization.drawio.png differ diff --git a/docs/cloud_operations/overview/introduction.md b/docs/cloud_operations/overview/introduction.md new file mode 100644 index 0000000..071d887 --- /dev/null +++ b/docs/cloud_operations/overview/introduction.md @@ -0,0 +1,64 @@ +--- +id: overview-introduction +title: Introduction +description: General Overview of the Course +slug: /overview/introduction +sidebar_position: 1 +--- + +# Introduction + +## Course Overview + +### Course tutors + +- Paris Cristian-Tănase (Cristi) +- Popescu Adrian (Adi) +- Cosma George (George) + +### Course structure + +1. First day: local setup, importance of containerization and orchestration, container basics, + containers vs virtual machines +1. Docker + docker compose +1. Kubernetes +1. Deployment + NGINX + +## What is this course about? + +The goal of this course is to teach you about **containers** and **orchestration with Kubernetes**. +After this course, you will be able to deploy an application into production that can serve users +worldwide with minimal latency and headache for you, and the development team (which may include you +😎). + +
+ +In brief, containerization is packaging an application and its dependencies into a standardized +unit, known as a container. This container can then be easily moved between environments, ensuring +consistency and reliability across different systems. + +We care about containerization because it allows developers to build, test, and deploy applications +more efficiently. By isolating applications within containers, we can avoid conflicts between +dependencies and ensure that the application runs consistently regardless of the underlying +infrastructure. + +Orchestration is managing and coordinating multiple containers in a distributed environment. This +includes tasks such as scaling containers up or down, load balancing traffic between containers, +and ensuring high availability of the application. Kubernetes is a popular orchestration tool that +helps automate these tasks, making it easier to manage containerized applications at scale. + +By the end of this course, you will have the skills and knowledge needed to deploy and manage +containerized applications in a production environment. + +
+ +## Setup instruction for required tools + +You need to install the following tools to solve the workshop: + +1. Docker and Docker Compose (see the [manual](https://docs.docker.com/manuals/)) +1. Kubectl (see the [manual](https://kubernetes.io/docs/tasks/tools/)) +1. Minikube (see the + [manual](https://minikube.sigs.k8s.io/docs/start/?arch=%2Flinux%2Fx86-64%2Fstable%2Fbinary+download)) +1. WSL2 (if you have Windows) (follow the [install + instructions](https://learn.microsoft.com/en-us/windows/wsl/install)) \ No newline at end of file diff --git a/docs/cloud_operations/overview/virtualization.drawio.png b/docs/cloud_operations/overview/virtualization.drawio.png new file mode 100644 index 0000000..70e79a5 Binary files /dev/null and b/docs/cloud_operations/overview/virtualization.drawio.png differ diff --git a/static/img/docs/kubernetes/1.svg b/static/img/docs/kubernetes/1.svg new file mode 100644 index 0000000..5e7289c --- /dev/null +++ b/static/img/docs/kubernetes/1.svg @@ -0,0 +1,412 @@ + + + Kubernetes components + + + + image/svg+xml + + Kubernetes components + + + Cloud Native Computing Foundation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + k-proxy + + + + + + + + + + kubelet + + + + + + + + + + + + sched + + + + + + + + + + sched + + + + + + + + + + sched + + + + + Control Plane + Node + + + + + + + + etcd + + + + + Kubernetes cluster + + + + + + + + api + + + + + + + + + + api + + + + + + + + + + api + + + + + + + + + + + + + c-c-m + + + + + + + + + + c-c-m + + + + + + + + + + c-c-m + + + + + + + + + + + + + c-m + + + + + + + + + + c-m + + + + + + + + + + c-m + + + + + Node + Node + + + + + + + + + + + + + + + + + + + + k-proxy + + + + + + + + + + kubelet + + + + + + + + + + kubelet + + + + + + + + + + k-proxy + + + + + + + + + + + + + + + + + + + + Control plane + Scheduler + + + + + + + + sched + + + + + Cloud controllermanager(optional) + + + + + + + + c-c-m + + + + + Controllermanager + + + + + + + + c-m + + + + + kubelet + + + + + + + + kubelet + + + + + kube-proxy + + + + + + + + k-proxy + + + + + + (persistence store) + etcd + + + + + + + + + etcd + + + + + + Node + + API server + + + + + + + + api + + + + + + + diff --git a/static/img/docs/kubernetes/2.png b/static/img/docs/kubernetes/2.png new file mode 100644 index 0000000..db4f6e6 Binary files /dev/null and b/static/img/docs/kubernetes/2.png differ