Docker 101 workshop - introduction to Docker and basic concepts
You will need Mac OSX with at least
7GB RAM and
8GB free disk space available.
For Linux: follow instructions provided here.
If you have Mac OS X (Yosemite or newer), please download Docker for Mac here.
Older docker package for OSes older than Yosemite -- Docker Toolbox located here.
Xcode and local tools
Xcode will install essential console utilities for us. You can install it from AppStore.
Docker is as easy as Linux! To prove that let us write classic "Hello, World" in Docker
$ docker run busybox echo "hello world"
Docker containers are just as simple as linux processes, but they also provide many more features that we are going to explore.
Let's review the structure of the command:
docker run # executes command in a container busybox # container image echo "hello world" # command to run
Container image supplies environment - binaries with shell for example that is running the command, so you are not using host operating system shell, but the shell from busybox package when executing Docker run.
Sneak peek into container environment
Let's now take a look at process tree running in the container:
$ docker run busybox ps uax
My terminal prints out something like this:
1 root 0:00 ps uax
NOTE: Oh my! Am I running this command as root? Yes, although this is not your regular root user but a very limited one. We will get back to the topic of users and security a bit later.
As you can see, the process runs in a very limited and isolated environment, and the PID of the process is 1, so it does not see all other processes running on your machine.
Adding envrionment variables
Let's see what environment variables we have:
$ docker run busybox env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=0a0169cdec9a
The environment is different from your host environment.
We can extend environment by passing explicit enviornment variable flag to
$ docker run -e HELLO=world busybox env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=8ee8ba3443b6 HELLO=world HOME=/root
Adding host mounts
If we look at the disks we will see the OS directories are not here, as well:
$ docker run busybox ls -l /home total 0
What if we want to expose our current directory to the container? For this we can use host mounts:
$ docker run -v $(pwd):/home busybox ls -l /home total 72 -rw-rw-r-- 1 1000 1000 11315 Nov 23 19:42 LICENSE -rw-rw-r-- 1 1000 1000 30605 Mar 22 23:19 README.md drwxrwxr-x 2 1000 1000 4096 Nov 23 19:30 conf.d -rw-rw-r-- 1 1000 1000 2922 Mar 23 03:44 docker.md drwxrwxr-x 2 1000 1000 4096 Nov 23 19:35 img drwxrwxr-x 4 1000 1000 4096 Nov 23 19:30 mattermost -rw-rw-r-- 1 1000 1000 585 Nov 23 19:30 my-nginx-configmap.yaml -rw-rw-r-- 1 1000 1000 401 Nov 23 19:30 my-nginx-new.yaml -rw-rw-r-- 1 1000 1000 399 Nov 23 19:30 my-nginx-typo.yaml
This command "mounted" our current working directory inside the container, so it appears to be "/home"
inside the container! All changes that we do in this repository will be immediately seen in the container's
Networking in Docker containers is isolated, as well. Let us look at the interfaces inside a running container:
$ docker run busybox ifconfig eth0 Link encap:Ethernet HWaddr 02:42:AC:11:00:02 inet addr:172.17.0.2 Bcast:0.0.0.0 Mask:255.255.0.0 inet6 addr: fe80::42:acff:fe11:2/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1 errors:0 dropped:0 overruns:0 frame:0 TX packets:1 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:90 (90.0 B) TX bytes:90 (90.0 B) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
We can use
-p flag to forward a port on the host to the port 5000 inside the container:
$ docker run -p 5000:5000 library/python:3.3 python -m http.server 5000
This command blocks because the server listens for requests, open a new tab and access the endpoint
$ curl http://localhost:5000 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html> <head> ....
Ctrl-C to stop the running container.
A bit of background
A Docker container is a set of linux processes that run isolated from the rest of the processes.
Multiple linux subsystems help to create a container concept:
Namespaces create isolated stacks of linux primitives for a running process.
- NET namespace creates a separate networking stack for the container, with its own routing tables and devices
- PID namespace is used to assign isolated process IDs that are separate from host OS. For example, this is important if we want to send signals to a running process.
- MNT namespace creates a scoped view of a filesystem using VFS. It lets a container to get its own "root" filesystem and map directories from one location on the host to the other location inside container.
- UTS namespace lets container to get to its own hostname.
- IPC namespace is used to isolate inter-process communication (e.g. message queues).
- USER namespace allows container processes have different users and IDs from the host OS.
Kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, etc.)
Capabilitites provide enhanced permission checks on the running process, and can limit the interface configuration, even for a root user - for example (
You can find a lot of additional low level detail here.
More container operations
Our last python server example was inconvenient as it worked in foreground:
$ docker run -d -p 5000:5000 --name=simple1 library/python:3.3 python -m http.server 5000
-d instructs Docker to start the process in background. Let's see if still works:
Inspecting a running container
We can use
ps command to view all running containers:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES eea49c9314db library/python:3.3 "python -m http.serve" 3 seconds ago Up 2 seconds 0.0.0.0:5000->5000/tcp simple1
- Container ID - auto generated unique running id.
- Container image - image name.
- Command - linux process running as the PID 1 in the container.
- Names - user friendly name of the container, we have named our container with
We can use
logs to view logs of a running container:
$ docker logs simple1
Attaching to a running container
We can execute a process that joins container namespaces using
$ docker exec -ti simple1 /bin/sh
We can look around to see the process running as PID 1:
# ps uax USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.5 0.0 74456 17512 ? Ss 18:07 0:00 python -m http.server 5000 root 7 0.0 0.0 4336 748 ? Ss 18:08 0:00 /bin/sh root 13 0.0 0.0 19188 2284 ? R+ 18:08 0:00 ps uax #
This gives an illusion that you
SSH in a container. However, there is no remote network connection.
/bin/sh started an instead of running in the host OS joined all namespaces of the container.
-tflag attaches terminal for interactive typing.
-iflag attaches input/output from the terminal to the process.
Starting and stopping containers
To stop and start container we can use
$ docker stop simple1 $ docker start simple1
NOTE: container names should be unique. Otherwise, you will get an error when you try to create a new container with a conflicting name!
-it combination allows us to start interactive containers without attaching to existing ones:
$ docker run -ti busybox # ps uax PID USER TIME COMMAND 1 root 0:00 sh 7 root 0:00 ps uax
Attaching to containers input
To best illustrate the impact of
--interactive in the expanded version, consider this example:
$ echo "hello there " | docker run busybox grep hello
The example above won't work as the container's input is not attached to the host stdout. The
-i flag fixes just that:
$ echo "hello there " | docker run -i busybox grep hello hello there
Building Container images
So far we have been using container images downloaded from Docker's public registry.
Starting from scratch
Dockerfile is a special file that instructs
docker build command how to build an image
$ cd docker/scratch $ docker build -t hello . Sending build context to Docker daemon 3.072 kB Step 1 : FROM scratch ---> Step 2 : ADD hello.sh /hello.sh ---> 4dce466cf3de Removing intermediate container dc8a5b93d5a8 Successfully built 4dce466cf3de
The Dockerfile looks very simple:
FROM scratch ADD hello.sh /hello.sh
FROM scratch instructs a Docker build process to use empty image to start building the container image.
ADD hello.sh /hello.sh adds file
hello.sh to the container's root path
docker images command is used to display images that we have built:
docker images REPOSITORY TAG IMAGE ID CREATED SIZE hello latest 4dce466cf3de 10 minutes ago 34 B
- Repository - a name of the local (on your computer) or remote repository. Our current repository is local and is called
- Tag - indicates the version of our image, Docker sets
latesttag automatically if not specified.
- Image ID - unique image ID.
- Size - the size of our image is just 34 bytes.
NOTE: Docker images are very different from virtual image formats. Because Docker does not boot any operating system, but simply runs linux process in isolation, we don't need any kernel, drivers or libraries to ship with the image, so it could be as tiny as several bytes!
Running the image
Trying to run it though, will result in the error:
$ docker run hello /hello.sh write pipe: bad file descriptor
This is because our container is empty. There is no shell and the script won't be able to start!
Let's fix that by changing our base image to
busybox that contains a proper shell environment:
$ cd docker/busybox $ docker build -t hello . Sending build context to Docker daemon 3.072 kB Step 1 : FROM busybox ---> 00f017a8c2a6 Step 2 : ADD hello.sh /hello.sh ---> c8c3f1ea6ede Removing intermediate container fa59f3921ff8 Successfully built c8c3f1ea6ede
Listing the image shows that image id and size have changed:
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE hello latest c8c3f1ea6ede 10 minutes ago 1.11 MB
We can run our script now:
$ docker run hello /hello.sh hello, world!
Let us roll a new version of our script
$ cd docker/busybox-v2 docker build -t hello:v2 .
We will now see 2 images:
$ docker images hello v2 195aa31a5e4d 2 seconds ago 1.11 MB hello latest 47060b048841 20 minutes ago 1.11 MB
latest will not automatically point to the latest version, so you have to manually update it
Execute the script using
$ docker run hello:v2 /hello.sh hello, world v2!
We can improve our image by supplying
$ cd docker/busybox-entrypoint $ docker build -t hello:v3 .
Entrypoint remembers the command to be executed on start, even if you don't supply the arguments:
$ docker run hello:v3 hello, world !
What happens if you pass flags? they will be executed as arugments:
$ docker run hello:v3 woo hello, world woo!
This magic happens because our v3 script prints passed arguments:
#!/bin/sh echo "hello, world $@!"
We can pass environment variables during build and during runtime as well.
Here's our modified shell script:
#!/bin/sh echo "hello, $BUILD1 and $RUN1!"
Dockerfile now uses
ENV directive to provide environment variable:
FROM busybox ADD hello.sh /hello.sh ENV BUILD1 Bob ENTRYPOINT ["/hello.sh"]
Let's build and run:
cd docker/busybox-env $ docker build -t hello:v4 . $ docker run -e RUN1=Alice hello:v4 hello, Bob and Alice!
Sometimes it is helpful to supply arguments during build process
(for example, user ID to create inside the container). We can supply build arguments as flags to
$ cd docker/busybox-arg $ docker build --build-arg=BUILD1="Alice and Bob" -t hello:v5 . $ docker run hello:v5 hello, Alice and Bob!
Here is our updated Dockerfile:
FROM busybox ADD hello.sh /hello.sh ARG BUILD1 ENV BUILD1 $BUILD1 ENTRYPOINT ["/hello.sh"]
ARG have supplied the build argument and we have referred to it right away, exposing it as environment variable right away.
Build layers and caching
Let's take a look at the new build image in the
$ ls -l docker/cache/ total 12 -rw-rw-r-- 1 sasha sasha 76 Mar 24 16:23 Dockerfile -rw-rw-r-- 1 sasha sasha 6 Mar 24 16:23 file -rwxrwxr-x 1 sasha sasha 40 Mar 24 16:23 script.sh
We have a file and a script that uses the file:
$ cd docker/cache $ docker build -t hello:v6 . Sending build context to Docker daemon 4.096 kB Step 1 : FROM busybox ---> 00f017a8c2a6 Step 2 : ADD file /file ---> Using cache ---> 6f48df47cb1d Step 3 : ADD script.sh /script.sh ---> b052fd11bcc6 Removing intermediate container c555e8ab29dc Step 4 : ENTRYPOINT /script.sh ---> Running in 50f057fd89cb ---> db7c6f36cba1 Removing intermediate container 50f057fd89cb Successfully built db7c6f36cba1 $ docker run hello:v6 hello, hello!
Let's update the script.sh
cp script2.sh script.sh
They are only differrent by one letter, but this makes a difference:
$ docker build -t hello:v7 . $ docker run hello:v7 Hello, hello!
Using cache diagnostic output from the container:
$ docker build -t hello:v7 . Sending build context to Docker daemon 5.12 kB Step 1 : FROM busybox ---> 00f017a8c2a6 Step 2 : ADD file /file ---> Using cache ---> 6f48df47cb1d Step 3 : ADD script.sh /script.sh ---> b187172076e2 Removing intermediate container 7afa2631d677 Step 4 : ENTRYPOINT /script.sh ---> Running in 51217447e66c ---> d0ec3cfed6f7 Removing intermediate container 51217447e66c Successfully built d0ec3cfed6f7
Docker executes every command in a special container. It detects the fact that the content has (or has not) changed, and instead of re-exectuing the command, uses cached value isntead. This helps to speed up builds, but sometimes introduces problems.
NOTE: You can always turn caching off by using the
--no-cache=true option for the
docker build command.
Docker images are composed of layers:
Every layer is a the result of the execution of a command in the Dockerfile.
The most frequently used command is
RUN: it executes the command in a container,
captures the output and records it as an image layer.
Let's us use existing package managers to compose our images:
FROM ubuntu:14.04 RUN apt-get update RUN apt-get install -y curl ENTRYPOINT curl
The output of this build will look more like a real Linux install:
$ cd docker/ubuntu $ docker build -t myubuntu .
We can use our newly created ubuntu to curl pages:
$ docker run myubuntu https://google.com % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 220 100 220 0 0 1377 0 --:--:-- --:--:-- --:--:-- 1383 <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>301 Moved</TITLE></HEAD><BODY> <H1>301 Moved</H1> The document has moved <A HREF="https://www.google.com/">here</A>. </BODY></HTML>
However, it all comes at a price:
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE myubuntu latest 50928f386c70 53 seconds ago 221.8 MB
That is 220MB for curl! As we know, now there is no good reason to have images with all the OS inside. If you still need it though, Docker will save you some space by re-using the base layer, so images with slightly different bases would not repeat each other.
Operations with images
You are already familiar with one command,
docker images. You can also remove images, tag and untag them.
Removing images and containers
Let's start with removing the image that takes too much disk space:
$ docker rmi myubuntu Error response from daemon: conflict: unable to remove repository reference "myubuntu" (must force) - container 292d1e8d5103 is using its referenced image 50928f386c70
Docker complains that there are containers using this image. How is this possible? We thought that all our containers are gone. Actually, Docker keeps track of all containers, even those that have stopped:
$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 292d1e8d5103 myubuntu "curl https://google." 5 minutes ago Exited (0) 5 minutes ago cranky_lalande f79c361a24f9 440a0da6d69e "/bin/sh -c curl" 5 minutes ago Exited (2) 5 minutes ago nauseous_sinoussi 01825fd28a50 440a0da6d69e "/bin/sh -c curl --he" 6 minutes ago Exited (2) 5 minutes ago high_davinci 95ffb2131c89 440a0da6d69e "/bin/sh -c curl http" 6 minutes ago Exited (2) 6 minutes ago lonely_sinoussi
We can now delete the container:
$ docker rm 292d1e8d5103 292d1e8d5103
and the image:
$ docker rmi myubuntu Untagged: myubuntu:latest Deleted: sha256:50928f386c704610fb16d3ca971904f3150f3702db962a4770958b8bedd9759b
docker tag helps us to tag images.
We have quite a lot of versions of
hello built, but latest still points to the old
$ docker images | grep hello hello v7 d0ec3cfed6f7 33 minutes ago 1.11 MB hello v6 db7c6f36cba1 42 minutes ago 1.11 MB hello v5 1fbecb029c8e About an hour ago 1.11 MB hello v4 ddb5bc88ebf9 About an hour ago 1.11 MB hello v3 eb07be15b16a About an hour ago 1.11 MB hello v2 195aa31a5e4d 3 hours ago 1.11 MB hello latest 47060b048841 3 hours ago 1.11 MB
Let's change that by re-tagging
$ docker tag hello:v7 hello:latest $ docker images | grep hello hello latest d0ec3cfed6f7 38 minutes ago 1.11 MB hello v7 d0ec3cfed6f7 38 minutes ago 1.11 MB hello v6 db7c6f36cba1 47 minutes ago 1.11 MB hello v5 1fbecb029c8e About an hour ago 1.11 MB hello v4 ddb5bc88ebf9 About an hour ago 1.11 MB hello v3 eb07be15b16a About an hour ago 1.11 MB hello v2 195aa31a5e4d 3 hours ago 1.11 MB
latest point to the same image ID
Images are distributed with a special service -
Let us spin up a local registry:
$ docker run -p 5000:5000 --name registry -d registry:2
docker push is used to publish images to registries.
To instruct where we want to publish, we need to append registry address to repository name:
$ docker tag hello:v7 127.0.0.1:5000/hello:v7 $ docker push 127.0.0.1:5000/hello:v7
docker push pushed the image to our "remote" registry.
We can now download the image using the
docker pull command:
$ docker pull 127.0.0.1:5000/hello:v7 v7: Pulling from hello Digest: sha256:c472a7ec8ab2b0db8d0839043b24dbda75ca6fa8816cfb6a58e7aaf3714a1423 Status: Image is up to date for 127.0.0.1:5000/hello:v7
We have learned how to start, build and publish containers and learned the containers building blocks. However, there is much more to learn. Just check out this official docker documentation!.
Thanks to Docker team for such an amazing product!