Docker
Docker lets you deploy high-level virtual machines called containers. Containers are lightweight and fast to start as they don't virtualize the hardware like VMs but only the OS. In other words, your container will contain its own file-system, libraries and even linux distribution.
We use them extensively on the compute nodes: no need to install anything on the machine, just use a container. In the following sections we will explain how to make your own docker image (container build instructions) and run your container.
You will be able to run your experiments in its own contained environment : once your container posses all your project dependencies, you can be sure that your setup will be the same forever and you will never have dependencies issues between your projects.
The image represents a system that you can instantiate as a container. The image can be seen as a recipe of a system and docker lets you manage images as if they were code repository :
- You can commit changes;
- You can pull them from public databases;
- You can clone, backup, share it. You can see all loaded images on your system with the following command:
docker images
The container is an instance of the image : while the image describes a state of your system, the container is the actual system running. There you can maintain a file-system, change the system (e.g. install/remove libraries) and more importantly execute software. You can see all running containers with the following command:
docker ps
There are three main ways to have a docker image on your system :
- Build it from a Dockerfile;
- Pull it from a repository;
- Load it from a local file;
If you want to setup your own system, this might be the best way as you have a full control on what packages to include in your image. The Dockerfile is a recipe of your system, it is basically a list of instruction to install the packages that you want in your system. Here is an example for pytorch, read the documentation to know more about writing Dockerfiles. Once your file is written simply build it, run this command in the folder containing the Dockerfile :
docker build -t "<image name>:<image version>" .
If you want to use an image already on the public database you can use this command:
docker pull repository:tag
This is mostly useful if you want to transfer your image from host to host. Saving an existing image to a file:
docker save -o /path/to/file.tar <image name>:<image version>
Then on the other system load it:
docker load -i /path/to/file.tar
todo...
Cheat sheet made by Marc-André Gardner:
- Inspired by Marc-André's slides.