Skip to content

Latest commit

 

History

History
91 lines (66 loc) · 3.56 KB

README.md

File metadata and controls

91 lines (66 loc) · 3.56 KB

Docker Support

The Dockerfile in this directory will build Docker images with all the dependencies and code needed to run example notebooks or unit tests included in this repository.

Multiple environments are supported by using multistage builds. In order to efficiently build the Docker images in this way, Docker BuildKit is necessary. The following examples show how to build and run the Docker image for CPU, PySpark, and GPU environments.

Note: On some platforms, one needs to manually specify the environment variable for DOCKER_BUILDKITto make sure the build runs well. For example, on a Windows machine, this can be done by the powershell command as below, before building the image

$env:DOCKER_BUILDKIT=1

Warning: On some platforms using Docker Buildkit interferes with Anaconda environment installation. If you find that the docker build is hanging during Anaconda environment setup stage try building the container without Buildkit enabled.

Once the container is running you can access Jupyter notebooks at http://localhost:8888.

Building and Running with Docker

See examples below for the case of conda. If you use venv or virtualenv instead, replace --build-arg VIRTUAL_ENV=conda with --build-arg VIRTUAL_ENV=venv or --build-arg VIRTUAL_ENV=virtualenv, respectively.

CPU environment
DOCKER_BUILDKIT=1 docker build -t recommenders:cpu --build-arg ENV=cpu --build-arg VIRTUAL_ENV=conda .
docker run -p 8888:8888 -d recommenders:cpu
PySpark environment
DOCKER_BUILDKIT=1 docker build -t recommenders:pyspark --build-arg ENV=pyspark --build-arg VIRTUAL_ENV=conda .
docker run -p 8888:8888 -d recommenders:pyspark
GPU environment
DOCKER_BUILDKIT=1 docker build -t recommenders:gpu --build-arg ENV=gpu --build-arg VIRTUAL_ENV=conda .
docker run --runtime=nvidia -p 8888:8888 -d recommenders:gpu
GPU + PySpark environment
DOCKER_BUILDKIT=1 docker build -t recommenders:full --build-arg ENV=full --build-arg VIRTUAL_ENV=conda .
docker run --runtime=nvidia -p 8888:8888 -d recommenders:full

Build Arguments

There are several build arguments which can change how the image is built. Similar to the ENV build argument these are specified during the docker build command.

Build Arg Description
ENV Environment to use, options: cpu, psypark, gpu, full (defaults to cpu)
VIRTUAL_ENV Virtual environment to use; mandatory argument, must be one of "conda", "venv", "virtualenv"
ANACONDA Anaconda installation script (defaults to miniconda3 4.6.14)

Example:

DOCKER_BUILDKIT=1 docker build -t recommenders:cpu --build-arg ENV=cpu --build-arg VIRTUAL_ENV=conda .

In order to see detailed progress with BuildKit you can provide a flag during the build command: --progress=plain

Running tests with docker

To run the tests using e.g. the CPU image, do the following:

docker run -it recommenders:cpu bash -c 'pip install pytest; \
pip install pytest-cov; \
apt-get install -y git; \
git clone https://github.com/microsoft/recommenders.git; \
cd recommenders; \
pytest tests/unit -m "not spark and not gpu and not notebooks"'