Add Dockerfile and Docker quickstart guide #2

Merged
merged 3 commits into from Oct 4, 2016

Projects

None yet

4 participants

@neufeldtech
Contributor

I've created a simple Dockerfile and guide to be considered for inclusion in this repo.
The goal in mind is to lower the entry barrier for users that want to quickly get started and explore this repo without having to install all of the projects dependencies (Caffe).

@yahoocla
yahoocla commented Oct 1, 2016

Thank you for submitting this pull request, however I do not see a valid CLA on file for you. Before we can merge this request please visit https://yahoocla.herokuapp.com/ and agree to the terms. Thanks! 😄

@yahoocla
yahoocla commented Oct 1, 2016

CLA is valid!

@jay-mahadeokar
Contributor

Thanks for adding docker support. Can you also add steps / links on docker installation (if possible for Mac)? I will try it out.
In case there are lot of steps (for different OS) I think we could add a wiki page for docker setup and point to it from this Readme, so that the Readme remains compact. What do you think?

@neufeldtech
Contributor

I've added Docker Installation links for Windows, Mac and Ubuntu. I have tested the getting started guide on Mac OSX and Ubuntu but do not have a Windows machine for testing at this time.

This was referenced Oct 2, 2016
Dockerfile
@@ -0,0 +1,4 @@
+FROM kaixhin/caffe
@jay-mahadeokar
jay-mahadeokar Oct 3, 2016 Contributor

@neufeldtech Is this the official caffe docker package? There is also another guide to docker, lets consolidate it in this PR.

@loretoparisi

@neufeldtech it seems we did both the same! Please take a look here also if it can help #7 so that we will have just one PR 👍

@neufeldtech
Contributor

@jay-mahadeokar I've updated the README to build the docker image directly from the official BVLC caffe repo now.

@loretoparisi I like your approach! I've collapsed a few steps of your instructions into fewer commands. Take a look at what I've got so far here: 68040e6
Note: I opted not to include how to build the GPU image because I could not get it to work. What specific requirements/dependencies would someone need in order for this to work with the GPU image?

@jay-mahadeokar

This looks good!

@jay-mahadeokar
Contributor

Thanks for adding docker support @neufeldtech @loretoparisi.

@jay-mahadeokar jay-mahadeokar merged commit 79f77bc into yahoo:master Oct 4, 2016
@loretoparisi
loretoparisi commented Oct 5, 2016 edited

@neufeldtech basically you need to install on the host machine the nvidia CUDA 8.0 drivers, what we could do is

  1. use caffe build through the caffe:gpu directly on the host machine:
$ docker build -t caffe:gpu ./

we verify

$ docker run -ti caffe:gpu caffe --version
libdc1394 error: Failed to initialize libdc1394
caffe version 1.0.0-rc3

This will download nvdia CUDA drivers and install it

$ nvidia-smi
Wed Oct  5 10:36:29 2016       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.44                 Driver Version: 367.44                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GT 620      Off  | 0000:01:00.0     N/A |                  N/A |
| 52%   30C   P12    N/A /  N/A |    247MiB /   962MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0                  Not Supported                                         |
+-----------------------------------------------------------------------------+

Note that we have one GPU (a very cheap GeForce GT 620 here) addressed at 0

And our container is up & running

CONTAINER ID        IMAGE                         COMMAND                  CREATED             STATUS                     PORTS                                            NAMES
443a7addcdff        caffe:gpu                     "caffe --version"        2 minutes ago       Exited (0) 2 minutes ago                                                    naughty_hamilton
  1. Using the nvidia-docker container. In this way it would be very easy to add optional support to gpu that will work linux machines (Ubuntu/CentOS) with a supported CUDA 8.0 gpu (the problem here is that CUDA 8.0 moved to the new Pascal architecture, so it needs new graphic cards drivers on old gpus, etc.

In this case we could do

# Install nvidia-docker and nvidia-docker-plugin
wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb
sudo dpkg -i /tmp/nvidia-docker*.deb && rm /tmp/nvidia-docker*.d

and verify again that the nvidia docker container is getting the gpu as well

$ nvidia-docker run --rm nvidia/cuda nvidia-smi
Wed Oct  5 08:39:16 2016       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.44                 Driver Version: 367.44                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GT 620      Off  | 0000:01:00.0     N/A |                  N/A |
| 52%   31C   P12    N/A /  N/A |    247MiB /   962MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0                  Not Supported                                         |
+-----------------------------------------------------------------------------+

ok it seems so,

./nvidia-docker-run --volume=$(pwd):/workspace --rm -it caffe:gpu bash

so that we will have caffe running with the gpu on the host machine:

root@1bd4c3073ef0:/workspace# python ./classify_nsfw.py --model_def nsfw_model/deploy.prototxt --pretrained_model nsfw_model/resnet_50_1by2_nsfw.caffemodel anderson_210.jpg

If we can this typical error

F1005 08:48:48.429678    16 cudnn_conv_layer.cpp:52] Check failed: error == cudaSuccess (35 vs. 0)  CUDA driver version is insufficient for CUDA runtime version

it means that we have to update the CUDA driver as stated above - when it is possibile, in my case on this old ubuntu it is not possibile due to the graphic card installed:

$ sudo lshw -c video
  *-display               
       description: VGA compatible controller
       product: GF108 [GeForce GT 620]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:49 memory:fa000000-faffffff memory:f0000000-f7ffffff memory:f8000000-f9ffffff ioport:e000(size=128) memory:fb000000-fb07ffff

and

$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module  367.44  Wed Aug 17 22:24:07 PDT 2016
GCC version:  gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.2) 

but with let's say a Nvidia GTX 1080 it will work.

@jay-mahadeokar
Contributor
jay-mahadeokar commented Oct 5, 2016 edited

I think it is best to have a wiki page with docker information and documentation. We can move all the documentation to that page, so that the Readme itself remains concise. @loretoparisi we can also move the existing docker documentation in Readme to that wiki page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment