how to use GPU

[bowenroom]

I pulled kaggle python image and seems to work fine, but I'm not sure what to do to use my GPU with theano or tensorflow.

Is it as simple as changing flags in .theanorc file? Can i edit it directly in the docker image?

New to docker here :)

[d-henderson]

Think, you might need to use nvidia-docker to make use of your gpu.

[nerdcha]

GPU support is something that we're going to add soon.

[hyzhak]

@nerdcha I see that there is script to build GPU image. But Is there any documentation how could I pull GPU image without assembling it from the source code?

[breckuh]

I was successfully able to build and run the GPU image from gpu.Dockerfile.

However, running a GPU script inside that image I hit this:

THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1535491974311/work/aten/src/THC/THCGeneral.cpp line=74 error=35 : CUDA driver version is insufficient for CUDA runtime version
Traceback (most recent call last):
  File "kesnet.py", line 115, in <module>
    display_imgs(np.asarray(md.trn_ds.denorm(x)))
  File "/opt/conda/lib/python3.6/site-packages/fastai/dataset.py", line 252, in denorm
    if type(arr) is not np.ndarray: arr = to_np(arr)
  File "/opt/conda/lib/python3.6/site-packages/fastai/core.py", line 60, in to_np
    if isinstance(v, torch.cuda.HalfTensor): v=v.float()
RuntimeError: cuda runtime error (35) : CUDA driver version is insufficient for CUDA runtime version at /opt/conda/conda-bld/pytorch_1535491974311/work/aten/src/THC/THCGeneral.cpp:74

I am able to run this script on Kaggle.com and outside of the Docker image.

I've tried on 2 machines. One has Nvidia Driver Version: 410.48 and the other has Nvidia Driver Version: 396.54.

I'm not an expert at Docker or CUDA so could be something simple I've got configured wrong, but thought I'd post here in case there's an issue or the GPU Docker isn't ready for mass consumption yet.

[rosbo]

Hi @breckuh,

Both Nvidia driver version should be fine.

Have you updated any packages inside your GPU image?

[breckuh]

@rosbo , I have not.

I just downloaded one of the base Nvidia Docker images and was not able to detect the GPU inside even that container ("CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected"), so my issue appears to be more of a system wide one and not with this Kaggle image per se. I will keep digging. Thanks!

[rosbo]

@breckuh are you using the docker NVIDIA runtime?
https://github.com/NVIDIA/nvidia-docker

# you will need to add some apt repos first, see link above
sudo apt-get install -y nvidia-docker2

docker run --runtime=nvidia --rm kaggle/python-build-gpu /bin/bash

[breckuh]

@rosbo I am not! Thanks. Attempting to install now (but running into unrelated typical dependency problems :) ).

[breckuh]

I was able to get the docker NVIDIA runtime installed, able to run jobs on the GPU in the base NVIDIA CUDA devel image, and can now see the GPUs with nividia-smi from within the running kaggle/python-build-gpu image, but still get the "CUDA driver version is insufficient for CUDA runtime version" error message, same as above.

Still investigating....