RTX 3090

[mschart]

A docker container based on a tensorflow docker image from nvidia that comes with tensorflow 1.5 compiled with CUDA 11 can be used to run deeplabcut with an RTX 3090 GPU.

• make sure Nvidia driver is installed by typing nvidia-smi
• install docker if not already installed, instructions here
• Install nvidia toolkit
• docker pull nvcr.io/nvidia/tensorflow:21.03-tf1-py3
• Save the following text file as Dockerfile in a new folder

##########################################
# Dockerfile for DeepLabCut GPU training #
##########################################
#We use Tensorflow v1.5.5 for deeplabcut
FROM nvcr.io/nvidia/tensorflow:21.03-tf1-py3
# install needed tools
RUN apt-get update && apt-get install -y wget ffmpeg
# install deeplabcut
RUN python3 -m pip install deeplabcut --no-cache-dir
# download git repo
RUN git clone https://github.com/AlexEMG/DeepLabCut /root/DeepLabCut/
RUN apt-get --yes --force-yes install python3-tk
RUN pip install git+https://github.com/int-brain-lab/ibllib.git
RUN pip install git+https://github.com/int-brain-lab/iblvideo.git
WORKDIR /root
ADD resources /usr/local/resources
ADD integration /usr/local/integration
#Instructions:
#cd in folder with this file and then build the docker image with (use --no-cache if you update)
#   docker build -t dlc .
#then run the image and open interactive shell with:
#   docker run --gpus all -it --rm dlc /bin/bash

• from Downloads/Flatiron, copy the folders resources and integration into the folder where you have the Dockerfile

• cd in folder with the Dockerfile and then build the docker image with
  docker build -t dlc .

• then run the image and open interactive shell with:
  docker run --gpus all -it --rm dlc /bin/bash

• Navigate to home folder inside the interactive docker shell, create and save there your .one_params file

• start ipython, then from iblvideo import dlc, set weights path and test video path

 path_dlc = '/usr/local/resources/dlc/weights_v1.1'
 mp4 = '/usr/local//integration/dlc/test_data/dlc_test_data_v1.1/input/_iblrig_bodyCamera.raw.mp4'

• test if GPU is engaged
  dlc(file_mp4, path_dlc=path_dlc)

• see https://github.com/int-brain-lab/iblvideo to run queue

[mschart]

If you get:

docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]

Fix it via:

sudo apt-get install -y nvidia-container-toolkit