Skip to content

Troubleshooting Tips

Mathis Lab edited this page Jun 18, 2019 · 30 revisions

Welcome to the DeepLabCut wiki!

Here are some Troubleshooting tips for DeepLabCut 2.0: You can search within the whole repository by using the search box in the top left corner! (just be sure to click: in this repository)

TIPS: The Jupyter Notebooks provided are fully “run" so you can see expected outputs. They are also annotated with potential errors and considerations. Additionally, with testing of the Notebooks on Colaboratory (https://colab.research.google.com) the user can directly search for any error messages that arise, as typically they involve TensorFlow or other packages not managed by these authors. Please check the GitHub open/closed issues as well!

You can also always get more info about any function with:

HELP:

In ipython/Jupyter notebook:

deeplabcut.nameofthefunction?

In Python & pythonw (MacOS):

help(deeplabcut.nameofthefunction)

DeepLabCut Installation:

"import deeplabcut" fails in Jupyter Notebook If you installed deeplabcut in a conda environment, you need to switch to that environment within Jupyter (go to Kernel > change Kernel). If this is not possible first run: conda install nb_conda then restart the terminal, and then in Jupyter now you can change the kernel to specific environments in Jupyter

.bash: git: command not found. Git is not installed on the computer. Install git using https://git-scm.com/book/en/v2/Getting-Started-Installing-Git

conda: command not found. Anaconda is not installed on the computer. Install Anaconda using https://conda.io/docs/user-guide/install/index.html

During labeling videos, OSError: [WinError 6] The handle is invalid. This happens on some Windows installation if ffmeg is not installed correctly. Please install ffmeghttps://video.stackexchange.com/questions/20495

Access Denied. Insufficient permission to create directories. OSError: symbolic link privilege not held. This error is specific to the users of Windows operating system. Open command prompt with administrative rights. Use `Run as administrator' while opening the command prompt.

TensorFlow session or import errors. TensorFlow, CUDA & Driver versions & When you install TensorFlow, you must have the correct CUDA library and driver for the GPU card. & Check the following guide: https://developer.nvidia.com/cuda-toolkit-archive.

Windows: one user reports an issue with PyTables (i.e. here) pd.read_hdf(path). The solution was to upgrade tables from 3.4.4 to 3.4.5.dev using: pip install git+https://github.com/PyTables/PyTables

New! As of 2.0.5 MacOS is supported! For previous versions, MacOS was not supported. See here: https://github.com/AlexEMG/DeepLabCut/issues/101#issuecomment-437628340, and here: https://github.com/AlexEMG/DeepLabCut/issues/141

Launching Jupyter Notebooks: ERROR:browser_gpu_channel_host_factory.cc(103)] Failed to launch GPU process. Please see the fix here, i.e. you must run "export BROWSER=google-chrome" https://github.com/jupyter/notebook/issues/2836#issuecomment-337643224

Create New Project:

FilesExistsError. When creating a project, if you do not encase the video path with [ ], it will create a corrupt directory! Delete the corrupt directory and correct the error by placing the video/s inside brackets [ ].

The path must be a string that is passed for videos and for the working directory. If not, you will get errors! i.e. in Windows enter paths by either:

r`C:\Users\computername\Videos\reachingvideo1.avi'

`C:\\Users\\computername\\Videos\\reachingvideo1.avi' (i.e. double backslashes \ \ between C: and Users)

UnboundLocalError: local variable 'cfg' referenced before assignment. Extract frames, or any point where you set the config.yaml path, if it is improperly formatted, you will get this error. Solution: re-set your config_path with the correct formatting (see above).

Training the network (stopping training):

KeyboardInterrupt. Stopping training causes (seemingly many) error messages. These are normal, as you've interrupted the process by hitting CTRL+C or it reached the end point (default 1.03 M iterations)

Weights not downloaded. If you get permission errors FileNotFound (or ValueError: The passed save_path is not a valid checkpoint: xxx\Deeplabcut\lib\site-packages\deeplabcut\pose_estimation_tensorflow\models\pretrained\resnet_v1_50.ckpt) when you run training, first check if the RestNet weights downloaded.

This is an error more common in Windows, but some docker containers might not have privileges for this (it can be user specific). They should be under 'init_weights' (see path in the pose_cfg.yaml file). For Windows, also please install GITBASH https://gitforwindows.org/ and OPEN cmd by right-clicking "run as ADIM"

You can "cd" in the terminal to the package location!

  • i.e. in linux copy and paste this in: cd /usr/local/lib/python3.6/dist-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/

  • in Windows: cd C:\Users\YourComputerName\Anaconda3\envs\DLC\Lib\site-packages\deeplabcut\pose_estimation_tensorflow\models\pretrained

And if you type "ls" to see the list of files, you should see the resnet: resnet_v1_50.ckpt

If it is not there, run sudo download.sh then change the permissions: sudo chown yourusername:yourusername resnet_v1_50.ckpt

See more tips here: https://github.com/MMathisLab/Docker4DeepLabCut2.0#using-the-docker-for-training-and-video-analysis

Troubleshooting (2): if it appears the training does not start (i.e. "Starting training..." does not print immediately), then you have another session running on your GPU. Go check "nvidia-smi" and look at the process names. You can only have 1 per GPU!)

Evaluate the trained network:

Could not load matplotlib icon: ... In Jupyter notebooks or in the terminal this is just a warning, but the plots are being made elsewhere. We do not suppress error messages by default.

Video Analysis:

The videos are analyzed. but there are videos that were not analyzed!... When using deeplabcut.analyze_videos make sure DeepLabCut looks for the right type of videos in the directory you passed. Perhaps change the videotype accordingly. For instance, use: videotype=`.mp4'. By default it looks for videos that end with ".avi".

This also happens if you do not put the video in [ ], i.e. this is incorrect: deeplabcut.analyze_videos(config_path, '/yourvideofolder')

This is correct: deeplabcut.analyze_videos(config_path,[ '/yourvideofolder'])

SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

In windows enter paths by either:

r`C:\Users\computername\Videos\reachingvideo1.avi'

`C:\\Users\\computername\\Videos\\reachingvideo1.avi'

see also: https://stackoverflow.com/questions/1347791/

Outlier Extraction

In Windows, you need to run most all the steps with ADMIN rights (i.e. when you open cwd you need to run as admin. You can get a [WinError 5] or [WinError 6] if not!

You can’t perform that action at this time.