Stuck at demo_detect

[dichen-cd]

Hi there~

I got a problem here running tools/demo.py. The script stuck at demo_detect(net, query_img) tracing back to blobs_out = net.forward(**forward_kwargs) and self._forward(start_ind, end_ind)

There is no other information printed while the python process keeps running(checked by top). This problem won't appear when running on CPU(--gpu -1).

The CUDA version is 8.0. Didn't compile with cudnn 'cuz there would be a Check failed: status == CUDNN_STATUS_SUCCESS (8 vs. 0) CUDNN_STATUS_EXECUTION_FAILED issue when running this script.

Thank you.

[Cysu]

Did you use cudnn-v5.1 for cuda-8.0 (note that there are versions for cuda-7.5 and cuda-8.0)? Did the following command link to the correct library?

ldd caffe/build/install/bin/caffe | grep cudnn

[dichen-cd]

Hi Tong,

Yes, I used cudnn-5.1 for cuda 8.0.

if built with cudnn, caffe links to:

libcudnn.so.5 => /usr/local/cuda/lib64/libcudnn.so.5 (0x00007fa1c979a000)

else the ldd command returns nothing related to cudnn.

ps: the cmake command was cmake .. -DUSE_MPI=ON -DCUDNN_INCLUDE=/usr/local/cuda/include/ -DCUDNN_LIBRARY=/usr/local/cuda/lib64/libcudnn.so

[Cysu]

Could you please check the output of

ls -al /usr/local/cuda/lib64/libcudnn*

And what's the GPU's memory size?

[dichen-cd]

The output is

-rwxr-xr-x 1 root root 84163560 Feb 25 02:47 /usr/local/cuda/lib64/libcudnn.so
-rwxr-xr-x 1 root root 84163560 Feb 25 02:47 /usr/local/cuda/lib64/libcudnn.so.5
-rwxr-xr-x 1 root root 84163560 Feb 25 02:47 /usr/local/cuda/lib64/libcudnn.so.5.1.10
-rw-r--r-- 1 root root 70364814 Feb 25 02:47 /usr/local/cuda/lib64/libcudnn_static.a

The GPU memory size is 11439MiB(Tesla K80).

[Cysu]

I suspect that our cmake config for CUDA is outdated and your GPU arch is currently not supported.
Could you please replace caffe/cmake/Cuda.cmake by the config from the official repo, remove the caffe/build directory, and recompile caffe from scratch?

[dichen-cd]

Hi, I suppose this isn't working. Using the official config would cause

make: *** No targets specified and no makefile found.  Stop.

when executing make.

I tried replace the whole caffe/cmake/ folder with the official one, but the error is same.

Yet the official repo can be compiled and runs well.

[njustzjw]

Hey,could you tell me how to get the dataset the author provided in README?Thank you very much!

[Cysu]

@DeanChan is it able to run with GPU now?

[dichen-cd]

Hi @Cysu Forgot this issue being opening here, sorry.

It is able to run on a Tesla K80 GPU now, without docker, which was where I compile the code. I still don't know why the issue happened, given the fact that the official caffe runs pretty well inside docker.

Anyway, I'll close this issue now. Many thanks for your help!

[dichen-cd]

@njustzjw You can request the dataset via e-mail:
Request the dataset from sli [at] ee.cuhk.edu.hk or xiaotong [at] ee.cuhk.edu.hk (academic only)

[njustzjw]

Many thanks for your links! I got it. I have another question。Do you write a new class similar to the psdb.py? At 2017-07-04 09:47:28, "陈狄" <notifications@github.com> wrote: @njustzjw You can request the dataset via e-mail: Request the dataset from sli [at] ee.cuhk.edu.hk or xiaotong [at] ee.cuhk.edu.hk (academic only) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[dichen-cd]

@njustzjw Didn't do that. Just trying to run the demo.

[njustzjw]

Then,haven't you trained your own data? At 2017-07-04 10:01:36, "陈狄" <notifications@github.com> wrote: @njustzjw Didn't do that. Just trying to run the demo. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[dichen-cd]

@njustzjw Nope.

[Cysu]

@DeanChan good to know that! Some library dependency might be different between ours and the official caffe. Thanks very much for pointing out this docker issue. I will check about it later when available.