make runtest stuck

[thomhert]

Hi Du Tran,
I checked several closed issues for "make runtest" but still couldn't find similar problem.
After doing make all and make test without error, I did "make runtest" but always got stuck at test point below. At first I thought it was because having two GPUs, then I do "make runtest CUDA_VISIBLE_DEVICES=0" or export CUDA_VISIBLE_DEVICES=0 before doing make runtest (based on here and here). However, i still got stuck at the same point.
Do you have any idea why I experienced this issue? Thank you.

Here is the test point that I got stuck:
[----------] 5 tests from Deconvolution3DLayerTest/1, where TypeParam = double
[ RUN ] Deconvolution3DLayerTest/1.TestSetup
[ OK ] Deconvolution3DLayerTest/1.TestSetup (0 ms)
[ RUN ] Deconvolution3DLayerTest/1.TestCPUGradient

or
[----------] 5 tests from Deconvolution3DLayerTest/1, where TypeParam = double
[ RUN ] Deconvolution3DLayerTest/1.TestSetup
[ OK ] Deconvolution3DLayerTest/1.TestSetup (0 ms)
[ RUN ] Deconvolution3DLayerTest/1.TestGPUSimpleDeconvolution3D
[ OK ] Deconvolution3DLayerTest/1.TestGPUSimpleDeconvolution3D (1 ms)
[ RUN ] Deconvolution3DLayerTest/1.TestCPUSimpleDeconvolution3D
[ OK ] Deconvolution3DLayerTest/1.TestCPUSimpleDeconvolution3D (1 ms)
[ RUN ] Deconvolution3DLayerTest/1.TestGPUGradient

---

I managed to solve the problem.. Thank you anyway.

[dutran]

@thomhert have you got your issue solved?

[thomhert]

@dutran Yes, I finally solved it. So far I could try examples/c3d_feature_extraction.
Well, I guess my problem was because multi GPUs. However, to make it sure, I will re-try make runtest with multi GPUs and comment here if problems come up.

[dutran]

glad to hear!

[weizhou-geek]

@thomhert ,I also met this problem. How did you solve it? Thanks!

[thomhert]

@weizhou-geek I assumed my problem was because there are two GPUs on my PC.
To solve this problem simply export CUDA_VISIBLE_DEVICES=0 before doing make runtest.
Follow these steps:

• make clean (cleaning up the previous one)
• make all -j $(($(nproc) + 1))
• make test -j $(($(nproc) + 1))
• export CUDA_VISIBLE_DEVICES=0
• make runtest -j $(($(nproc) + 1))
  it seems stuck when I met this test point: [ RUN ] Deconvolution3DLayerTest/1.TestGPUGradient, so i just left it. When I checked it out in the morning, all test were finished.
  Hope it helps.

[weizhou-geek]

@thomhert , Thanks for your guidance. I have tried your method, but still stuck at the following point:
[----------] 3 tests from ImageDataLayerTest/1, where TypeParam = double
[ RUN ] ImageDataLayerTest/1.TestResize
[ OK ] ImageDataLayerTest/1.TestResize (464 ms)
[ RUN ] ImageDataLayerTest/1.TestShuffle
[ OK ] ImageDataLayerTest/1.TestShuffle (4600 ms)
[ RUN ] ImageDataLayerTest/1.TestRead
[ OK ] ImageDataLayerTest/1.TestRead (4477 ms)
[----------] 3 tests from ImageDataLayerTest/1 (9541 ms total)

[----------] Global test environment tear-down
[==========] 417 tests from 74 test cases ran. (24272972 ms total)
[ PASSED ] 417 tests.

YOU HAVE 2 DISABLED TESTS

[thomhert]

@weizhou-geek what did you mean by stuck here? Because based on your comment, you passed 417 tests which means C3D is installed properly.

[weizhou-geek]

@thomhert But there still have two disabled tests.

[thomhert]

@weizhou-geek just ignore it, your installation is done. Follow C3D guide to test your installation.
you can also read about disable test here.

[weizhou-geek]

@thomhert Thanks! Have you finished the example? I follow the C3D user guide and after feature extraction and volume mean creation, I use the command "sh train_ucf101.sh" but the procedures has been aborted:
I1021 19:19:25.417078 22871 train_net.cpp:26] Starting Optimization
I1021 19:19:25.417163 22871 solver.cpp:41] Creating training net.
I1021 19:19:25.417635 22871 net.cpp:76] Creating Layer data
I1021 19:19:25.417650 22871 net.cpp:112] data -> data
I1021 19:19:25.417671 22871 net.cpp:112] data -> label
I1021 19:19:25.417687 22871 video_data_layer.cpp:283] Opening file ../c3d_finetuning/train_01.lst
I1021 19:19:25.461426 22871 video_data_layer.cpp:312] Shuffling data
I1021 19:19:25.779758 22871 video_data_layer.cpp:317] A total of 107258 video chunks.
I1021 19:19:25.779816 22871 video_data_layer.cpp:344] read video from /data/users/trandu/datasets/ucf101/frm/JumpingJack/v_JumpingJack_g25_c05/
F1021 19:19:25.779863 22871 video_data_layer.cpp:346] Check failed: ReadImageSequenceToVolumeDatum(file_list_[id].c_str(), start_frm_list_[id], label_list_[id], new_length, new_height, new_width, sampling_rate, &datum)
*** Check failure stack trace: ***
@ 0x7f80fa829daa (unknown)
@ 0x7f80fa829ce4 (unknown)
@ 0x7f80fa8296e6 (unknown)
@ 0x7f80fa82c687 (unknown)
@ 0x4b68e3 caffe::VideoDataLayer<>::SetUp()
@ 0x458d4e caffe::Net<>::Init()
@ 0x45a005 caffe::Net<>::Net()
@ 0x433f6f caffe::Solver<>::Init()
@ 0x43767b caffe::Solver<>::Solver()
@ 0x40b726 main
@ 0x7f80f600dec5 (unknown)
@ 0x40e671 (unknown)
@ (nil) (unknown)
Aborted (core dumped)

[thomhert]

@weizhou-geek you have the same issue to #132 #43 and i think you might not put your data path correctly.
see error here --> "read video from /data/users/trandu/datasets/ucf101/frm/JumpingJack/v_JumpingJack_g25_c05/" this is your wrong point.