It seems like out of memory

[lfxxxxx]

I have compiled rtpose-caffe with cuDNN. (I have set USE_CUDNN:=1)
but when I test in just 2 image ,it show "out of memory"

[lfx@gpu-com image]$ ls
image2.jpg image3.jpg

[lfx@gpu-com caffe_rtpose-master]$ ./build/examples/rtpose/rtpose.bin --image_dir image/
F0407 10:27:17.102958 1129 syncedmem.cpp:64] Check failed: error == cudaSuccess (2 vs. 0) out of memory
*** Check failure stack trace: ***
@ 0x7efeaa736e6d (unknown)
@ 0x7efeaa738ced (unknown)
@ 0x7efeaa736a5c (unknown)
@ 0x7efeaa73963e (unknown)
@ 0x7efeb1905912 caffe::SyncedMemory::to_gpu()
@ 0x7efeb1904c79 caffe::SyncedMemory::gpu_data()
@ 0x7efeb177e762 caffe::Blob<>::gpu_data()
@ 0x7efeb197b9d3 caffe::CuDNNConvolutionLayer<>::Forward_gpu()
@ 0x7efeb18cdb8b caffe::Net<>::ForwardFromTo()
@ 0x40bcff warmup()
@ 0x411cbb processFrame()
@ 0x7efe9ed99dc5 start_thread
@ 0x7efe9eac873d __clone
Aborted (core dumped)

I have two GPUs(GT730+TITAN X), it seems already run in TITAN X, but i can't make sure

how can I make sure it run in TITAN X?
If it already run in TITAN X, how can i fix the error: out of memory?

[lfxxxxx]

[lfx@gpu-com image]$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Tue_Jan_10_13:22:03_CST_2017
Cuda compilation tools, release 8.0, V8.0.61

[gineshidalgo99]

To be sure in which Nvidia you are running it, open 2 terminals, halp screen each one. Then run in one of them:
watch -n 0.1 nvidia-smi

Execute our program from the other terminal and look to the terminal with the Nvidia information. Check which GPU is loading memory with the name of our program.

Let me know if my steps were not clear enough. Thanks!

[lfxxxxx]

Thanks! I finally find it run in the first GPU(GT730)
Now, another question is coming up ,how can I choose the second gpu to run this

In origin caffe , I can use the flag -gpu to choose, but caffe-rtpose seems haven't this choice?(only num_gpu)

[gineshidalgo99]

Sorry if its a little bit confusing, there is another flag for that, check --help to see the options. It was something like: device_start or start_device. We will change the name back to gpu_start in our next version.