Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OOM on a GTX 1060 with 6Gb of VRAM on CUDA 11.1 and cuDNN 8.04 on Ubuntu #2008

Open
kognat-docs opened this issue Sep 4, 2021 · 10 comments
Open

OOM on a GTX 1060 with 6Gb of VRAM on CUDA 11.1 and cuDNN 8.04 on Ubuntu #2008

kognat-docs opened this issue Sep 4, 2021 · 10 comments

Comments

@kognat-docs
Copy link

Issue Summary

Fresh build from the main branch is OOMing , yet almost 6Gb of VRAM is available

discobot@mirrorball:~/Documents/github/openpose$ ./build/examples/openpose/openpose.bin --camera 0
Starting OpenPose demo...
Configuring OpenPose...
Starting thread(s)...
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Auto-detecting all available GPUs... Detected 1 GPU(s), using 1 of them starting at GPU 0.
F0904 16:51:04.484730 32478 syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0)  out of memory
*** Check failure stack trace: ***
    @     0x7fb5c6d4c1c3  google::LogMessage::Fail()
    @     0x7fb5c6d5125b  google::LogMessage::SendToLog()
    @     0x7fb5c6d4bebf  google::LogMessage::Flush()
    @     0x7fb5c6d4c6ef  google::LogMessageFatal::~LogMessageFatal()
    @     0x7fb5c6a8ce2a  caffe::SyncedMemory::mutable_gpu_data()
    @     0x7fb5c69066a6  caffe::Blob<>::mutable_gpu_data()
    @     0x7fb5c6ac79ee  caffe::CuDNNConvolutionLayer<>::Forward_gpu()
    @     0x7fb5c6a4cb62  caffe::Net<>::ForwardFromTo()
    @     0x7fb5c7403364  op::NetCaffe::forwardPass()
    @     0x7fb5c74204aa  op::PoseExtractorCaffe::forwardPass()
    @     0x7fb5c741b52b  op::PoseExtractor::forwardPass()
    @     0x7fb5c7418fc8  op::WPoseExtractor<>::work()
    @     0x7fb5c744a2cf  op::Worker<>::checkAndWork()
    @     0x7fb5c744a45b  op::SubThread<>::workTWorkers()
    @     0x7fb5c745812d  op::SubThreadQueueInOut<>::work()
    @     0x7fb5c744f131  op::Thread<>::threadFunction()
    @     0x7fb5c708bde4  (unknown)
    @     0x7fb5c6d8c609  start_thread
    @     0x7fb5c6eca293  clone
Aborted (core dumped)

Memory

discobot@mirrorball:~/Documents/github/openpose$ nvidia-smi 
Sat Sep  4 16:51:11 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.84       Driver Version: 460.84       CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 106...  Off  | 00000000:0F:00.0  On |                  N/A |
|  0%   49C    P0    27W / 120W |    118MiB /  6077MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       965      G   /usr/lib/xorg/Xorg                 84MiB |
|    0   N/A  N/A      1285      G   /usr/bin/gnome-shell               30MiB |
+-----------------------------------------------------------------------------+

No missing libraries

discobot@mirrorball:~/Documents/github/openpose$ ldd ./build/examples/openpose/openpose.bin
	linux-vdso.so.1 (0x00007ffef01eb000)
	libopenpose.so.1.7.0 => /home/discobot/Documents/github/openpose/build/src/openpose/libopenpose.so.1.7.0 (0x00007faab0c71000)
	libgflags.so.2.2 => /lib/x86_64-linux-gnu/libgflags.so.2.2 (0x00007faab0c27000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007faab0a45000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007faab0a2a000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007faab0838000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007faab0813000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007faab080d000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007faab0802000)
	libglog.so.0 => /lib/x86_64-linux-gnu/libglog.so.0 (0x00007faab07cf000)
	libcaffe.so.1.0.0 => /home/discobot/Documents/github/openpose/build/caffe/lib/libcaffe.so.1.0.0 (0x00007faab0254000)
	libopencv_highgui.so.4.2 => /lib/x86_64-linux-gnu/libopencv_highgui.so.4.2 (0x00007faab023a000)
	libopencv_video.so.4.2 => /lib/x86_64-linux-gnu/libopencv_video.so.4.2 (0x00007faab01be000)
	libopencv_videoio.so.4.2 => /lib/x86_64-linux-gnu/libopencv_videoio.so.4.2 (0x00007faab013d000)
	libopencv_imgcodecs.so.4.2 => /lib/x86_64-linux-gnu/libopencv_imgcodecs.so.4.2 (0x00007faab00e4000)
	libopencv_objdetect.so.4.2 => /lib/x86_64-linux-gnu/libopencv_objdetect.so.4.2 (0x00007faab007e000)
	libopencv_calib3d.so.4.2 => /lib/x86_64-linux-gnu/libopencv_calib3d.so.4.2 (0x00007faaafeab000)
	libopencv_imgproc.so.4.2 => /lib/x86_64-linux-gnu/libopencv_imgproc.so.4.2 (0x00007faaaf904000)
	libopencv_core.so.4.2 => /lib/x86_64-linux-gnu/libopencv_core.so.4.2 (0x00007faaaf552000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007faaaf403000)
	/lib64/ld-linux-x86-64.so.2 (0x00007faab1137000)
	libunwind.so.8 => /lib/x86_64-linux-gnu/libunwind.so.8 (0x00007faaaf3e6000)
	libboost_thread.so.1.71.0 => /lib/x86_64-linux-gnu/libboost_thread.so.1.71.0 (0x00007faaaf3ba000)
	libboost_filesystem.so.1.71.0 => /lib/x86_64-linux-gnu/libboost_filesystem.so.1.71.0 (0x00007faaaf39c000)
	libprotobuf.so.17 => /lib/x86_64-linux-gnu/libprotobuf.so.17 (0x00007faaaf0a2000)
	libhdf5_serial.so.103 => /lib/x86_64-linux-gnu/libhdf5_serial.so.103 (0x00007faaaed25000)
	libhdf5_serial_hl.so.100 => /lib/x86_64-linux-gnu/libhdf5_serial_hl.so.100 (0x00007faaaecfe000)
	libcudart.so.11.0 => /usr/local/cuda/lib64/libcudart.so.11.0 (0x00007faaaea79000)
	libcurand.so.10 => /usr/local/cuda/lib64/libcurand.so.10 (0x00007faaa9b49000)
	libcublas.so.11 => /usr/local/cuda/lib64/libcublas.so.11 (0x00007faaa172d000)
	libcudnn.so.8 => /usr/local/cuda/lib64/libcudnn.so.8 (0x00007faaa1502000)
	libcblas.so.3 => /lib/x86_64-linux-gnu/libcblas.so.3 (0x00007faaa14dd000)
	libgtk-3.so.0 => /lib/x86_64-linux-gnu/libgtk-3.so.0 (0x00007faaa0d2c000)
	libgdk-3.so.0 => /lib/x86_64-linux-gnu/libgdk-3.so.0 (0x00007faaa0c27000)
	libcairo.so.2 => /lib/x86_64-linux-gnu/libcairo.so.2 (0x00007faaa0b04000)
	libgdk_pixbuf-2.0.so.0 => /lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x00007faaa0adc000)
	libgobject-2.0.so.0 => /lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007faaa0a7a000)
	libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007faaa0951000)
	libdc1394.so.22 => /lib/x86_64-linux-gnu/libdc1394.so.22 (0x00007faaa08d9000)
	libgstreamer-1.0.so.0 => /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0 (0x00007faaa0792000)
	libgstapp-1.0.so.0 => /lib/x86_64-linux-gnu/libgstapp-1.0.so.0 (0x00007faaa0780000)
	libgstriff-1.0.so.0 => /lib/x86_64-linux-gnu/libgstriff-1.0.so.0 (0x00007faaa076f000)
	libgstpbutils-1.0.so.0 => /lib/x86_64-linux-gnu/libgstpbutils-1.0.so.0 (0x00007faaa072f000)
	libavcodec.so.58 => /lib/x86_64-linux-gnu/libavcodec.so.58 (0x00007faa9f172000)
	libavformat.so.58 => /lib/x86_64-linux-gnu/libavformat.so.58 (0x00007faa9eefa000)
	libavutil.so.56 => /lib/x86_64-linux-gnu/libavutil.so.56 (0x00007faa9edd1000)
	libswscale.so.5 => /lib/x86_64-linux-gnu/libswscale.so.5 (0x00007faa9ed3c000)
	libjpeg.so.8 => /lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007faa9ecb7000)
	libwebp.so.6 => /lib/x86_64-linux-gnu/libwebp.so.6 (0x00007faa9ec4b000)
	libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007faa9ec13000)
	libgdcmMSFF.so.3.0 => /lib/x86_64-linux-gnu/libgdcmMSFF.so.3.0 (0x00007faa9e96d000)
	libtiff.so.5 => /lib/x86_64-linux-gnu/libtiff.so.5 (0x00007faa9e8ec000)
	libIlmImf-2_3.so.24 => /lib/x86_64-linux-gnu/libIlmImf-2_3.so.24 (0x00007faa9e60d000)
	libgdal.so.26 => /lib/libgdal.so.26 (0x00007faa9d286000)
	libgdcmDSED.so.3.0 => /lib/x86_64-linux-gnu/libgdcmDSED.so.3.0 (0x00007faa9d152000)
	libopencv_features2d.so.4.2 => /lib/x86_64-linux-gnu/libopencv_features2d.so.4.2 (0x00007faa9d07e000)
	libopencv_flann.so.4.2 => /lib/x86_64-linux-gnu/libopencv_flann.so.4.2 (0x00007faa9d01f000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007faa9d003000)
	libtbb.so.2 => /lib/x86_64-linux-gnu/libtbb.so.2 (0x00007faa9cfbd000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007faa9cf92000)
	libsz.so.2 => /lib/x86_64-linux-gnu/libsz.so.2 (0x00007faa9cf8d000)
	libcublasLt.so.11 => /usr/local/cuda/lib64/libcublasLt.so.11 (0x00007faa8ef99000)
	libatlas.so.3 => /lib/x86_64-linux-gnu/libatlas.so.3 (0x00007faa8ec0d000)
	libgmodule-2.0.so.0 => /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007faa8ec07000)
	libpangocairo-1.0.so.0 => /lib/x86_64-linux-gnu/libpangocairo-1.0.so.0 (0x00007faa8ebf3000)
	libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007faa8eab6000)
	libXi.so.6 => /lib/x86_64-linux-gnu/libXi.so.6 (0x00007faa8eaa4000)
	libXfixes.so.3 => /lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007faa8ea9c000)
	libcairo-gobject.so.2 => /lib/x86_64-linux-gnu/libcairo-gobject.so.2 (0x00007faa8ea90000)
	libatk-1.0.so.0 => /lib/x86_64-linux-gnu/libatk-1.0.so.0 (0x00007faa8ea66000)
	libatk-bridge-2.0.so.0 => /lib/x86_64-linux-gnu/libatk-bridge-2.0.so.0 (0x00007faa8ea2d000)
	libepoxy.so.0 => /lib/x86_64-linux-gnu/libepoxy.so.0 (0x00007faa8e8fa000)
	libfribidi.so.0 => /lib/x86_64-linux-gnu/libfribidi.so.0 (0x00007faa8e8dd000)
	libgio-2.0.so.0 => /lib/x86_64-linux-gnu/libgio-2.0.so.0 (0x00007faa8e6fc000)
	libpangoft2-1.0.so.0 => /lib/x86_64-linux-gnu/libpangoft2-1.0.so.0 (0x00007faa8e6e3000)
	libpango-1.0.so.0 => /lib/x86_64-linux-gnu/libpango-1.0.so.0 (0x00007faa8e694000)
	libharfbuzz.so.0 => /lib/x86_64-linux-gnu/libharfbuzz.so.0 (0x00007faa8e58d000)
	libfontconfig.so.1 => /lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007faa8e546000)
	libfreetype.so.6 => /lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007faa8e487000)
	libXinerama.so.1 => /lib/x86_64-linux-gnu/libXinerama.so.1 (0x00007faa8e482000)
	libXrandr.so.2 => /lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007faa8e475000)
	libXcursor.so.1 => /lib/x86_64-linux-gnu/libXcursor.so.1 (0x00007faa8e466000)
	libXcomposite.so.1 => /lib/x86_64-linux-gnu/libXcomposite.so.1 (0x00007faa8e461000)
	libXdamage.so.1 => /lib/x86_64-linux-gnu/libXdamage.so.1 (0x00007faa8e45c000)
	libxkbcommon.so.0 => /lib/x86_64-linux-gnu/libxkbcommon.so.0 (0x00007faa8e41a000)
	libwayland-cursor.so.0 => /lib/x86_64-linux-gnu/libwayland-cursor.so.0 (0x00007faa8e40f000)
	libwayland-egl.so.1 => /lib/x86_64-linux-gnu/libwayland-egl.so.1 (0x00007faa8e40a000)
	libwayland-client.so.0 => /lib/x86_64-linux-gnu/libwayland-client.so.0 (0x00007faa8e3f9000)
	libXext.so.6 => /lib/x86_64-linux-gnu/libXext.so.6 (0x00007faa8e3e2000)
	libpixman-1.so.0 => /lib/x86_64-linux-gnu/libpixman-1.so.0 (0x00007faa8e33b000)
	libxcb-shm.so.0 => /lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007faa8e336000)
	libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007faa8e30c000)
	libxcb-render.so.0 => /lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007faa8e2fd000)
	libXrender.so.1 => /lib/x86_64-linux-gnu/libXrender.so.1 (0x00007faa8e0f1000)
	libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007faa8e0e5000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007faa8e072000)
	libraw1394.so.11 => /lib/x86_64-linux-gnu/libraw1394.so.11 (0x00007faa8de63000)
	libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007faa8de47000)
	libgstbase-1.0.so.0 => /lib/x86_64-linux-gnu/libgstbase-1.0.so.0 (0x00007faa8ddc9000)
	libgstaudio-1.0.so.0 => /lib/x86_64-linux-gnu/libgstaudio-1.0.so.0 (0x00007faa8dd4a000)
	libgsttag-1.0.so.0 => /lib/x86_64-linux-gnu/libgsttag-1.0.so.0 (0x00007faa8dd0c000)
	libgstvideo-1.0.so.0 => /lib/x86_64-linux-gnu/libgstvideo-1.0.so.0 (0x00007faa8dc5b000)
	libswresample.so.3 => /lib/x86_64-linux-gnu/libswresample.so.3 (0x00007faa8dc39000)
	libvpx.so.6 => /lib/x86_64-linux-gnu/libvpx.so.6 (0x00007faa8da01000)
	libwebpmux.so.3 => /lib/x86_64-linux-gnu/libwebpmux.so.3 (0x00007faa8d9f5000)
	librsvg-2.so.2 => /lib/x86_64-linux-gnu/librsvg-2.so.2 (0x00007faa8d0c8000)
	libzvbi.so.0 => /lib/x86_64-linux-gnu/libzvbi.so.0 (0x00007faa8d039000)
	libsnappy.so.1 => /lib/x86_64-linux-gnu/libsnappy.so.1 (0x00007faa8d02e000)
	libaom.so.0 => /lib/x86_64-linux-gnu/libaom.so.0 (0x00007faa8cb90000)
	libcodec2.so.0.9 => /lib/x86_64-linux-gnu/libcodec2.so.0.9 (0x00007faa8bdac000)
	libgsm.so.1 => /lib/x86_64-linux-gnu/libgsm.so.1 (0x00007faa8bd9d000)
	libmp3lame.so.0 => /lib/x86_64-linux-gnu/libmp3lame.so.0 (0x00007faa8bd23000)
	libopenjp2.so.7 => /lib/x86_64-linux-gnu/libopenjp2.so.7 (0x00007faa8bccd000)
	libopus.so.0 => /lib/x86_64-linux-gnu/libopus.so.0 (0x00007faa8bc6e000)
	libshine.so.3 => /lib/x86_64-linux-gnu/libshine.so.3 (0x00007faa8ba63000)
	libspeex.so.1 => /lib/x86_64-linux-gnu/libspeex.so.1 (0x00007faa8ba45000)
	libtheoraenc.so.1 => /lib/x86_64-linux-gnu/libtheoraenc.so.1 (0x00007faa8ba08000)
	libtheoradec.so.1 => /lib/x86_64-linux-gnu/libtheoradec.so.1 (0x00007faa8b9e6000)
	libtwolame.so.0 => /lib/x86_64-linux-gnu/libtwolame.so.0 (0x00007faa8b9c0000)
	libvorbis.so.0 => /lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007faa8b992000)
	libvorbisenc.so.2 => /lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007faa8b8e7000)
	libwavpack.so.1 => /lib/x86_64-linux-gnu/libwavpack.so.1 (0x00007faa8b8bb000)
	libx264.so.155 => /lib/x86_64-linux-gnu/libx264.so.155 (0x00007faa8b5fd000)
	libx265.so.179 => /lib/x86_64-linux-gnu/libx265.so.179 (0x00007faa8a68d000)
	libxvidcore.so.4 => /lib/x86_64-linux-gnu/libxvidcore.so.4 (0x00007faa8a57a000)
	libva.so.2 => /lib/x86_64-linux-gnu/libva.so.2 (0x00007faa8a551000)
	libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x00007faa8a397000)
	libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007faa8a384000)
	libgme.so.0 => /lib/x86_64-linux-gnu/libgme.so.0 (0x00007faa8a335000)
	libopenmpt.so.0 => /lib/x86_64-linux-gnu/libopenmpt.so.0 (0x00007faa8a148000)
	libchromaprint.so.1 => /lib/x86_64-linux-gnu/libchromaprint.so.1 (0x00007faa8a132000)
	libbluray.so.2 => /lib/x86_64-linux-gnu/libbluray.so.2 (0x00007faa8a0e1000)
	libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007faa89f0b000)
	libssh-gcrypt.so.4 => /lib/x86_64-linux-gnu/libssh-gcrypt.so.4 (0x00007faa89e7f000)
	libva-drm.so.2 => /lib/x86_64-linux-gnu/libva-drm.so.2 (0x00007faa89e78000)
	libva-x11.so.2 => /lib/x86_64-linux-gnu/libva-x11.so.2 (0x00007faa89e70000)
	libvdpau.so.1 => /lib/x86_64-linux-gnu/libvdpau.so.1 (0x00007faa89e6a000)
	libdrm.so.2 => /lib/x86_64-linux-gnu/libdrm.so.2 (0x00007faa89e56000)
	libOpenCL.so.1 => /usr/local/cuda/lib64/libOpenCL.so.1 (0x00007faa89c4f000)
	libgdcmDICT.so.3.0 => /lib/x86_64-linux-gnu/libgdcmDICT.so.3.0 (0x00007faa8999d000)
	libgdcmjpeg8.so.3.0 => /lib/x86_64-linux-gnu/libgdcmjpeg8.so.3.0 (0x00007faa89973000)
	libgdcmjpeg12.so.3.0 => /lib/x86_64-linux-gnu/libgdcmjpeg12.so.3.0 (0x00007faa89949000)
	libgdcmjpeg16.so.3.0 => /lib/x86_64-linux-gnu/libgdcmjpeg16.so.3.0 (0x00007faa8991f000)
	libCharLS.so.2 => /lib/x86_64-linux-gnu/libCharLS.so.2 (0x00007faa898d0000)
	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007faa898c7000)
	libjson-c.so.4 => /lib/x86_64-linux-gnu/libjson-c.so.4 (0x00007faa898b5000)
	libgdcmIOD.so.3.0 => /lib/x86_64-linux-gnu/libgdcmIOD.so.3.0 (0x00007faa8989b000)
	libgdcmCommon.so.3.0 => /lib/x86_64-linux-gnu/libgdcmCommon.so.3.0 (0x00007faa89871000)
	libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007faa897c8000)
	libjbig.so.0 => /lib/x86_64-linux-gnu/libjbig.so.0 (0x00007faa895ba000)
	libHalf.so.24 => /lib/x86_64-linux-gnu/libHalf.so.24 (0x00007faa89575000)
	libIex-2_3.so.24 => /lib/x86_64-linux-gnu/libIex-2_3.so.24 (0x00007faa89552000)
	libIlmThread-2_3.so.24 => /lib/x86_64-linux-gnu/libIlmThread-2_3.so.24 (0x00007faa89548000)
	libarmadillo.so.9 => /lib/libarmadillo.so.9 (0x00007faa89535000)
	libpoppler.so.97 => /lib/x86_64-linux-gnu/libpoppler.so.97 (0x00007faa891f0000)
	libfreexl.so.1 => /lib/x86_64-linux-gnu/libfreexl.so.1 (0x00007faa891e5000)
	libqhull.so.7 => /lib/x86_64-linux-gnu/libqhull.so.7 (0x00007faa88f8b000)
	libgeos_c.so.1 => /lib/x86_64-linux-gnu/libgeos_c.so.1 (0x00007faa88f48000)
	libepsilon.so.1 => /lib/x86_64-linux-gnu/libepsilon.so.1 (0x00007faa88f2c000)
	libodbc.so.2 => /lib/x86_64-linux-gnu/libodbc.so.2 (0x00007faa88eba000)
	libodbcinst.so.2 => /lib/x86_64-linux-gnu/libodbcinst.so.2 (0x00007faa88ea2000)
	libkmlbase.so.1 => /lib/x86_64-linux-gnu/libkmlbase.so.1 (0x00007faa88e85000)
	libkmldom.so.1 => /lib/x86_64-linux-gnu/libkmldom.so.1 (0x00007faa88de4000)
	libkmlengine.so.1 => /lib/x86_64-linux-gnu/libkmlengine.so.1 (0x00007faa88daa000)
	libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007faa88d7a000)
	libxerces-c-3.2.so => /lib/x86_64-linux-gnu/libxerces-c-3.2.so (0x00007faa889e7000)
	libnetcdf.so.15 => /lib/x86_64-linux-gnu/libnetcdf.so.15 (0x00007faa888c2000)
	libmfhdfalt.so.0 => /lib/libmfhdfalt.so.0 (0x00007faa88898000)
	libdfalt.so.0 => /lib/libdfalt.so.0 (0x00007faa887f3000)
	libogdi.so.4.1 => /lib/libogdi.so.4.1 (0x00007faa887d7000)
	libgif.so.7 => /lib/x86_64-linux-gnu/libgif.so.7 (0x00007faa887ca000)
	libgeotiff.so.5 => /lib/x86_64-linux-gnu/libgeotiff.so.5 (0x00007faa88795000)
	libcfitsio.so.8 => /lib/x86_64-linux-gnu/libcfitsio.so.8 (0x00007faa88496000)
	libpq.so.5 => /lib/x86_64-linux-gnu/libpq.so.5 (0x00007faa88446000)
	libproj.so.15 => /lib/x86_64-linux-gnu/libproj.so.15 (0x00007faa88157000)
	libdapclient.so.6 => /lib/x86_64-linux-gnu/libdapclient.so.6 (0x00007faa8810f000)
	libdap.so.25 => /lib/x86_64-linux-gnu/libdap.so.25 (0x00007faa87f6f000)
	libspatialite.so.7 => /lib/x86_64-linux-gnu/libspatialite.so.7 (0x00007faa879e0000)
	libcurl-gnutls.so.4 => /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007faa87951000)
	libfyba.so.0 => /lib/x86_64-linux-gnu/libfyba.so.0 (0x00007faa878f9000)
	libmysqlclient.so.21 => /lib/x86_64-linux-gnu/libmysqlclient.so.21 (0x00007faa87202000)
	libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007faa86f2c000)
	libaec.so.0 => /lib/x86_64-linux-gnu/libaec.so.0 (0x00007faa86f21000)
	libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007faa86ed0000)
	libatspi.so.0 => /lib/x86_64-linux-gnu/libatspi.so.0 (0x00007faa86e99000)
	libmount.so.1 => /lib/x86_64-linux-gnu/libmount.so.1 (0x00007faa86e39000)
	libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007faa86e0c000)
	libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007faa86df0000)
	libthai.so.0 => /lib/x86_64-linux-gnu/libthai.so.0 (0x00007faa86de5000)
	libgraphite2.so.3 => /lib/x86_64-linux-gnu/libgraphite2.so.3 (0x00007faa86db8000)
	libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007faa86db2000)
	libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007faa86da8000)
	libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007faa86d7b000)
	liborc-0.4.so.0 => /lib/x86_64-linux-gnu/liborc-0.4.so.0 (0x00007faa86cf8000)
	libsoxr.so.0 => /lib/x86_64-linux-gnu/libsoxr.so.0 (0x00007faa86c8d000)
	libogg.so.0 => /lib/x86_64-linux-gnu/libogg.so.0 (0x00007faa86c7e000)
	libnuma.so.1 => /lib/x86_64-linux-gnu/libnuma.so.1 (0x00007faa86c71000)
	libicuuc.so.66 => /lib/x86_64-linux-gnu/libicuuc.so.66 (0x00007faa86a8b000)
	libmpg123.so.0 => /lib/x86_64-linux-gnu/libmpg123.so.0 (0x00007faa86a2b000)
	libvorbisfile.so.3 => /lib/x86_64-linux-gnu/libvorbisfile.so.3 (0x00007faa86a20000)
	libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007faa868e8000)
	libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007faa868c7000)
	libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007faa86745000)
	libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007faa8672f000)
	libnettle.so.7 => /lib/x86_64-linux-gnu/libnettle.so.7 (0x00007faa866f5000)
	libhogweed.so.5 => /lib/x86_64-linux-gnu/libhogweed.so.5 (0x00007faa866be000)
	libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007faa86638000)
	libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007faa8651a000)
	libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007faa864f7000)
	libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007faa864aa000)
	libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x00007faa860e2000)
	liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3 (0x00007faa85a1a000)
	libarpack.so.2 => /lib/x86_64-linux-gnu/libarpack.so.2 (0x00007faa859d2000)
	libsuperlu.so.5 => /lib/x86_64-linux-gnu/libsuperlu.so.5 (0x00007faa85962000)
	liblcms2.so.2 => /lib/x86_64-linux-gnu/liblcms2.so.2 (0x00007faa85907000)
	libnss3.so => /lib/x86_64-linux-gnu/libnss3.so (0x00007faa857b8000)
	libsmime3.so => /lib/x86_64-linux-gnu/libsmime3.so (0x00007faa85786000)
	libnspr4.so => /lib/x86_64-linux-gnu/libnspr4.so (0x00007faa85746000)
	libgeos-3.8.0.so => /lib/x86_64-linux-gnu/libgeos-3.8.0.so (0x00007faa8557d000)
	libltdl.so.7 => /lib/x86_64-linux-gnu/libltdl.so.7 (0x00007faa85572000)
	libminizip.so.1 => /lib/x86_64-linux-gnu/libminizip.so.1 (0x00007faa85367000)
	liburiparser.so.1 => /lib/x86_64-linux-gnu/liburiparser.so.1 (0x00007faa85346000)
	libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007faa852b3000)
	libldap_r-2.4.so.2 => /lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007faa8525d000)
	libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007faa85134000)
	libnghttp2.so.14 => /lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007faa85109000)
	librtmp.so.1 => /lib/x86_64-linux-gnu/librtmp.so.1 (0x00007faa850e9000)
	libssh.so.4 => /lib/x86_64-linux-gnu/libssh.so.4 (0x00007faa8507b000)
	libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007faa85068000)
	liblber-2.4.so.2 => /lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007faa85057000)
	libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007faa85049000)
	libfyut.so.0 => /lib/x86_64-linux-gnu/libfyut.so.0 (0x00007faa8503b000)
	libfygm.so.0 => /lib/x86_64-linux-gnu/libfygm.so.0 (0x00007faa85032000)
	libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007faa84f83000)
	libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007faa84f2c000)
	libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007faa84e9c000)
	libdatrie.so.1 => /lib/x86_64-linux-gnu/libdatrie.so.1 (0x00007faa84e90000)
	libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007faa84e76000)
	libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007faa84e34000)
	libicudata.so.66 => /lib/x86_64-linux-gnu/libicudata.so.66 (0x00007faa83373000)
	libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007faa83296000)
	libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007faa83263000)
	libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007faa8325c000)
	libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007faa8324d000)
	libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007faa82f85000)
	libnssutil3.so => /lib/x86_64-linux-gnu/libnssutil3.so (0x00007faa82f52000)
	libplc4.so => /lib/x86_64-linux-gnu/libplc4.so (0x00007faa82f49000)
	libplds4.so => /lib/x86_64-linux-gnu/libplds4.so (0x00007faa82f44000)
	libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007faa82f27000)
	libgssapi.so.3 => /lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007faa82ee2000)
	libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007faa82ebf000)
	liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007faa82e9c000)
	libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007faa82e95000)
	libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007faa82e4b000)
	libheimntlm.so.0 => /lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007faa82e3f000)
	libkrb5.so.26 => /lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007faa82dac000)
	libasn1.so.8 => /lib/x86_64-linux-gnu/libasn1.so.8 (0x00007faa82d03000)
	libhcrypto.so.4 => /lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007faa82ccb000)
	libroken.so.18 => /lib/x86_64-linux-gnu/libroken.so.18 (0x00007faa82cb2000)
	libwind.so.0 => /lib/x86_64-linux-gnu/libwind.so.0 (0x00007faa82c88000)
	libheimbase.so.1 => /lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007faa82c76000)
	libhx509.so.5 => /lib/x86_64-linux-gnu/libhx509.so.5 (0x00007faa82c26000)
	libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007faa82beb000)

Executed Command (if any)

Note: add --logging_level 0 --disable_multi_thread to get higher debug information.

$ ./build/examples/openpose/openpose.bin --camera 0 --logging_level 0 --disable_multi_thread

OpenPose Output (if any)

Starting OpenPose demo...
Configuring OpenPose...
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToProducer():210
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToProducerType():170
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToPoseMode():11
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToPoseModel():33
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToScaleMode():82
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToHeatMapScaleMode():116
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToDetector():147
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToDetector():147
Starting thread(s)...
Running configureThreadManager...
/home/discobot/Documents/github/openpose/src/openpose/producer/producer.cpp:createProducer():417
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
renderModePose = 3
renderModeFace = 3
renderModeHand = 3
renderOutput = 1
renderOutputGpu = 1
renderFace = 0
renderHand = 0
renderHandGpu = 0
/home/discobot/Documents/github/openpose/src/openpose/wrapper/wrapperAuxiliary.cpp:wrapperConfigureSanityChecks():17
/home/discobot/Documents/github/openpose/src/openpose/wrapper/wrapperAuxiliary.cpp:wrapperConfigureSanityChecks():192
userInputAndPreprocessingWsEmpty = 1
userOutputWsEmpty = 1
numberGpuThreads = -1
gpuNumberStart = 0
Auto-detecting all available GPUs... Detected 1 GPU(s), using 1 of them starting at GPU 0.
writeImagesCleaned = 
writeKeypointCleaned = 
writeJsonCleaned = 
writeHeatMapsCleaned = 
modelFolder = models/
finalOutputSize = [1280,720]
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():370
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():435
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():440
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():451
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():505
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():557
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():601
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():645
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():662
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():679
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():698
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():714
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():747
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():772
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():781
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():797
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():798
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():807
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():824
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():844
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():865
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():874
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():878
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():884
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():943
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():959
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():965
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1041
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1075
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1179
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1203
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1210
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapper.hpp:exec():419
/home/discobot/Documents/github/openpose/include/openpose/thread/threadManager.hpp:exec():185
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/threadManager.hpp:exec():190
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:threadFunction():182
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:initializationOnThread():167
Starting initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():172
Finished initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():191
Starting initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseGpuRenderer.cpp:initializationOnThread():74
Finished initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseGpuRenderer.cpp:initializationOnThread():84
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:threadFunction():185
/home/discobot/Documents/github/openpose/include/openpose/producer/wDatumProducer.hpp:workProducer():64
/home/discobot/Documents/github/openpose/include/openpose/producer/wDatumProducer.hpp:workProducer():81
/home/discobot/Documents/github/openpose/include/openpose/thread/wIdGenerator.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/thread/wIdGenerator.hpp:work():78
/home/discobot/Documents/github/openpose/include/openpose/core/wScaleAndSizeExtractor.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/core/wScaleAndSizeExtractor.hpp:work():76
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpInput.hpp:work():61
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpInput.hpp:work():72
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpOutput.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpOutput.hpp:work():75
/home/discobot/Documents/github/openpose/include/openpose/pose/wPoseExtractor.hpp:work():69
F0904 16:57:44.499598 32903 syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0)  out of memory
*** Check failure stack trace: ***
    @     0x7f960b43a1c3  google::LogMessage::Fail()
    @     0x7f960b43f25b  google::LogMessage::SendToLog()
    @     0x7f960b439ebf  google::LogMessage::Flush()
    @     0x7f960b43a6ef  google::LogMessageFatal::~LogMessageFatal()
    @     0x7f960b17ae2a  caffe::SyncedMemory::mutable_gpu_data()
    @     0x7f960aff46a6  caffe::Blob<>::mutable_gpu_data()
    @     0x7f960baf1331  op::NetCaffe::forwardPass()
    @     0x7f960bb0e4aa  op::PoseExtractorCaffe::forwardPass()
    @     0x7f960bb0952b  op::PoseExtractor::forwardPass()
    @     0x7f960bb06fc8  op::WPoseExtractor<>::work()
    @     0x7f960bb382cf  op::Worker<>::checkAndWork()
    @     0x7f960bb3845b  op::SubThread<>::workTWorkers()
    @     0x7f960bb4612d  op::SubThreadQueueInOut<>::work()
    @     0x7f960bb3d131  op::Thread<>::threadFunction()
    @     0x7f960bb3d4d5  op::Thread<>::exec()
    @     0x7f960bb40194  op::ThreadManager<>::exec()
    @     0x7f960bb9909e  op::WrapperT<>::exec()
    @     0x55e271d4322c  openPoseDemo()
    @     0x55e271d3f6c7  main
    @     0x7f960b4bd0b3  __libc_start_main
    @     0x55e271d3f6fe  _start
Aborted (core dumped)

Errors (if any)

OOM

Type of Issue

Select the topic(s) on your post, delete the rest:

  • Execution error
  • Help wanted

Your System Configuration

  1. Whole console output
Starting OpenPose demo...
Configuring OpenPose...
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToProducer():210
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToProducerType():170
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToPoseMode():11
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToPoseModel():33
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToScaleMode():82
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToHeatMapScaleMode():116
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToDetector():147
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToDetector():147
Starting thread(s)...
Running configureThreadManager...
/home/discobot/Documents/github/openpose/src/openpose/producer/producer.cpp:createProducer():417
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
renderModePose = 3
renderModeFace = 3
renderModeHand = 3
renderOutput = 1
renderOutputGpu = 1
renderFace = 0
renderHand = 0
renderHandGpu = 0
/home/discobot/Documents/github/openpose/src/openpose/wrapper/wrapperAuxiliary.cpp:wrapperConfigureSanityChecks():17
/home/discobot/Documents/github/openpose/src/openpose/wrapper/wrapperAuxiliary.cpp:wrapperConfigureSanityChecks():192
userInputAndPreprocessingWsEmpty = 1
userOutputWsEmpty = 1
numberGpuThreads = -1
gpuNumberStart = 0
Auto-detecting all available GPUs... Detected 1 GPU(s), using 1 of them starting at GPU 0.
writeImagesCleaned = 
writeKeypointCleaned = 
writeJsonCleaned = 
writeHeatMapsCleaned = 
modelFolder = models/
finalOutputSize = [1280,720]
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():370
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():435
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():440
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():451
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():505
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():557
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():601
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():645
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():662
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():679
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():698
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():714
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():747
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():772
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():781
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():797
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():798
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():807
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():824
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():844
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():865
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():874
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():878
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():884
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():943
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():959
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():965
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1041
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1075
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1179
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1203
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1210
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapper.hpp:exec():419
/home/discobot/Documents/github/openpose/include/openpose/thread/threadManager.hpp:exec():185
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/threadManager.hpp:exec():190
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:threadFunction():182
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:initializationOnThread():167
Starting initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():172
Finished initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():191
Starting initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseGpuRenderer.cpp:initializationOnThread():74
Finished initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseGpuRenderer.cpp:initializationOnThread():84
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:threadFunction():185
/home/discobot/Documents/github/openpose/include/openpose/producer/wDatumProducer.hpp:workProducer():64
/home/discobot/Documents/github/openpose/include/openpose/producer/wDatumProducer.hpp:workProducer():81
/home/discobot/Documents/github/openpose/include/openpose/thread/wIdGenerator.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/thread/wIdGenerator.hpp:work():78
/home/discobot/Documents/github/openpose/include/openpose/core/wScaleAndSizeExtractor.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/core/wScaleAndSizeExtractor.hpp:work():76
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpInput.hpp:work():61
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpInput.hpp:work():72
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpOutput.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpOutput.hpp:work():75
/home/discobot/Documents/github/openpose/include/openpose/pose/wPoseExtractor.hpp:work():69
F0904 16:57:44.499598 32903 syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0)  out of memory
*** Check failure stack trace: ***
    @     0x7f960b43a1c3  google::LogMessage::Fail()
    @     0x7f960b43f25b  google::LogMessage::SendToLog()
    @     0x7f960b439ebf  google::LogMessage::Flush()
    @     0x7f960b43a6ef  google::LogMessageFatal::~LogMessageFatal()
    @     0x7f960b17ae2a  caffe::SyncedMemory::mutable_gpu_data()
    @     0x7f960aff46a6  caffe::Blob<>::mutable_gpu_data()
    @     0x7f960baf1331  op::NetCaffe::forwardPass()
    @     0x7f960bb0e4aa  op::PoseExtractorCaffe::forwardPass()
    @     0x7f960bb0952b  op::PoseExtractor::forwardPass()
    @     0x7f960bb06fc8  op::WPoseExtractor<>::work()
    @     0x7f960bb382cf  op::Worker<>::checkAndWork()
    @     0x7f960bb3845b  op::SubThread<>::workTWorkers()
    @     0x7f960bb4612d  op::SubThreadQueueInOut<>::work()
    @     0x7f960bb3d131  op::Thread<>::threadFunction()
    @     0x7f960bb3d4d5  op::Thread<>::exec()
    @     0x7f960bb40194  op::ThreadManager<>::exec()
    @     0x7f960bb9909e  op::WrapperT<>::exec()
    @     0x55e271d4322c  openPoseDemo()
    @     0x55e271d3f6c7  main
    @     0x7f960b4bd0b3  __libc_start_main
    @     0x55e271d3f6fe  _start
Aborted (core dumped)

  1. OpenPose version: Latest GitHub code? latest b1cb2b6

  2. General configuration:

    • Installation mode: CMake, Linux Makefile

    • Operating system (lsb_release -a in Ubuntu):
      Distributor ID: Ubuntu
      Description: Ubuntu 20.04.2 LTS
      Release: 20.04
      Codename: focal

    • Operating system version : Ubuntu 20.04.2 LTS

    • Release or Debug mode: Release

    • Compiler : gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

  3. Non-default settings:
    All default

  4. 3rd-party software:

    • Caffe version: Default from OpenPose, 1.0.0
    • CMake version : cmake version 3.16.3
    • OpenCV version: pre-compiled apt-get install libopencv-dev

  5. If GPU mode issue:

    • CUDA version: 11.1.1
    • cuDNN version: 8.0.4
    • GPU model (nvidia-smi in Ubuntu): Geforce GTX 1060 6Gb

  6. If CPU-only mode issue:

    • CPU brand & model: Intel(R) Xeon(R) CPU X5650 @ 2.67GHz 8 cores
    • Total RAM memory available: 24Gb

  7. If Python API:

    • Python version: 3.8.10
    • Numpy version 1.21.2

  8. If Windows system:

    • Portable demo or compiled library?

  9. If speed performance issue:
@kognat-docs
Copy link
Author

Never mind , I will try v1.6.0 on CUDA 10.1 and CuDNN 7.6 and see if the memory use is better it runs with --network_resolution "-1x128" but it is jittery

@424778940z
Copy link

Try turn of blending

@AlexLuya
Copy link

Still got out of memory after turning off blending by --disable_blending

@424778940z
Copy link

Try cudnn 7.x, don't remember exact version.
8.x seems take lots of vram

@AlexLuya
Copy link

Thanks,it seems like no suitable cuda+cudnn 7x for ubuntu 21.04.Any clue about this huge memory consumptions.

@424778940z
Copy link

well nvidia cudnn 8.x currently supported to 20.04. you are using an unsupported OS.
Try install Ubuntu 20.04, or Xubuntu to save some vram since you only have 6g...

@kognat-docs
Copy link
Author

You need to choose an older version of gcc/g++ to be compatible with nvcc in cudnn 7.x and cuda 10.x if you have an ampere card with only 6Gb of VRAM you might be in dire straits.

@AlexLuya
Copy link

Anyone know the inside problem which cause such bigger memory comsuption?

@stale
Copy link

stale bot commented Jan 9, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale/old label Jan 9, 2022
@LeroyR
Copy link

LeroyR commented Dec 7, 2023 •
edited

Our setup with Openpose 1.5.1 uses <2gb (cuda 10.1.105, cudnn 7.6)
Openpose(master) consumes 4.4GB for the same task, same models (cuda 11.6, cudnn 8.4.1)

@stale stale bot removed the stale/old label Dec 7, 2023
@LeroyR LeroyR mentioned this issue Dec 7, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@AlexLuya @424778940z @LeroyR @kognat-docs