test_data_layer.bin fails

[drahozin]

Issue summary

while running "make runtest" various errors are experienced
The error looks like "floating"...

================= ERROR 1 ======================================
[----------] 4 tests from NetUpgradeTest
[ RUN ] NetUpgradeTest.TestAllParams
[ OK ] NetUpgradeTest.TestAllParams (1 ms)
[ RUN ] NetUpgradeTest.TestUpgradeV1LayerType
*** Error in `.build_release/test/test_all.testbin': munmap_chunk(): invalid pointer: 0x0000000000f00fe0 ***
*** Aborted at 1483030739 (unix time) try "date -d @1483030739" if you are using GNU date ***
PC: @ 0x7f70b5875267 (unknown)
*** SIGABRT (@0x3e800000bb1) received by PID 2993 (TID 0x7f70b895ba40) from PID 2993; stack trace: ***
@ 0x7f70b5c1ad10 (unknown)
@ 0x7f70b5875267 (unknown)
@ 0x7f70b5876eca (unknown)
@ 0x7f70b58b8c53 (unknown)
@ 0x7f70b58c49f8 (unknown)
@ 0x7f70b64c426b caffe::BasePrefetchingDataLayer<>::~BasePrefetchingDataLayer()
@ 0x7f70b652b562 boost::detail::sp_counted_impl_p<>::dispose()
@ 0x65e589 caffe::NetUpgradeTest_TestUpgradeV1LayerType_Test::TestBody()
@ 0x726993 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x71eaca testing::Test::Run()
@ 0x71ec18 testing::TestInfo::Run()
@ 0x71ecf5 testing::TestCase::Run()
@ 0x71f688 testing::internal::UnitTestImpl::RunAllTests()
@ 0x71f953 testing::UnitTest::Run()
@ 0x45aec2 main
@ 0x7f70b5860a40 (unknown)
@ 0x461fc9 _start
@ 0x0 (unknown)
Aborted (core dumped)
Makefile:526: recipe for target 'runtest' failed

========================= ERROR 2 ================================
[----------] 12 tests from DataLayerTest/0, where TypeParam = caffe::CPUDevice
[ RUN ] DataLayerTest/0.TestReshapeLevelDB
*** Aborted at 1483030926 (unix time) try "date -d @1483030926" if you are using GNU date ***
PC: @ 0x7fce07cae29b caffe::BasePrefetchingDataLayer<>::~BasePrefetchingDataLayer()
*** SIGSEGV (@0x18) received by PID 3372 (TID 0x7fce0a145a40) from PID 24; stack trace: ***
@ 0x7fce07404d10 (unknown)
@ 0x7fce07cae29b caffe::BasePrefetchingDataLayer<>::~BasePrefetchingDataLayer()
@ 0x5e86c8 caffe::DataLayerTest<>::TestReshape()
@ 0x726993 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x71eaca testing::Test::Run()
@ 0x71ec18 testing::TestInfo::Run()
@ 0x71ecf5 testing::TestCase::Run()
@ 0x71f688 testing::internal::UnitTestImpl::RunAllTests()
@ 0x71f953 testing::UnitTest::Run()
@ 0x45aec2 main
@ 0x7fce0704aa40 (unknown)
@ 0x461fc9 _start
@ 0x0 (unknown)
Segmentation fault (core dumped)
Makefile:526: recipe for target 'runtest' failed
make: *** [runtest] Error 139

======================= ERROR 3 =======================================
[----------] 12 tests from DataLayerTest/0, where TypeParam = caffe::CPUDevice
[ RUN ] DataLayerTest/0.TestReshapeLevelDB
*** Aborted at 1483030926 (unix time) try "date -d @1483030926" if you are using GNU date ***
PC: @ 0x7fce07cae29b caffe::BasePrefetchingDataLayer<>::~BasePrefetchingDataLayer()
*** SIGSEGV (@0x18) received by PID 3372 (TID 0x7fce0a145a40) from PID 24; stack trace: ***
@ 0x7fce07404d10 (unknown)
@ 0x7fce07cae29b caffe::BasePrefetchingDataLayer<>::~BasePrefetchingDataLayer()
@ 0x5e86c8 caffe::DataLayerTest<>::TestReshape()
@ 0x726993 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x71eaca testing::Test::Run()
@ 0x71ec18 testing::TestInfo::Run()
@ 0x71ecf5 testing::TestCase::Run()
@ 0x71f688 testing::internal::UnitTestImpl::RunAllTests()
@ 0x71f953 testing::UnitTest::Run()
@ 0x45aec2 main
@ 0x7fce0704aa40 (unknown)
@ 0x461fc9 _start
@ 0x0 (unknown)
Segmentation fault (core dumped)
Makefile:526: recipe for target 'runtest' failed
make: *** [runtest] Error 139

========================= ERROR 4 ==========================================
[----------] 12 tests from DataLayerTest/0, where TypeParam = caffe::CPUDevice
[ RUN ] DataLayerTest/0.TestReadCropTrainSequenceUnseededLevelDB
*** Aborted at 1483031372 (unix time) try "date -d @1483031372" if you are using GNU date ***
PC: @ 0x7fbc78edd29b caffe::BasePrefetchingDataLayer<>::~BasePrefetchingDataLayer()
*** SIGSEGV (@0x10) received by PID 5684 (TID 0x7fbc7b374a40) from PID 16; stack trace: ***
@ 0x7fbc78633d10 (unknown)
@ 0x7fbc78edd29b caffe::BasePrefetchingDataLayer<>::~BasePrefetchingDataLayer()
@ 0x5ea549 caffe::DataLayerTest<>::TestReadCropTrainSequenceUnseeded()
@ 0x726993 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x71eaca testing::Test::Run()
@ 0x71ec18 testing::TestInfo::Run()
@ 0x71ecf5 testing::TestCase::Run()
@ 0x71f688 testing::internal::UnitTestImpl::RunAllTests()
@ 0x71f953 testing::UnitTest::Run()
@ 0x45aec2 main
@ 0x7fbc78279a40 (unknown)
@ 0x461fc9 _start
@ 0x0 (unknown)
Segmentation fault (core dumped)
Makefile:526: recipe for target 'runtest' failed
make: *** [runtest] Error 139

[TimoSaemann]

I do not encounter errors while "make runtest". Using Ubuntu 14.04, Cuda 8.0, cuDNN v4, v5, v5.1, Titan X.
How did you compile it? Make or CMake? Cuda version? cuDNN version?
Which os do you use?
Can you train and test SegNet anyway?

[Vishruit]

Hi,
While running 'make runtest', I get the following error. Interesting, it compiles perfectly for the original version of caffe-segnet.
I am using
rise@karthik-biotech:~/Vishruit/soft/caffe-segnet$ uname -a
Linux karthik-biotech 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
and
CUDA version = 7.5

On a separate note, if it helps: while make all and make test ran properly, initailly make runtest showed error for unable to find libcudart.so.7.5. I then updated the LD config path to include /usr/local/cuda/cuda7.5/lib64 to the LD path. But I didn't rerun the make all and make test after doing so.

Can you please help me in here? Thanks

It occurs in the DataLayerTest tests

[----------] 12 tests from DataLayerTest/3, where TypeParam = caffe::GPUDevice
[ RUN ] DataLayerTest/3.TestReadCropTrainLevelDB
*** Aborted at 1486477109 (unix time) try "date -d @1486477109" if you are using GNU date ***
PC: @ 0x4756d0 boost::detail::sp_counted_base::release()
*** SIGSEGV (@0x10) received by PID 8636 (TID 0x2b86b1cb4640) from PID 16; stack trace: ***
@ 0x2b86b897e330 (unknown)
@ 0x4756d0 boost::detail::sp_counted_base::release()
@ 0x476bc3 caffe::Blob<>::~Blob()
@ 0x2b86b7bceb51 caffe::BasePrefetchingDataLayer<>::~BasePrefetchingDataLayer()
@ 0x4f05ca caffe::DataLayerTest<>::TestReadCrop()
@ 0x8fcb33 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x8f3817 testing::Test::Run()
@ 0x8f38be testing::TestInfo::Run()
@ 0x8f39c5 testing::TestCase::Run()
@ 0x8f6d08 testing::internal::UnitTestImpl::RunAllTests()
@ 0x8f6f97 testing::UnitTest::Run()
@ 0x46c4bf main
@ 0x2b86b8badf45 (unknown)
@ 0x474119 (unknown)
@ 0x0 (unknown)
make: *** [runtest] Segmentation fault (core dumped)