[C++] cuda-test failure

[asfimport]

This seems to have started recently. Weirdly, the test passes if I run cuda-test directly:

$ ctest -V -R cuda
UpdateCTestConfiguration  from :/home/antoine/arrow/cpp/build-test/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/antoine/arrow/cpp/build-test/DartConfiguration.tcl
Test project /home/antoine/arrow/cpp/build-test
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 16
    Start 16: cuda-test

16: Test command: /home/antoine/arrow/cpp/build-support/run-test.sh "/home/antoine/arrow/cpp/build-test" "test" "/home/antoine/arrow/cpp/build-test/debug//cuda-test"
16: Test timeout computed to be: 10000000
16: Running cuda-test, redirecting output into /home/antoine/arrow/cpp/build-test/build/test-logs/cuda-test.txt (attempt 1/1)
16: Running main() from gtest_main.cc
16: [==========] Running 8 tests from 4 test cases.
16: [----------] Global test environment set-up.
16: [----------] 3 tests from TestCudaBuffer
16: [ RUN      ] TestCudaBuffer.Allocate
16: [       OK ] TestCudaBuffer.Allocate (59 ms)
16: [ RUN      ] TestCudaBuffer.CopyFromHost
16: [       OK ] TestCudaBuffer.CopyFromHost (0 ms)
16: [ RUN      ] TestCudaBuffer.FromBuffer
16: [       OK ] TestCudaBuffer.FromBuffer (1 ms)
16: [----------] 3 tests from TestCudaBuffer (60 ms total)
16: 
16: [----------] 3 tests from TestCudaBufferWriter
16: [ RUN      ] TestCudaBufferWriter.UnbufferedWrites
16: [       OK ] TestCudaBufferWriter.UnbufferedWrites (3 ms)
16: [ RUN      ] TestCudaBufferWriter.BufferedWrites
16: [       OK ] TestCudaBufferWriter.BufferedWrites (3 ms)
16: [ RUN      ] TestCudaBufferWriter.EdgeCases
16: [       OK ] TestCudaBufferWriter.EdgeCases (2 ms)
16: [----------] 3 tests from TestCudaBufferWriter (8 ms total)
16: 
16: [----------] 1 test from TestCudaBufferReader
16: [ RUN      ] TestCudaBufferReader.Basics
16: [       OK ] TestCudaBufferReader.Basics (0 ms)
16: [----------] 1 test from TestCudaBufferReader (0 ms total)
16: 
16: [----------] 1 test from TestCudaArrowIpc
16: [ RUN      ] TestCudaArrowIpc.BasicWriteRead
16: ../src/arrow/gpu/cuda-test.cc:331: Failure
16: Failed
16: 'ReadRecordBatch(batch->schema(), device_serialized, default_memory_pool(), &device_batch)' failed with Invalid: Message is length 0
16: [  FAILED  ] TestCudaArrowIpc.BasicWriteRead (5 ms)
16: [----------] 1 test from TestCudaArrowIpc (5 ms total)
16: 
16: [----------] Global test environment tear-down
16: [==========] 8 tests from 4 test cases ran. (73 ms total)
16: [  PASSED  ] 7 tests.
16: [  FAILED  ] 1 test, listed below:
16: [  FAILED  ] TestCudaArrowIpc.BasicWriteRead
16: 
16:  1 FAILED TEST
16:   YOU HAVE 1 DISABLED TEST
16: 
16: ~/arrow/cpp/build-test/src/arrow/gpu
1/1 Test #16: cuda-test ........................***Failed    0.15 sec

0% tests passed, 1 tests failed out of 1

Label Time Summary:
unittest    =   0.15 sec*proc (1 test)

Total Test time (real) =   0.16 sec

The following tests FAILED:
	 16 - cuda-test (Failed)
Errors while running CTest

Reporter: Antoine Pitrou / @pitrou
Assignee: Antoine Pitrou / @pitrou

PRs and other links:

• GitHub Pull Request #3200

_{Note: This issue was originally created as ARROW-3879. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
cc @pearu

[asfimport]

Antoine Pitrou / @pitrou:
There must have been something wrong in my build files. After I cleaned up the git working copy, I don't have this issue anymore.

[asfimport]

Antoine Pitrou / @pitrou:
I'm having this again, reopening.

[asfimport]

Antoine Pitrou / @pitrou:
I've determined that the test fails if I run it from the cmake build directory, not from a higher level directory:

$ ./debug/arrow-cuda-test 
[...]
[----------] 1 test from TestCudaArrowIpc
[ RUN      ] TestCudaArrowIpc.BasicWriteRead
../src/arrow/gpu/cuda-test.cc:331: Failure
Failed
'ReadRecordBatch(batch->schema(), device_serialized, default_memory_pool(), &device_batch)' failed with Invalid: Message is length 0
[  FAILED  ] TestCudaArrowIpc.BasicWriteRead (4 ms)
[...]

$ ./build-test/debug/arrow-cuda-test 
[...]
[----------] 1 test from TestCudaArrowIpc
[ RUN      ] TestCudaArrowIpc.BasicWriteRead
[       OK ] TestCudaArrowIpc.BasicWriteRead (6 ms)
[----------] 1 test from TestCudaArrowIpc (6 ms total)
[...]

[asfimport]

Antoine Pitrou / @pitrou:
Also, it seems that passing an option can be enough to let the test pass:

$ ./debug/arrow-cuda-test --gtest_print_time=1
[...]
[----------] 1 test from TestCudaArrowIpc
[ RUN      ] TestCudaArrowIpc.BasicWriteRead
[       OK ] TestCudaArrowIpc.BasicWriteRead (5 ms)
[----------] 1 test from TestCudaArrowIpc (5 ms total)
[...]

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 3200
#3200