[CI][C++] Nightly test-conda-cpp-valgrind is flaky due to arrow-compute-vector-test taking closer to 500 seconds

[raulcd]

Describe the bug, including details regarding any error messages, version, and platform.

Sometimes the test-conda-cpp-valgrind nightly test fails due to a timeout on arrow-compute-vector-test. Examples:

• test-conda-cpp-valgrind-8th-May --> 8th May
• test-conda-cpp-valgrind-17th-May --> 17th May

with the failure:

The following tests FAILED:
	 25 - arrow-compute-vector-test (Timeout)

When it succeeds it is close to the 500 seconds, example:

• test-conda-cpp-valgrind-11th-May --> 11th May, 405 seconds
• test-conda-cpp-valgrind-13th-May--> 13th May, 468 seconds

I am not sure whether we should reduce the tests time or split the test so we stop getting those timeouts.

Component(s)

C++, Continuous Integration

[westonpace]

I think we could probably be more selective about which types we test (e.g. testing one or two of the 8 integer types is often sufficient) if we really wanted to trim down the runtimes.

[raulcd]

@westonpace @pitrou should we do something about this flaky one?

[pitrou]

@raulcd Someone should ideally take a look into it and try to reduce the runtime under Valgrind.

For example, it's not obvious that testing all datatypes exhaustively with random input is useful (why all integers, both signed and unsigned, for example?). Test sizes could also be reduced a bit.

@felipecrv Since you've been working quite a bit on kernels, would you like to take a look at this?
Note that you can use Archery to run the Valgrind CI job, for example archery docker run conda-cpp-valgrind.
I would recommended shelling in the Docker image, though (archery docker run conda-cpp-valgrind bash), allowing you to build and test more selectively.

[pitrou]

An orthogonal solution is to split arrow-compute-vector-test into two or three independent test executables (for example: 1) sort+rank+top/bottomK 2) select+filter 3) all the rest). But, while that would workaround the timeout problem, it wouldn't reduce the total CI runtime.

[felipecrv]

An orthogonal solution is to split arrow-compute-vector-test into two or three independent test executables (for example: 1) sort+rank+top/bottomK 2) select+filter 3) all the rest). But, while that would workaround the timeout problem, it wouldn't reduce the total CI runtime.

I will try to reduce number of allocations in tests (I hope it's possible to do so), but will split these tests as I don't feel safe selecting which tests to disable. When developing kernels, exhaustive tests were very helpful in catching unexpected UB and mistakes in the type dispatching logic.

[felipecrv]

I generate a bunch of test cases for run-end encoding (and decoding), but it's really fast compared to other kernels. Number of variations is a bad predictor of test cost.

[----------] Global test environment tear-down
[==========] 423 tests from 1 test suite ran. (326 ms total)
[  PASSED  ] 423 tests.
debug/arrow-compute-vector-test --gtest_filter="EncodeArrayTest*"  0.44s user 0.06s system 96% cpu 0.511 total

[felipecrv]

I tried looking for inefficiencies in test code, but none of the changes I tried made a difference. So I opened a PR containing just tests extracted.