Cuda Clang spurious test failure in impl_view_accessible

[crtrott]

I reduced it down to running:

./KokkosCore_UnitTest_Cuda --gtest_filter=cuda.impl_view_accessible:cuda.team_scratch*:cuda.team_poli*:cuda.multi_level_scratch:cuda.uvm
Note: Google Test filter = cuda.impl_view_accessible:cuda.team_scratch*:cuda.team_poli*:cuda.multi_level_scratch:cuda.uvm
[==========] Running 5 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 5 tests from cuda
[ RUN      ] cuda.team_scratch_request
[       OK ] cuda.team_scratch_request (1 ms)
[ RUN      ] cuda.multi_level_scratch
[       OK ] cuda.multi_level_scratch (125 ms)
[ RUN      ] cuda.team_policy_max_recommended
[       OK ] cuda.team_policy_max_recommended (595 ms)
[ RUN      ] cuda.uvm
[       OK ] cuda.uvm (3 ms)
[ RUN      ] cuda.impl_view_accessible
/home/crtrott/Kokkos/kokkos/core/unit_test/cuda/TestCuda_Spaces.cpp:323: Failure
Value of: 0
Expected: error_count
Which is: -9160321642071588864
[  FAILED  ] cuda.impl_view_accessible (5 ms)
[----------] 5 tests from cuda (729 ms total)

[----------] Global test environment tear-down
[==========] 5 tests from 1 test case ran. (729 ms total)
[  PASSED  ] 4 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] cuda.impl_view_accessible

If I remove any of the other 4 tests it doesn't fail reliably anymore.

[crtrott]

This is with Clang 6 Cuda 9.0 on Volta.

[crtrott]

Running this thing with cuda-memcheck reports no errors and the test passes too!! How very helpful.
valgrind doesn't report anything either.

[crtrott]

Workaround looks similar to previous issue on nvcc with permanent block divergence, but can't detect permanent block divergence in this case with clang ...

[crtrott]

Might be that this is only an issue in Clang 6 with Cuda 9 for Volta (which is somewhat beta capability). Need to check again with Cuda 9.2 and Clang 7.

[crtrott]

Not fixed in Clang 7/Cuda 9.1

[ibaned]

@crtrott is this fixed by your rewrite of the CUDA reduction?

[crtrott]

Yeah but am addressing more performance issues right now.