CI doesn't run CUDA tests

[ViliamVadocz]

Testing seems to happen only with default features enabled. It would be good to also test with features like test-cuda and test-f64 (added in #421).

Potential issues are that it might be difficult to get a machine with CUDA support for the CI. If it is not possible, at least running cargo check --features test-cuda would be a good idea to make sure the CUDA-specific code compiles.

[coreylowman]

Currently even to run cargo check --features test-cuda nvcc is needed to compile the kernels in the build.rs script, which requires cuda installed AFAIK.

I've been using lambdalabs for gpu dev, I wonder if its possible to spin up a machine during the workflows?

Otherwise I think you can do self-hosted, which may be a good option as we scale

[coreylowman]

I think we could add a feature called ci-check that we can check for in build.rs, and if so, just do some version of simple file creation for each kernel file.

Then the CI can do cargo check --features test-cuda,ci-check

[coreylowman]

Hmm it looks like find_cuda_helper from cudarc dependency fails if no cuda dependency is found:

  thread 'main' panicked at 'Could not find a cuda installation', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/find_cuda_helper-0.2.0/src/lib.rs:12:13