Building without GPU available

[filiptibell]

Hi! Thank you for this great library.

I am wondering how we could build without a GPU available (such as in GitHub Actions)? I found CUDA_COMPUTE_CAP in the source code, and we are targeting a T4 GPU, so setting that to 75 should be correct and seems to have worked. However, I still hit this and panic:

bindgen_cuda/src/lib.rs

Line 519 in 348e73f

let max_nvcc_code = *codes.last().expect("no gpu codes parsed from nvcc");

Would it be possible to also manually specify gpu codes?
Would this actually work & build bindings correctly without a GPU?

[andrenatal]

I'm having the same issue. In my case, I can't build inside a docker image because gpus and nvidia-smi are not available at build time, even with a gpu present at the host, it seems.

If I start the container, then nvidia-smi works. My base image is: FROM nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04

#14 34.60 Caused by:
#14 34.60   process didn't exit successfully: `/app/stt/target/release/build/candle-kernels-1f44ba2b45c1de35/build-script-build` (exit status: 101)
#14 34.60   --- stdout
#14 34.60   cargo:rerun-if-changed=build.rs
#14 34.60   cargo:rerun-if-changed=src/compatibility.cuh
#14 34.60   cargo:rerun-if-changed=src/cuda_utils.cuh
#14 34.60   cargo:rerun-if-changed=src/binary_op_macros.cuh
#14 34.60   cargo:info=["/usr", "/usr/local/cuda", "/opt/cuda", "/usr/lib/cuda", "C:/Program Files/NVIDIA GPU Computing Toolkit", "C:/CUDA"]
#14 34.60   cargo:rerun-if-env-changed=CUDA_COMPUTE_CAP
#14 34.60
#14 34.60   --- stderr
#14 34.60   thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen_cuda-0.1.5/src/lib.rs:489:18:
#14 34.60   `nvidia-smi` failed. Ensure that you have CUDA installed and that `nvidia-smi` is in your PATH.: Os { code: 2, kind: NotFound, message: "No such file or directory" }
#14 34.60   stack backtrace:
#14 34.60      0: rust_begin_unwind
#14 34.60      1: core::panicking::panic_fmt
#14 34.60      2: core::result::unwrap_failed
#14 34.60      3: core::result::Result<T,E>::expect
#14 34.60      4: bindgen_cuda::compute_cap
#14 34.60      5: <bindgen_cuda::Builder as core::default::Default>::default
#14 34.60      6: build_script_build::main
#14 34.60      7: core::ops::function::FnOnce::call_once
#14 34.60   note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
#14 34.60 warning: build failed, waiting for other jobs to finish...
#14 41.76 error: failed to compile `stt-server v0.1.0 (/app/stt)`, intermediate artifacts can be found at `/app/stt/target`.
#14 41.76 To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.
#14 ERROR: process "/bin/sh -c ~/.cargo/bin/cargo install --path ." did not complete successfully: exit code: 101

[sidharthrajaram]

Facing the same issue as above ^

[andrenatal]

This is related: huggingface/candle#2105

[sidharthrajaram]

A workaround for the buildtime nvidia-smi call seems to be setting CUDA_COMPUTE_CAP to the compute capability value based on your GPU.

Based on: huggingface/candle#1516 (comment)