Fully static binaries

[milot-mirdita]

I recently integrated a protein language model into our homology search method foldseek. Foldseek is written in C++ and we use candle to run the pLM.

Everything is linked into one binary in the end, which is working great. However, if I also want to give users the option to do GPU-inference and here only dynamic linking works. I really would like to offer precompiled binaries that run everywhere. These are a huge help to some of our not-super-tech-savvy users.

I tried to use cudarc 0.10.0, static-linking and everything seemed to nearly work, until I figured out that cudarc uses the driver API instead of the runtime API and always links to libcuda.so.

I saw that the recent 0.11.0 release got rid of static-linking completely. Would you consider adding support for the runtime API, and also re-adding static linking support?

The changes required on the Candle side don't look to bad, and I guess I could maintain them in a fork if upstream is not interested (I haven't asked).

[coreylowman]

Yeah unfortunately with how bindgen supports libloading (you either do or don't use dynamic loading), I think to support all three options (dynamic linking, dynamic loading, static linking) we'd need to have 2 versions of each of the bindgen files. It's just a lot of code to carry around (we already need to support multiple cuda driver versions, so this would double the amount of bindgen files). I'm not necessarily opposed to it, and maybe the real solution here is to split the bindgen files into their own crates and have different crates for static vs dynamic vs dynamic load.