New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CUDA-Team: increase Nixpkgs adoption within the scientific computing community #232433
Comments
One of the things that might be helpful to increase uptake is to have an easily discoverable central page showing how to enable platform specific optimizations for common packages. I think I've seen one for BLAS, but emphasis on easily discoverable/centralized. As it stands now, I don't think an end user in HPC would be able to easily find all the necessary pieces to build a flake for their codebase, add a nixpkgs overlay for customized build flags, how to run the code they built with their flake **, etc. My personal experience with nix was a very slow learning curve trying to figure out what the relationship was between flakes, nixpkgs, nix the package manager, nix the language, and nixos. HPC end users typically have to get something up and running very quickly for it to be worthwhile, so maybe the community would benefit from a well written blog post here ** This I found confusing at first personally |
Agreed!
Yeah, that's definitely been a pain point. Beyond just setting the BLAS provider to something fast, typically want to enable optimizations for Python which are unset by default in Nixpkgs (which results in rebuilding... a lot of Nixpkgs). And then on top of that, set flags to build for a more recent target (like with AVX2/AVX512 support) or a specific GPU. I'm hoping to get something that can serve as a tutorial and a minimal working example up at some point. Would also be good to get it in CI. |
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/cuda-team-roadmap-and-call-for-sponsors/29495/1 |
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/cuda-team-roadmap-update-2023-08-29/32379/1 |
Are you aware of BioNix? https://academic.oup.com/gigascience/article/9/11/giaa121/5987272?login=false @jbedo is maintaining BioNix and the R modules here in nixpkgs. |
FWIW there's also an Nix HPC matrix room. There's definitely some HPC specific issues, however I've found they're usually pretty specific to the HPC facility. Documenting the more common ones and work arounds would be useful though. |
Increase Nixpkgs adoption within the scientific computing community
This issue tracks the feature-level work to increase adoption of CUDA-related Nixpkgs packages within the scientific computing community.
The text was updated successfully, but these errors were encountered: