Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cudnn: init 8.0.5, 8.2.4, 8.7.0; update 8.3.2 -> 8.3.3, 8.4.0 -> 8.4.1 #215578

Merged
merged 1 commit into from Feb 11, 2023

Conversation

ConnorBaker
Copy link
Contributor

@ConnorBaker ConnorBaker commented Feb 9, 2023

Description of changes

Adds the following versions:

  • 8.0.5
  • 8.2.4
  • 8.7.0

Bumps the following versions:

  • 8.3.2 -> 8.3.3
  • 8.4.0 -> 8.4.1

Corrects supported CUDA versions:

  • 8.3.x does not support CUDA 11.7 (stops at 11.6)
  • 8.1.1 supports CUDA 11.0 and 11.1 (in addition to 11.2)

Changes default CUDNN versions:

  • CUDA 10.1 now uses the last supported release, 8.0.5
  • CUDA 10.2 now uses the last supported release, 8.7.0
  • CUDA 11.x now use 8.7.0
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@ConnorBaker ConnorBaker mentioned this pull request Feb 9, 2023
13 tasks
@samuela
Copy link
Member

samuela commented Feb 9, 2023

LGTM is there a reason we need to wait on 8.8.0? I feel like we could also create another PR for that once it becomes available

@ConnorBaker
Copy link
Contributor Author

Fair enough! I'll remove it and clean this up.

@ConnorBaker ConnorBaker changed the title [DRAFT] cudnn: init 8.0.5, 8.2.4, 8.7.0, 8.8.0; update 8.3.2 -> 8.3.3, 8.4.0 … [DRAFT] cudnn: init 8.0.5, 8.2.4, 8.7.0; update 8.3.2 -> 8.3.3, 8.4.0 … Feb 9, 2023
@ConnorBaker ConnorBaker changed the title [DRAFT] cudnn: init 8.0.5, 8.2.4, 8.7.0; update 8.3.2 -> 8.3.3, 8.4.0 … [DRAFT] cudnn: init 8.0.5, 8.2.4, 8.7.0; update 8.3.2 -> 8.3.3, 8.4.0 -> 8.4.1 Feb 9, 2023
@ConnorBaker ConnorBaker changed the title [DRAFT] cudnn: init 8.0.5, 8.2.4, 8.7.0; update 8.3.2 -> 8.3.3, 8.4.0 -> 8.4.1 cudnn: init 8.0.5, 8.2.4, 8.7.0; update 8.3.2 -> 8.3.3, 8.4.0 -> 8.4.1 Feb 9, 2023
@ConnorBaker ConnorBaker marked this pull request as ready for review February 9, 2023 23:14
@ConnorBaker
Copy link
Contributor Author

@samuela who should I tag for review in general on CUDA-related issues? The CUDA maintainers team listed at the beginning of https://nixos.wiki/wiki/CUDA gives a 404: https://github.com/orgs/NixOS/teams/cuda-maintainers. Does the tag still work? When I tried typing it, GitHub didn't provide an auto-complete so I wasn't sure if it's valid.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-install-a-specific-version-of-cuda-and-cudnn/21725/6

@SomeoneSerge
Copy link
Contributor

@NixOS/cuda-maintainers is the right tag, maybe it's something about the github organization settings?

@samuela
Copy link
Member

samuela commented Feb 11, 2023

Result of nixpkgs-review pr 215578 run on x86_64-linux 1

1 package marked as broken and skipped:
  • python310Packages.caffeWithCuda
4 packages failed to build:
  • caffeWithCuda
  • cudaPackages.tensorrt (cudaPackages.tensorrt_8_4_0)
  • python310Packages.tensorrt
  • python311Packages.tensorrt
11 packages built:
  • cudaPackages.cudnn (cudaPackages.cudnn_8_7_0)
  • cudaPackages.cudnn_8_4_1
  • katagoWithCuda
  • python310Packages.TheanoWithCuda
  • python310Packages.cupy
  • python310Packages.jaxlibWithCuda
  • python310Packages.tensorflowWithCuda
  • python310Packages.torchWithCuda
  • python311Packages.TheanoWithCuda
  • python311Packages.cupy
  • python311Packages.jaxlibWithCuda

@ConnorBaker
Copy link
Contributor Author

Woah nixpkgs-review is neat! I'm running it to try to reproduce the failures.

First thing I've seen so far is this, but it's still going.

[connorbaker@fedora nixpkgs]$ nixpkgs-review pr 215578
$ git -c fetch.prune=false fetch --no-tags --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/215578/head:refs/nixpkgs-review/1
remote: Enumerating objects: 1824, done.
remote: Counting objects: 100% (1119/1119), done.
remote: Compressing objects: 100% (86/86), done.
remote: Total 1824 (delta 1060), reused 1067 (delta 1028), pack-reused 705
Receiving objects: 100% (1824/1824), 1.36 MiB | 46.56 MiB/s, done.
Resolving deltas: 100% (1285/1285), completed with 302 local objects.
From https://github.com/NixOS/nixpkgs
 * [new branch]              master                -> refs/nixpkgs-review/0
 * [new ref]                 refs/pull/215578/head -> refs/nixpkgs-review/1
$ git worktree add /home/connorbaker/.cache/nixpkgs-review/pr-215578/nixpkgs 592d3352e74deaa2a97fb31db36163d30a92eb36
Preparing worktree (detached HEAD 592d3352e74)
HEAD is now at 592d3352e74 Merge pull request #215823 from fabaff/renault-api-bump
$ git merge --no-commit --no-ff 56fd717ee3069dc07805afc78ca6af5f02e4ac9b
Automatic merge went well; stopped before committing as requested
$ nix --extra-experimental-features nix-command no-url-literals build --no-link --keep-going --no-allow-import-from-derivation --option build-use-sandbox relaxed -f /home/connorbaker/.cache/nixpkgs-review/pr-215578/build.nix
error: builder for '/nix/store/ixb1rldrrb92848s2sgvkd5kx3p3xgxn-TensorRT-8.4.0.6.Linux.x86_64-gnu.cuda-11.6.cudnn8.3.tar.gz.drv' failed with exit code 1;
       last 10 log lines:
       > download the 8.4.0.6 Linux x86_64 TAR package for CUDA 11.7 from
       > https://developer.nvidia.com/tensorrt.
       >
       > Once you have downloaded the file, add it to the store with the following
       > command, and try building this derivation again.
       >
       > $ nix-store --add-fixed sha256 TensorRT-8.4.0.6.Linux.x86_64-gnu.cuda-11.6.cudnn8.3.tar.gz
       >
       > ***
       >
       For full logs, run 'nix log /nix/store/ixb1rldrrb92848s2sgvkd5kx3p3xgxn-TensorRT-8.4.0.6.Linux.x86_64-gnu.cuda-11.6.cudnn8.3.tar.gz.drv'.
[4/11/47 built (1 failed), 207 copied (3856.2/3857.0 MiB), 1394.5 MiB DL] building cuda_10.1.243_418.87.00_linux.run: 100 2453M  100 2453M    0     0  39.7M      0  0:01:01  0:01:01 --:--:-- 84.5M

@samuela
Copy link
Member

samuela commented Feb 11, 2023

Yeah, nixpkgs-review is a life saver!

TensorRT always fails for that same reason in nixpkgs-review, and caffeWithCuda is marked as broken anyhow, so this looks like a success to me.

Thanks for contributing @ConnorBaker!

@samuela samuela merged commit 441c0e5 into NixOS:master Feb 11, 2023
@ConnorBaker ConnorBaker deleted the cudnn-8.8.0 branch February 11, 2023 19:07
@ConnorBaker ConnorBaker mentioned this pull request Feb 20, 2023
12 tasks
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-45/26397/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants