Add TensorRT packages

[aidalgol]

Description of changes

Add derivations for TensorRT, a high-performance deep learning interface SDK from NVIDIA, which is at this point non-redistributable. Uses different upstream tarballs depending on the current CUDA version, similar to the cudnn derivations.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.11 Release Notes (or backporting 22.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[SuperSandro2000]

Missing meta

[aidalgol]

@SuperSandro2000 I appreciate the quick feedback, but this is nowhere ready for review. I created this draft PR at the suggestion of @SomeoneSerge, after much back and forth in the Nix CUDA Maintainers Matrix room, to ease collaboration efforts on this package. I am aware of the code quality issues, but I am trying to get this to a state where the derivations will build and the python module finds the necessary libraries at runtime. Once this actually works, I will bring the nix code in line with nixpkgs conventions, and parameterise the version similar to the mathematica package.

[SomeoneSerge]

Good! Now let's expose the expression e.g. as an atttribute, e.g. cudaPackages.tensorrt
You'd need to write an extension for cudaPackages namespace.
It's going to be a function of the form final: prev: { tensorrt = final.callPackage ... { }; } where final an instance of cudaPackages.
You'll append it to this list:

nixpkgs/pkgs/top-level/cuda-packages.nix

Line 61 in 473669d

composedExtension = composeManyExtensions [

For now, I think you could define it right there, next to cutensorExtension (which you can use as a template, by the way)

Also, as Sandro suggests, it won't hurt to start bringing things into a shape right away: let's add a meta attribute that would declare the unfree license, limit the platforms to (I suppose) x86_64-linux, and provide a link to some changelog s.t. it's easier to track updates at least manually; let's move the version into a let ... in that we can reuse it when forming the filenames

I'd also suggest that you run nixpkgs-fmt prior to commits if you're not doing that yet

[aidalgol]

@SomeoneSerge With my latest changes, you should now be able to reproduce the error I was describing on matrix (from before creating this draft PR) by running NIXPKGS_ALLOW_UNFREE=1 nix-build -A python39Packages.tensorrt. I'm not sure whether the checkPhase I have written is appropriate for normal regression testing once we have this working, but it makes testing changes at this stage easier.

[SuperSandro2000]

If you want to collaborate on very WIP things I would suggest to work in a PR or branch in your fork.

[SomeoneSerge]

@SuperSandro2000 thank you. It explicitly was my suggestion to @aidalgol that we move the discussion from matrix straight into a draft PR in nixpkgs that we can iterate faster. This is a messy non-redist package, which means there's some trivia about manually gluing pieces together to handle in the beginning

With that done, we'll mark the draft as "ready for review" and tag you

Cheers

[aidalgol]

@SomeoneSerge With your suggested changes, I am back to where I was before moving this to a nixpkgs branch.

$ NIX_PATH=nixpkgs=$PWD NIXPKGS_ALLOW_UNFREE=1 nix-shell -p 'python310.withPackages (pypkgs: [ pypkgs.tensorrt ])'
$ python3 -c 'import tensorrt; assert tensorrt.Builder(tensorrt.Logger())'
[06/23/2022-12:09:18] [TRT] [E] 6: [libLoader.h::DynamicLibrary::49] Error Code 6: Internal Error (Unable to load library: libnvinfer_builder_resource.so.8.4.0)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: pybind11::init(): factory function returned nullptr

From running the same python3 command under strace, something appears to be looking for libnvinfer_builder_resource.so.8.4.0 in the wrong places:

openat(AT_FDCWD, "/run/opengl-driver/lib/libnvinfer_builder_resource.so.8.4.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/wk3xam5pd0a0jwirggg5zpbjzw8zzaf3-gcc-11.3.0-lib/lib/libnvinfer_builder_resource.so.8.4.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/fz33c1mfi2krpg1lwzizfw28kj705yg0-glibc-2.34-210/lib/libnvinfer_builder_resource.so.8.4.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

I have no idea what is looking for it here, though.

[samuela]

Thanks for showing these packages some love @aidalgol! It's def a step forward.

I'm afraid I may not have enough context to be very useful here, since I don't use TensorRT personally. I just have a few notes re the cudaPackages migration that we've been working on.

[samuela]

I'm not part of the NVIDIA dev program, so I don't have a means to test building this but the changes LGTM.

Any remaining hold ups on merging this?

[aidalgol]

Any remaining hold ups on merging this?

Nothing on my end. Just waiting for my last set of changes to be reviewed.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-install-a-specific-version-of-cuda-and-cudnn/21725/4