maturin: Sorts RECORD file in wheel archives to make them deterministic

[3pleX-dev]

Things done

Should resolve #384709
@raboof please help review

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[raboof]

As you say in #384708 (comment) it's a dilemma whether we should apply this fix 'surgically' to packages that are known to be affected, or to the 'global' setuptools package.

I think applying it 'surgically' in nixpkgs while discussing it for 'global' inclusion upstream might be a reasonable start.

[3pleX-dev]

As you say in #384708 (comment) it's a dilemma whether we should apply this fix 'surgically' to packages that are known to be affected, or to the 'global' setuptools package.

I think applying it 'surgically' in nixpkgs while discussing it for 'global' inclusion upstream might be a reasonable start.

Thanks! That sounds like a good approach to me—Would you like me open an upstream issue or PR to start the conversation?

[raboof]

I think applying it 'surgically' in nixpkgs while discussing it for 'global' inclusion upstream might be a reasonable start.

Thanks! That sounds like a good approach to me—Would you like me open an upstream issue or PR to start the conversation?

Yeah go for it!

[mweinelt]

The wheel builder in this case would be https://github.com/PyO3/maturin.

[raboof]

The wheel builder in this case would be https://github.com/PyO3/maturin.

Really? But then that patch shouldn't have had any effect, right? As that seems to patch https://github.com/pypa/wheel/blob/main/src/wheel/wheelfile.py#L232 ?

[mweinelt]

Not sure what modifying the build system like that will do, but the build does in fact use maturin.

error: derivation '1kz0sb3y7fm8sm0b1kbyad0r2w8gzl4b-python3.12-pydantic-core-2.27.2.drv' may not be deterministic: outputs differ
         output '/nix/store/7jj8r27i84c2fmkmmpy36dqq3hb2h9r8-python3.12-pydantic-core-2.27.2-dist' differs
         output '/nix/store/vl8yhs7qr9w4h0hqr4g35q303l7jvczn-python3.12-pydantic-core-2.27.2' differs

No visible effect.

[3pleX-dev]

Not sure what modifying the build system like that will do, but the build does in fact use maturin.

error: derivation '1kz0sb3y7fm8sm0b1kbyad0r2w8gzl4b-python3.12-pydantic-core-2.27.2.drv' may not be deterministic: outputs differ
         output '/nix/store/7jj8r27i84c2fmkmmpy36dqq3hb2h9r8-python3.12-pydantic-core-2.27.2-dist' differs
         output '/nix/store/vl8yhs7qr9w4h0hqr4g35q303l7jvczn-python3.12-pydantic-core-2.27.2' differs

No visible effect.

Thank you for the feedback @mweinelt, I'll look again into the source and maturin to see what actually causes the ordering differences.

[3pleX-dev]

So I checked the source and didn’t find anything suspicious there. Then I decided to look into other Nix packages that use the maturin hook to see if they had the same ordering issue—and I confirmed that at least four other packages show the same problem. I didn’t keep going after that since it was clear the issue likely originates from maturin.

I took a look at the maturin source code (I'm not very familiar with Rust), but I tried patching #L156 and #L392 to sort the self.record before writing to the RECORD file, and the patch seems to work fine.

I'll push my update shortly and also propose this in a pr upstream

[3pleX-dev]

@raboof since this issue affects other packages, I don't this change should be in this pr right? it should be on the maturin package itself.

Also I don't think I'm applying this patch the right way and it looks like the build system is deceiving me just like when I patched the python setuptools wheel. If you notice I commented out the sort commands in the patch but somehow this ends up building with no differences.

[raboof]

since you verified other modules that use maturin are also affected, it might indeed make more sense to create a PR against the maturin package.

[raboof]

Since this PR now causes many rebuilds, you'll likely want it to target staging. For more information, see https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#changes-causing-mass-rebuilds

[mweinelt]

Thanks for submitting this upstream. I think we're not in a hurry to land this in 25.05, so we can wait for some review upstream.

[3pleX-dev]

Hi @raboof, the upstream pr has been merged :)

[raboof]

Hi @raboof, the upstream pr has been merged :)

that is awesome, congratulations!

since AFAIK none of the packages in our milestones (ISO runtime closures) are affected by this issue, I agree with @mweinelt that we're not in a particular rush to get this in. Upstream seems to be making regular releases, so we could consider just waiting for that release. Up to the maintainer (@getchoo) I'd say.

[3pleX-dev]

Hi @raboof, the upstream pr has been merged :)

that is awesome, congratulations!

Thank you ❤️

since AFAIK none of the packages in our milestones (ISO runtime closures) are affected by this issue, I agree with @mweinelt that we're not in a particular rush to get this in. Upstream seems to be making regular releases, so we could consider just waiting for that release. Up to the maintainer (@getchoo) I'd say.

Okay no problem at all. I've include a comment that the patch should be removed on the next version bump

[getchoo]

Otherwise LGTM. Thanks for getting this upstreamed!

[getchoo]

Thanks again!

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 397290 --package python312Packages.cryptography --package maturin

---

x86_64-linux

✅ 3 packages built:

• maturin
• python312Packages.cryptography
• python312Packages.cryptography.dist (python312Packages.cryptography.dist.dist)

[nixpkgs-ci]

Successfully created backport PR for staging-24.11:

• [Backport staging-24.11] maturin: Sorts RECORD file in wheel archives to make them deterministic #399661

[3pleX-dev]

Thank you all for your help with this. I'm excited to see this get merged! 🥹🙏