haskellPackages: mark linux-only packages

[stephank]

Motivation for this change

ZHF: #122042
cc @NixOS/nixos-release-managers, @NixOS/haskell

A small subset of Haskell packages that are currently failing on Darwin are simply not supported on Darwin by design, but not marked as such. This PR adds a helper and starts a list of these packages. I also included cpuid and hsignal under this helper umbrella, because they do a similar thing, but for x86-only.

Haskell in Nix has the unsupported-platforms list, but it's flexibility is limited, only allowing to eliminate specific platforms. For example, marking a package as not supported on Darwin would've worked for a while with x86_64-darwin, but then we got aarch64-darwin. Similarly, marking a package as Linux-only is impossible: there is no opposite supported-platforms, plus we're continuously adding Linux platforms like riscv64-linux.

I think fixing this properly is outside the scope of ZHF and the upcoming release, and hopefully this stop-gap solution is okay for now.

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[sternenseemann]

Thanks for doing this, I was meaning to do more towards this direction as well!

Can you please rebase this onto haskell-updates? It makes testing and stabilizing haskell changes when everything happens on one branch.

[sternenseemann]

Can you add an example and type signature like this: https://github.com/tazjin/nixdoc#comment-format?

[sternenseemann]

Not sure if I'm too happy with using a function here — there is really no need to as there is only one platforms set (as opposed to other cases where we use this type of API like ghcWithPackages where there are different sets the function could take packages from).

I guess the shortcut is nice, but I fear we introduce a unnecessary and potentially unintuitive API here. Interested to hear other opinions.

[stephank]

I don't disagree, and it was definitely for convenience only. My original plan was to see if I could accomplish this with even less boilerplate, as maybe platformOverrides = { honk = platforms.linux; ... }, but I went for something like this to keep the diff small this close to release.

So, we can of course inline it all:

{
  honk = overridePlatforms super.honk pkgs.lib.platforms.linux;
}

Or maybe introduce a variable?

let
  inherit (pkgs.lib) platforms;
in {
  honk = overridePlatforms super.honk platforms.linux;
}

Though this file does not contain any yet, and I'm not sure if it's desirable.

[stephank]

I pushed a version that inlines everything.

I also rebased on the merge-base of haskell-updates and master. Is that sufficient for testing, or do we need to be on the latest haskell-updates?

The reason I'm targeting master is because of ZHF, and the next haskell-updates merge is scheduled post-branchoff: #122719

[maralorn]

No, the next merge is not supposed to be happening past branch-off. The date given there is the usual time window of two weeks.

[stephank]

Alright, in that case, I switched target branch and rebased on current haskell-updates. 👍

[peti]

These settings belong into configuration-hackage2nix.yaml.

[sternenseemann]

These settings belong into configuration-hackage2nix.yaml.

I disagree. The unsupported-platforms mechanism is too cumbersome for these kinds of changes: It doesn't allow us to reuse lib.platforms and it requires us to expand the list of unsupported platforms as new platforms are added to haskell packages or even nixpkgs itself.

[maralorn]

I am a bit torn on the issue. I feel like if this is a only a temporary fix it would be okay to do it via the yaml file (I mean there won‘t be any new platforms added before branch off, will they?) And if this is not temporary we should also do it via the yaml file but need to change hackage2nix a bit.

Introducing this solution will introduce an inconsistent split. Between some platform being configured via yaml and some via nix.

[sternenseemann]

I want this as an intermediate solution at the very least because it allows us to clean up the haskell-updates jobset in a simple and maintainable way.

In the future, I'd like to rework how platforms work in hackage2nix by allowing us to prescribe meta.platforms and meta.badPlatforms (which would be the successor to unsupported-platforms). The only conceptual trouble left to solve is how to distinguish between platforms as strings (like we have in unsupported-platforms) and named platform lists from lib.platforms.

[cdepillabout]

I think I agree with @maralorn here.

If this is just a temporary fix, then it doesn't really matter how we do it for the 21.05 release.

However, in the long run, I'd like to go with a solution that is automatable (well, really, just some solution that makes us do minimal work). Up until now, most of our automatable solutions have been based on yaml files.

@sternenseemann Do you have some sort of long-term vision for how platform-related stuff should work in the Haskell infrastructure? I guess I'm mainly interested in what exactly the three of us will have to do for this. I don't personally want to deal with problems on multiple platforms more than my current level of almost-completely-ignoring-it. (Although don't get me wrong, I do like our current approach of trying to push more of the burden on drive-by-contributors.)

[stephank]

I'm not familiar with Haskell, nor am I a big Nix contributor, but my 2 cents:

I don't think this can be automated, unless upstream lists this somewhere in package metadata. But I don't see anything on hackage or in the cabal files of these packages. So pushing this to drive-by contributions (and efforts like ZHF) is probably the right solution for now.

What could help longer term is more automated nixpkgs-review of PRs on Darwin machines, like @r-rmcgibbo currently does for x86_64-linux and aarch64-linux. (Though that sort of review is currently not a requirement for merging.)

Right now, listing the two Darwin platforms in the existing unsupported-platforms is sufficient as a stop-gap to satisfy Hydra. But technically the list of 'non-Linux' is a lot longer:

nix-repl> lib.filter (x: !(lib.hasSuffix "-linux" x)) lib.platforms.all
[ "i686-cygwin" "x86_64-cygwin" "x86_64-darwin" "i686-darwin" "aarch64-darwin" "armv7a-darwin" "i686-freebsd" "x86_64-freebsd" "aarch64-genode" "i686-genode" "x86_64-genode" "x86_64-solaris" "js-ghcjs" "mmix-mmixware" "aarch64-netbsd" "armv6l-netbsd" "armv7a-netbsd" "armv7l-netbsd" "i686-netbsd" "mipsel-netbsd" "powerpc-netbsd" "riscv32-netbsd" "riscv64-netbsd" "x86_64-netbsd" "aarch64-none" "arm-none" "armv6l-none" "avr-none" "i686-none" "msp430-none" "or1k-none" "powerpc-none" "riscv32-none" "riscv64-none" "vc4-none" "x86_64-none" "i686-openbsd" "x86_64-openbsd" "x86_64-redox" "wasm64-wasi" "wasm32-wasi" "x86_64-windows" "i686-windows" ]

I think a long-term solution should at least:

• incorporate lib.platforms, for better matching, and because distribution-nixpkgs is prone to be out of sync. (allKnownPlatforms is very limited)
• Allow matching the opposite direction (ie. supported-platforms), which is the convention for Nixpkgs in general (meta.platforms = lib.platforms.linux;)
• Allow combining sets (linux ++ darwin)

[stephank]

I pushed a version that lists these in the YAML just for x86_64-darwin, which should be enough for Hydra.

I cannot test it, though, because regenerate-hackage-packages.sh fails:

maintainers/scripts/haskell/regenerate-hackage-packages.sh: line 19: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8): No such file or directory
Starting hackage2nix to regenerate pkgs/development/haskell-modules/hackage-packages.nix ...
hackage2nix: constraint cassava ==0.5.2.0 cannot be resolved in Hackage
This could be due the package being missing in the hackage directory
or the file system not being case sensitive.

CallStack (from HasCallStack):
  error, called at hackage2nix/Main.hs:205:34 in main:Main
hackage2nix: thread blocked indefinitely in an MVar operation

I created a case-sensitive APFS volume for this, so that should not be the problem. I also don't believe the LC_ALL setting is the cause, though it may be a separate issue. (Seems like C.UTF-8 doesn't exist on Darwin. I have the same issue in #122634. I tried with en_US.UTF_8, no dice.)

[cdepillabout]

Thanks, this looks good to merge in for now at least.