haskell: update package set #122719

cdepillabout · 2021-05-12T13:05:36Z

This Merge

This PR is the regularly merge of the haskell-updates branch into master.

This branch is being continually built and tested by hydra at https://hydra.nixos.org/jobset/nixpkgs/haskell-updates.

I will aim to merge this PR by 2021-05-28. If I can merge it earlier, there might be successor PRs in that time window. As part of our rotation @sternenseemann will continue these merges from 2021-05-28 to 2021-06-11.

haskellPackages Workflow Summary

Our workflow is currently described at pkgs/development/haskell-modules/HACKING.md.

The short version is this:

We regularly update the stackage and hackage pins on haskell-updates (normally at the beginning of a merge window).
The community fixes builds of Haskell packages on that branch.
We aim at at least one merge of haskell-updates into master every two weeks.
We only do the merge if the mergeable job is succeeding on hydra.
If a maintained package is still broken at the time of merge, we will only merge if the maintainer has been pinged 7 days in advance. (If you care about a Haskell package, become a maintainer!)

This is the follow-up to #122510.

This commit has been generated by maintainers/scripts/haskell/update-hackage.sh

This commit has been generated by maintainers/scripts/haskell/regenerate-hackage-packages.sh

This commit has been generated by maintainers/scripts/haskell/update-hackage.sh

This commit has been generated by maintainers/scripts/haskell/regenerate-hackage-packages.sh

versions in stackage is too old to support 0.19.5, so we pin futhark to a version that does.

New release 0.13.0.1 introduced support with ref-tf >= 0.5, so we can remove our patch for that purpose.

haskell-nix/hnix#922 removed a few build-depends from hnix.cabal which are still required until the following constraints apply to stackage: * relude >= 1.0.0.0 * semialign >= 1.2 Luckily, we can simply revert a few commits from master and add semialign-indexed to resolve this without too much hassle nor extra-packages (which may cause us trouble through propagation of a newer relude).

maralorn · 2021-05-13T12:03:09Z

Phew, conflicting files pkgs/development/haskell-modules/hackage-packages.nix seems like a bad sign.

Edit: Ah, that was my fault I merged the taffybar change into master. Mea culpa. I’ll merge master into haskell-updates and resolve the merge conflict manually.

maralorn · 2021-05-13T12:04:28Z

Anyways I think our aim should be that our last merge before branch-off will have (nearly) 0 failing builds on all platforms.

cdepillabout · 2021-05-13T12:12:15Z

Anyways I think our aim should be that our last merge before branch-off will have (nearly) 0 failing builds on all platforms.

Does this mean that you definitely think we should have at least one more merge to master? (I'm not actually sure if there is an easy way to see if we have any failing Haskell-related builds on master.)

maralorn · 2021-05-13T12:24:31Z

Does this mean that you definitely think we should have at least one more merge to master? (I'm not actually sure if there is an easy way to see if we have any failing Haskell-related builds on master.)

Well, when I merged everything was fine on x86 but the other platforms had still a lot of failing builds. I think we should fix or disable those on the release branch. We now have two options.

Fix them on haskell-updates and then merge to master shortly before branch-off.
Fix them on master.

I think 1) is better because we have our hydra-job to see the progress we are making. But I am okay either way. We just probably need a decision soon because people who do PRs right now want there fixes in the stable release so we need to tell them which branch to target.

Of course, it would also be okay if we can‘t fix all builds before branch-off. Then we just backport them. But that wouldn‘t be less work.

maralorn · 2021-05-13T12:26:28Z

I assume that a query like https://hydra.nixos.org/eval/1669648?filter=haskellPackages on an evaluation of https://hydra.nixos.org/jobset/nixpkgs/trunk will give us basically the information we want.

cdepillabout · 2021-05-13T12:30:12Z

Okay, that sounds good. Let's aim for 1) then.

I'll try to post a Hydra build report in on this PR in the next day or two. I probably won't have much time to focus on this until the weekend.

If the branch off happens on the 21st, let's try to merge this PR on the 20th just in case.

maralorn · 2021-05-13T12:33:08Z

Yeah, no hurries. I didn‘t expect you to pick up on this before the weekend. There will still be plenty of time to fix everything starting on saturday.

Depends on zlib somehow (https://hydra.nixos.org/build/143009645/nixlog/1) which is not declared in the cabal file.

haskell.compiler.ghcHEAD: disable DWARF on non x86

This should conclude a pass of direct aarch64-linux failures related to this issue on hydra. Subsequent evaluation may of course reveal more.

All of these packages use x86 intrinsics-related headers and don't compile on non x86 platforms as a result. These overrides should be refactored into the yaml configuration at some point. Resolves #122014.

x86 assembler doesn't compile on aarch64 of course.

* charsetdetect: dependency library which is vendored fails with a cpp failure on aarch64. Could probably theoretically support aarch64, but doesn't in practice. * persist-state: aarch64 (no UNALIGNED_MEMORY) and armv7l (32 bit) fail in cpp.

Main executable uses x86 assembler, so we can't build it anywhere at all.

Requires x86 assembler, so no luck building it anywhere else.

New hydra evaluation brought some additional intstances of this happening to light.

teto · 2021-05-18T23:44:52Z

I've been trying to run the maintainer script to update one of my haskell pacakges but maintainers/scripts/haskell/regenerate-hackage-packages.sh fails with from nixpkgs be1e5f9 :

maintainers/scripts/haskell/regenerate-hackage-packages.sh
❯ maintainers/scripts/haskell/update-hackage.sh                                                                                                                                                                                                      
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5423  100  5423    0     0  22881      0 --:--:-- --:--:-- --:--:-- 22881
warning: unknown setting 'experimental-features'
[57.8 MiB DL]
path is '/nix/store/vp6dy5a94hbkrmvsl51n6x2k14qh04xs-45a46e6deaf5d304b5f604035fc301b4d5a957a3.tar.gz'
nixpkgs2 on  haskell-updates took 13s 
➜ maintainers/scripts/haskell/regenerate-hackage-packages.sh                                                                                                                                                                                         
warning: unknown setting 'experimental-features'
these derivations will be built:
  /nix/store/658ks0xlvr6zc7mz2g6dpd50d8l7yqay-unpacked-cabal-hashes.drv
Starting hackage2nix to regenerate pkgs/development/haskell-modules/hackage-packages.nix ...
warning: unknown setting 'experimental-features'
error: `stdenv.lib` is a deprecated alias for `lib`
(use '--show-trace' to show detailed location information)
hackage2nix: user error (Error in $: Failed reading: not a valid json value. Expecting object value at '}')

I didn't want to open an issue but if the error is LGTM I might.

sternenseemann · 2021-05-18T23:55:36Z

@teto Feel free to open an issue about this in ~~NixOS/cabal2nix~~ (or here in nixpkgs). The failure is very interesting and I can't reproduce it (perhaps you are on non x86_64-linux?).

Does nix-env -qaP --json -f /path/to/nixpkgs --arg config "{ allowAliases = false; }" trigger the failure as well? My best guess is that this check is triggered somehow:

nixpkgs/pkgs/stdenv/generic/default.nix

Line 167 in a8f71f0

lib else throw "`stdenv.lib` is a deprecated alias for `lib`";

teto · 2021-05-19T00:02:51Z

nix-env -qaP --json -f /path/to/nixpkgs --arg config "{ allowAliases = false; overlays = []; }" returns a seemlingly valid (visually) json but is followed by

error: `stdenv.lib` is a deprecated alias for `lib`
(use '--show-trace' to show detailed location information)
}%

as well. I am on x86 but with nixExperimental. I've disabled the overlay so it's strange you can't reproduce.

EDIT: I've created #123600 not to pollute this more.

This commit has been generated by maintainers/scripts/haskell/mark-broken.sh

cdepillabout · 2021-05-19T01:40:38Z

This is the latest build report immediately before merging in.

haskell-updates build report from hydra

evaluation 1671119 of nixpkgs commit be1e5f9 as of 2021-05-19 01:34 UTC

Build summary

Platform	Failed ❌	DependencyFailed ❗	TimedOut ⌛🚫	Unfinished ⏳	Success ✔️
aarch64-linux 📱	21	69	4	1	6575
x86_64-darwin 🍎	73	63	8		6483
x86_64-linux 🐧	1	1		1	6711

Unmaintained packages with build failure

95 job(s)

Unmaintained packages with failed dependency

135 job(s)

Report generated with maintainers/scripts/haskell/hydra-report.hs

cdepillabout · 2021-05-19T02:00:42Z

I've merged haskell-updates into master. Thanks to everyone that fixed things over the past week or so. Special thanks to @sternenseemann for fixing a bunch of the aarch64 and darwin builds. For the first time, we have over 20,000 Haskell-related packages building (over x86_64-linux, aarch64-linux, and darwin)!

@jonringer Just FYI for 21.05, this will be the last time haskell-updates is merged into master before the branch-off. Any Haskell-related problems should be fixed directly on the release-21.05 branch moving forward.

We still have about 250 failing packages, all of which are either on darwin or aarch64-linux. All of our x86_64-linux packages are either correctly marked broken, or can be built successfully.

maralorn added 3 commits May 12, 2021 21:56

all-cabal-hashes: 2021-05-10T22:01:59Z -> 2021-05-11T22:36:02Z

441c9e9

This commit has been generated by maintainers/scripts/haskell/update-hackage.sh

hackage-packages.nix: Regenerate based on current config

3d33fad

This commit has been generated by maintainers/scripts/haskell/regenerate-hackage-packages.sh

hackage2nix: maralorn maintains more packages

73d4fa3

github-actions bot added the 6.topic: haskell label May 12, 2021

all-cabal-hashes: 2021-05-11T22:36:02Z -> 2021-05-12T11:46:04Z

27157c7

This commit has been generated by maintainers/scripts/haskell/update-hackage.sh

ofborg bot added 8.has: package (new) 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels May 12, 2021

hackage-packages.nix: Regenerate based on current config

6773b45

This commit has been generated by maintainers/scripts/haskell/regenerate-hackage-packages.sh

cdepillabout marked this pull request as ready for review May 12, 2021 13:38

cdepillabout requested review from maralorn and sternenseemann as code owners May 12, 2021 13:38

cdepillabout mentioned this pull request May 12, 2021

haskell-updates: add documentation for workflow #122728

Merged

sternenseemann and others added 4 commits May 12, 2021 18:39

haskellPackages.futhark: pin to 0.19.4 for compat with versions

bc10fb7

versions in stackage is too old to support 0.19.5, so we pin futhark to a version that does.

haskellPackages.hnix: remove now unnecessary patch

52c8b2d

New release 0.13.0.1 introduced support with ref-tf >= 0.5, so we can remove our patch for that purpose.

Merge pull request #122706 from sternenseemann/hnix-fix

dac6cc8

stephank mentioned this pull request May 13, 2021

haskellPackages: mark linux-only packages #122806

Merged

10 tasks

Merge branch 'master' into haskell-updates

814c7e8

stephank and others added 4 commits May 14, 2021 20:30

haskellPackages: mark linux-only packages

bc2528b

haskellPackages.generic-optics: Fix build

8c4331f

haskellPackages: Update transitively disabled packages

5caa523

release-haskell: add documentation explaining what this file is used for

ea304f2

sternenseemann and others added 3 commits May 18, 2021 00:02

haskellPackages.update-nix-fetchgit: fix with hnix 0.13

17a71f3

haskellPackages.taglib: add zlib to librarySystemDepends

5568210

Depends on zlib somehow (https://hydra.nixos.org/build/143009645/nixlog/1) which is not declared in the cabal file.

Merge pull request #123377 from sternenseemann/fix-ghc-head

c5cd6a2

haskell.compiler.ghcHEAD: disable DWARF on non x86

ofborg bot requested review from MarcWeber, peti and kosmikus May 18, 2021 01:00

sternenseemann and others added 9 commits May 18, 2021 17:08

haskellPackages: disable some more failing doctests on aarch64

a3fd86a

This should conclude a pass of direct aarch64-linux failures related to this issue on hydra. Subsequent evaluation may of course reveal more.

haskellPackages.{blake3,seqalign,crc32c}: restrict platforms to x86

d640ce8

All of these packages use x86 intrinsics-related headers and don't compile on non x86 platforms as a result. These overrides should be refactored into the yaml configuration at some point. Resolves #122014.

haskellPackages.iniline-asm: only test on x86

84f49ed

x86 assembler doesn't compile on aarch64 of course.

haskellPackages.inline-asm: restrict to x86

a9a5465

Main executable uses x86 assembler, so we can't build it anywhere at all.

haskellPackages.hw-prim-bits: restrict to x86

cbaa1c8

Requires x86 assembler, so no luck building it anywhere else.

haskellPackages: disable failing doctests on aarch64

f4c038f

New hydra evaluation brought some additional intstances of this happening to light.

haskellPackages: fix eval of configuration-nix.nix

c7e7454

haskellPackages.futhark: use python3 instead of python37

be1e5f9

ofborg bot added 10.rebuild-linux: 101-500 and removed 10.rebuild-linux: 11-100 labels May 18, 2021

teto mentioned this pull request May 19, 2021

hackage2nix: user error (Error in $: Failed reading: not a valid json value. Expecting object value at '}') #123600

Closed

cdepillabout added 2 commits May 19, 2021 10:30

haskell.compiler.ghcHEAD: disable Hydra build on aarch64-linux

721c8ab

hackage2nix: Mark failing builds broken

3522051

This commit has been generated by maintainers/scripts/haskell/mark-broken.sh

cdepillabout merged commit b76684a into master May 19, 2021

sternenseemann deleted the haskell-updates branch May 19, 2021 01:53

sternenseemann restored the haskell-updates branch May 19, 2021 01:53

cdepillabout mentioned this pull request May 19, 2021

Haskell updates #123682

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

haskell: update package set #122719

haskell: update package set #122719

cdepillabout commented May 12, 2021 •

edited

maralorn commented May 13, 2021 •

edited

maralorn commented May 13, 2021

cdepillabout commented May 13, 2021 •

edited

maralorn commented May 13, 2021

maralorn commented May 13, 2021

cdepillabout commented May 13, 2021 •

edited

maralorn commented May 13, 2021

teto commented May 18, 2021

sternenseemann commented May 18, 2021 •

edited

teto commented May 19, 2021 •

edited

cdepillabout commented May 19, 2021 •

edited

cdepillabout commented May 19, 2021

haskell: update package set #122719

haskell: update package set #122719

Conversation

cdepillabout commented May 12, 2021 • edited

This Merge

haskellPackages Workflow Summary

maralorn commented May 13, 2021 • edited

maralorn commented May 13, 2021

cdepillabout commented May 13, 2021 • edited

maralorn commented May 13, 2021

maralorn commented May 13, 2021

cdepillabout commented May 13, 2021 • edited

maralorn commented May 13, 2021

teto commented May 18, 2021

sternenseemann commented May 18, 2021 • edited

teto commented May 19, 2021 • edited

cdepillabout commented May 19, 2021 • edited

haskell-updates build report from hydra

Build summary

Unmaintained packages with build failure

Unmaintained packages with failed dependency

cdepillabout commented May 19, 2021

cdepillabout commented May 12, 2021 •

edited

maralorn commented May 13, 2021 •

edited

cdepillabout commented May 13, 2021 •

edited

cdepillabout commented May 13, 2021 •

edited

sternenseemann commented May 18, 2021 •

edited

teto commented May 19, 2021 •

edited

cdepillabout commented May 19, 2021 •

edited