Zero Hydra Failures for release-16.03 branch

[domenkozar]

Creating a tracking issue for ZHF effort in 16.03 release.

Failing packages (https://hydra.nixos.org/jobset/nixos/release-16.03):

• rebar3-open http://hydra.nixos.org/build/32540794 /cc @ericbmerritt
• a bunch of haskell packages /cc @peti

[lucabrunox]

devhelp fixed at b2889ef

[cpages]

pvr-hts should also be fixed at 7eb1526. Thanks for poking me about that.

[domenkozar]

Please also cherry-pick to release-16.03 branch. Thanks! 🍺

[magnetophon]

I can not reproduce on 15.09, and I can't switch to 16.03 now, because I'm in the middle of a project.

The error is: <command-line>:0:1: error: macro names must be identifiers .
I have no idea how to troubleshoot that.

If wanted, I could do a PR with faust1 as the default, and see if that builds???

@pmahoney Do you have ideas?

[cleverca22]

multimc builds without any error on my end, when on the release-16.03 branch and all overrides disabled

aha, chroot builds break it

found and fixed, cmake was downloading .tar.gz files at buildtime, and when the chroot cut off network, it silently continued without the tar, and tried to patch the files the tar provided

[globin]

ceph is fixed

[zimbatm]

evolution builds fine here, it was a "No space left on device" error

[domenkozar]

I've removed fixed packages. Could we get the rest of them to build?

[abbradar]

StepMania is fixed in c82d282

[vcunat]

The tests are getting lots of these errors again:

machine# building path(s) ‘/nix/store/cy5h6d0swa5c1463dkz3y54aa6821ws3-libxslt-1.1.28.tar.gz’
machine# error checking the existence of http://tarballs.nixos.org/sha256/13029baw9kkyjgr7q3jccw2mz38amq7mmpr5p3bh775qawd1bisz:
machine# curl: (6) name lookup timed out
machine# [...]

Does someone know what's happening in there? I know we've had them relatively often in the past.

[domenkozar]

This looks to me like it's because it wants to build the manual due to some reference to a package.

[domenkozar]

@vcunat it's most probably due to https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/security/grsecurity.nix#L31

[vcunat]

Ah, right. My new code only purges derivations, and kernelPatches is a plain attrset that contains derivations in lists (which aren't traversed during purging).

[domenkozar]

@vcunat are you planning to fix this soonish or should I make a temporary fix?

[vcunat]

By "fix" you mean to traverse lists as well? I don't plan that soonish. And in this particular case I do not think we actually want to display the contents of the whole attrset with patch descriptions anyway. I'd go with the usual mkDefault, which I presume is the temporary fix you planned?

[domenkozar]

@vcunat yes, but we also want to catch this kind of errors. Didn't we have some hackish logic to detect hashes in manual?

[vcunat]

I thought we did have a check in there. I don't fully understand this issue, apparently.

[domenkozar]

OK, I'll look into it. I'll also write a script to generate a list of packages that need fixing since tracking them is becoming a bit tedious.

[domenkozar]

I've pushed a fix dbcb901 to release-16.03, but now I'm getting some really weird errors on idempotent nixos-install invokations:

machine# Installation finished. No error reported.                                                                                                                                                                                                 [132/1924]
machine# setting up /etc...
machine# Initializing machine ID from random generator.
machine# installation finished!
machine: exit status 0
machine: must succeed: nixos-install < /dev/null >&2
machine# building the system configuration...
machine# error: anonymous function at /tmp/root/nix/store/mix539jcyx47h4s0agp66zd9583m1vp2-nixos-16.03.git.58f9896/nixos/pkgs/development/libraries/polkit/default.nix:1:1 called with unexpected argument ‘spidermonkey’, at /tmp/root/nix/store/mix539jcyx4
7h4s0agp66zd9583m1vp2-nixos-16.03.git.58f9896/nixos/lib/customisation.nix:56:12
machine# (use ‘--show-trace’ to show detailed location information)

Happens a;so on 16.03 channel commit (that passed tests), so I'm really confused.

[peti]

No more Haskell packages fail to build (or evaluate) in 16.03. It's all in pretty fine condition.

[domenkozar]

We're down to ~180 packages. Still needs a bit of work.

[joachifm]

@domenkozar do you know what's up with those remaining failing grsec builds? They refer to an error that has been fixed. The fixed code should be used by all grsec variants.

[domenkozar]

I'm now mostly worried about lots of transient failures in tests, often blocking a channel update: http://hydra.nixos.org/job/nixos/release-16.03/tested#tabs-constituents

[domenkozar]

I see the following transient errors:

• error: unit ‘multi-user.target’ is inactive and there are no pending jobs
• udevadm settle timeouting
• mdadm failing with code 1

I'm going to increase timeout for udevadm settle, but for the rest of them no idea yet.

[domenkozar]

The first one seems to be due 146c727, @edolstra you seem to have marked it as fragile, any specific reason why?

[domenkozar]

For mdadm, I think we're hitting a timeout for mdadm to finish syncing, but mdadm doesn't let us specify one. I'm assuming here, maybe the sync really fails (can't reproduce locally).

[domenkozar]

Kernel crashes should be fixed once https://lkml.org/lkml/2016/3/29/1010 is released in the kernel update for 4.3+.

[domenkozar]

Eelco fixed the major issue in ad29b72, I'll check if mdadm issue still appears and open a new issue. Thanks everyone!

[edolstra]

The mdadm failures should be addressed by 3e7b510.