cabal2nix adds system-libyaml flag but not yaml system dependency

[nh2]

I just tried to run stack2nix commit 88fd8be0cad55e4f29575c5d55645f6321201c17 against stack commit 94d302c40d3d08dcaa0213f3f1ffdc1384842714.

Building stack with that fails with

builder for '/nix/store/cv0kawgaparngahh971hqi7kbkdjxra9-yaml-0.8.32.drv' failed with exit code 1; last 10 log lines:
  c/helper.h:4:10: error:
       fatal error: yaml.h: No such file or directory
       #include <yaml.h>
                ^~~~~~~~
    |
  4 | #include <yaml.h>
    |          ^
  compilation terminated.
  `cc' failed in phase `C Compiler'. (Exit code: 1)
  builder for '/nix/store/cv0kawgaparngahh971hqi7kbkdjxra9-yaml-0.8.32.drv' failed with exit code 1
cannot build derivation '/nix/store/8sk2bb84y04a71j9zf0dfkc4v46ky53a-stack-1.8.0.drv': 1 dependencies couldn't be built

The generated yaml stack derivation is:

      "yaml" = callPackage
        ({ mkDerivation, aeson, attoparsec, base, bytestring, conduit
         , containers, directory, filepath, resourcet, scientific
         , semigroups, stdenv, template-haskell, text, transformers
         , unordered-containers, vector
         }:
         mkDerivation {
           pname = "yaml";
           version = "0.8.32";
           sha256 = "dc20f863deb4ee75395bf1f78268781db76be6209af67b70c05f6e1a09f47a31";
           configureFlags = [ "-fsystem-libyaml" ];
           isLibrary = true;
           isExecutable = true;
           libraryHaskellDepends = [
             aeson attoparsec base bytestring conduit containers directory
             filepath resourcet scientific semigroups template-haskell text
             transformers unordered-containers vector
           ];
           doHaddock = false;
           doCheck = false;
           homepage = "https://github.com/snoyberg/yaml#readme";
           description = "Support for parsing and rendering YAML documents";
           license = stdenv.lib.licenses.bsd3;
         }) {};

Note here configureFlags = [ "-fsystem-libyaml" ];, but the absence of a system yaml package in the arguments.

The commit in cabal2nix that added this flag is b9d7046

What is going on here?

[nh2]

CC @peti @domenkozar

Note I didn't have this problem with an older version of stack2nix (I think it was commit 9070f91); it generated:

      "yaml" = callPackage
        ({ mkDerivation, aeson, attoparsec, base, bytestring, conduit
         , containers, directory, filepath, resourcet, scientific
         , semigroups, stdenv, template-haskell, text, transformers
         , unordered-containers, vector
         }:
         mkDerivation {
           pname = "yaml";
           version = "0.8.32";
           sha256 = "dc20f863deb4ee75395bf1f78268781db76be6209af67b70c05f6e1a09f47a31";
           configureFlags = [ "-fsystem-libyaml" ];
           isLibrary = true;
           isExecutable = true;
           libraryHaskellDepends = [
             aeson attoparsec base bytestring conduit containers directory
             filepath resourcet scientific semigroups template-haskell text
             transformers unordered-containers vector
           ];
           doHaddock = false;
           doCheck = false;
           homepage = "https://github.com/snoyberg/yaml#readme";
           description = "Support for parsing and rendering YAML documents";
           license = stdenv.lib.licenses.bsd3;
         }) {};

configureFlags = [ "-fsystem-libyaml" ]; is absent there.

The older version of stack2nix used this version of cabal2nix:

           pname = "cabal2nix";
           version = "2.7.2";
           src = fetchgit {
             url = "https://github.com/NixOS/cabal2nix";
             sha256 = "02v73ziqnqnxjbqmfrlc6c1g9pdamr8rbpar9afl7j51b4h8mai9";
             rev = "e72fbdfe57941627633006fa1fb4003e26619424";
           };

[nh2]

I see @domenkozar already encountered this: https://github.com/input-output-hk/stack2nix/blob/88fd8be0cad55e4f29575c5d55645f6321201c17/default.nix#L13-L14

Why needed until we upgrade to 18.09 though, is there already a plan to fix that?

[peti]

This is caused by a bug in yaml's Cabal file. The build instructions don't declare a dependency on the system's libyaml library when the system-libyaml flag is enabled. Therefore, the library is not added into scope by cabal2nix.

[peti]

Note that yaml-0.10.0 still doesn't work properly, despite the fix, because apparently there's some incompatibility with our libyaml library. I'm not sure what it is, but the error looks like this:

Building test suite 'spec' for yaml-0.10.0..
[1 of 5] Compiling Data.Yaml.IncludeSpec ( test/Data/Yaml/IncludeSpec.hs, dist/build/spec/spec-tmp/Data/Yaml/IncludeSpec.o )

<no location info>: error:
    <command line>: can't load .so/.DLL for: /build/yaml-0.10.0/dist/build/libHSyaml-0.10.0-64c8k00CFzUHPLsqeX6hIL-ghc8.4.3.so (/build/yaml-0.10.0/dist/build/libHSyaml-0.10.0-64c8k00CFzUHPLsqeX6hIL-ghc8.4.3.so: undefined symbol: yaml_stream_end_event_initialize)
[2 of 5] Compiling Data.Yaml.THSpec ( test/Data/Yaml/THSpec.hs, dist/build/spec/spec-tmp/Data/Yaml/THSpec.o )

<no location info>: error:
    ghc: panic! (the 'impossible' happened)
  (GHC version 8.4.3 for x86_64-unknown-linux):
        Dynamic linker not initialised

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

[3 of 5] Compiling Data.YamlSpec    ( test/Data/YamlSpec.hs, dist/build/spec/spec-tmp/Data/YamlSpec.o )

<no location info>: error:
    ghc: panic! (the 'impossible' happened)
  (GHC version 8.4.3 for x86_64-unknown-linux):
        Dynamic linker not initialised

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

[5 of 5] Compiling Paths_yaml       ( dist/build/spec/autogen/Paths_yaml.hs, dist/build/spec/spec-tmp/Paths_yaml.o )

[nh2]

The libyaml patches linked in the links above look like the same issue as snoyberg/yaml#110

[nh2]

@peti I'm a bit confused by the two opposite directions cabal2nix and nixpkgs are taking. In your fix you added the libyaml dependency and keep the system-libyaml flag enabled, but in configuration-nix.nix the flag is disabled instead:

https://github.com/NixOS/nixpkgs/blob/9540b1c5357acd63f89c985a85faee79bf28d902/pkgs/development/haskell-modules/configuration-nix.nix#L317-L318

Should these the two approaches agree with each other?

[nh2]

This is caused by a bug in yaml's Cabal file. The build instructions don't declare a dependency on the system's libyaml library when the system-libyaml flag is enabled. Therefore, the library is not added into scope by cabal2nix.

Fix PR in snoyberg/yaml#151

[peti]

@nh2, I had no idea that override existed in Nixpkgs. :-( Thanks for the heads up. I removed it in NixOS/nixpkgs@fbada9f, which will come to master soon.