Two versions of GHC depended on when useArchiveFilesForTemplateHaskell is true

[jonathanlking]

On master (88f1e2d) when setting useArchiveFilesForTemplateHaskell = true two versions of GHC are being depended on. Picking an arbitrary package (in this case vector):

nix-repl> pkgs = (import ./survey/default.nix { useArchiveFilesForTemplateHaskell = true; })

nix-repl> pkgs.haskellPackages.vector                                                        
«derivation /nix/store/v6hbyn8a5nydw81a9vjabr4qxfj093mr-vector-0.12.3.1.drv»

nix-repl> pkgs = (import ./survey/default.nix { })                                           

nix-repl> pkgs.haskellPackages.vector              
«derivation /nix/store/ggxx6mgws1vn22l668dsnpl76yx635ri-vector-0.12.3.1.drv»

Using nix-tree to inspect the derivations we find that the first derivation /nix/store/v6hbyn8a5nydw81a9vjabr4qxfj093mr-vector-0.12.3.1.drv depends on two different GHC derivations:

a. /nix/store/6yvclb4xwmkwg632nmm2mdcaw94vfj7k-ghc-musl-9.2.7.drv — Immediate Parents (41)
b. /nix/store/pd3h7k2f9p9290dhfch2wnwkp9714xgk-ghc-musl-9.2.7.drv — Immediate Parents (4): jailbreak-cabal-1.4.drv, hscolour-1.24.4.drv, Cabal-syntax-3.6.0.0.drv, hscolour-1.24.4.drv

While the second derivation /nix/store/ggxx6mgws1vn22l668dsnpl76yx635ri-vector-0.12.3.1.drv only depends on a.

Diffing the derivations for a and b, the only meaningful difference is in the preConfigure step:

- DYNAMIC_GHC_PROGRAMS = YES
+ DYNAMIC_GHC_PROGRAMS = NO
+ GhcLibHcOpts += -fPIC -fexternal-dynamic-refs
+ GhcRtsHcOpts += -fPIC -fexternal-dynamic-refs

Some observations/thoughts:

1. These are options changed by enableRelocatedStaticLibs and enableShared attributes in the GHC derivation, which is set by useArchiveFilesForTemplateHaskell.

2. The immediate parents suggest this is the GHC in buildHaskellPackages, as these parents match things set up in mkDerivationImpl.

3. We already override buildHaskellPackages to replace GHC (to avoid depending on 2 versions)

   static-haskell-nix/survey/default.nix

   Lines 1586 to 1596 in 88f1e2d

   	# To override GHC, we need to override both `ghc` and the one in
   	# `buildHaskellPackages` because otherwise this code in `geneic-builder.nix`
   	# will make our package depend on 2 different GHCs:
   	# nativeGhc = buildHaskellPackages.ghc;
   	# depsBuildBuild = [ nativeGhc ] ...
   	# nativeBuildInputs = [ ghc removeReferencesTo ] ...
   	#
   	ghc = fixGhc old.ghc;
   	buildHaskellPackages = old.buildHaskellPackages.override (oldBuildHaskellPackages: {
   	ghc = fixGhc oldBuildHaskellPackages.ghc;
   	});

   .

4. The GHC overrides are made by a fixGhc function, which is applied to both ghc and the ghc in buildHaskellPackages, so I'm not sure why it isn't working the same in both places?

5. buildHaskellPackages.ghc does have the right derivation 🤔

   nix-repl> pkgs.haskellPackages.buildHaskellPackages.ghc
   «derivation /nix/store/6yvclb4xwmkwg632nmm2mdcaw94vfj7k-ghc-musl-9.2.7.drv»
   
   nix-repl> pkgs.haskellPackages.ghc                      
   «derivation /nix/store/6yvclb4xwmkwg632nmm2mdcaw94vfj7k-ghc-musl-9.2.7.drv»
   

[jonathanlking]

I intend to continue investigating this and will share thoughts here 📝
It's also very possible that I'm making a very silly mistake somewhere 😅

[jonathanlking]

Commenting out

static-haskell-nix/survey/default.nix

Lines 1594 to 1596 in 88f1e2d

	buildHaskellPackages = old.buildHaskellPackages.override (oldBuildHaskellPackages: {
	ghc = fixGhc oldBuildHaskellPackages.ghc;
	});

I now get

nix-repl> pkgs = (import ./survey/default.nix { useArchiveFilesForTemplateHaskell = true; })

nix-repl> pkgs.haskellPackages.buildHaskellPackages.ghc                     
«derivation /nix/store/pd3h7k2f9p9290dhfch2wnwkp9714xgk-ghc-musl-9.2.7.drv»

nix-repl> pkgs.haskellPackages.ghc                      
«derivation /nix/store/6yvclb4xwmkwg632nmm2mdcaw94vfj7k-ghc-musl-9.2.7.drv»

So maybe this override isn't being applied "deep"/early enough? (So that it also applies to the packages within haskellPackages and not just to the exposed buildHaskellPackages?)

[jonathanlking]

Unfortunately I'm pretty stumped on this 😔

For posterity, one approach to debugging I tried was:

• Cloning a copy of nixpkgs (to ~/github/nixpkgs)
• Adding a breakpoint to mkDerivation with mkDerivation = (_: break _) (makeOverridable mkDerivationImpl);
• Launching nix repl --debugger --ignore-try
• Loading with my local modified nixpkgs pkgs = (import ./survey/default.nix { normalPkgs = (import ~/github/nixpkgs/default.nix {}); useArchiveFilesForTemplateHaskell = true; })
• Building the packages again with pkgs.haskellPackages.vector, which triggers the breakpoint (multiple times!)
• Printing out values of ghc/self.ghc/buildHaskellPackages.ghc/self.buildHaskellPackages.ghc that are in scope.

I didn't really learn anything new from this though, so I intend to stop investigating for now.
I think the only consequence of this issue are longer build times/more disk space, due to two copies of GHC, rather than bad build output.

[jonathanlking]

Fixed by #127 (which incorporated #123).