generating ghc.haddock uses a lot of memory

[juhp]

I tried building ghc-9.2.1-alpha2 on Fedora armv7hl and it runs out of memory somehow:
GHC.Tc.Gen.HsType.tc_ihaddock: out of memory

I originally reported this to ghc 2 days ago, not noticing the memory issue.
So I am now reporting it here instead:

https://koji.fedoraproject.org/koji/taskinfo?taskID=66762262

GHC error in desugarer lookup in Ghci1:
  Can't find interface-file declaration for variable $bSM
    Probable cause: bug in .hi-boot file, or inconsistent .hi file
    Use -ddump-if-trace to get an idea of which file caused the error
GHC error in desugarer lookup in Ghci1:
  Can't find interface-file declaration for variable $bSM
    Probable cause: bug in .hi-boot file, or inconsistent .hi file
    Use -ddump-if-trace to get an idea of which file caused the error
GHC/Unit/Module/ModIface.hs:147:1
Warning: 'eps_decls' is out of scope.
    If you qualify the identifier, haddock can try to link it anyway.
:
:
Warning: GHC.Tc.Instance.Class: could not find link destinations for:
	- GHC.Tc.Instance.Class.SafeOverlapping
	- GHC.Tc.Instance.Class.onlyNamedBndrsApplied
Warning: GHC.Tc.Instance.Typeable: could not find link destinations for:
	- GHC.Tc.Instance.Typeable.KindRepM
Warning: GHC.Rename.Env: could not find link destinations for:
	- GHC.Rename.Env.lookupQualifiedNameGHCi
Warning: GHC.Tc.Gen.HsType: could not find link destinations for:
	- GHC.Tc.Gen.HsType.TcTyMode
	- GHC.Tc.Gen.HsType.mode_tyki
	- GHC.Tc.Gen.HsType.AnyBoxedKind
	- GHC.Tc.Gen.HsType.LiftedKind
	- GHC.Tc.Gen.HsType.isAllowedDataResKind
	- GHC.Tc.Gen.HsType.SkolemMode
	- GHC.Tc.Gen.HsType.tc_ihaddock: out of memory (requested 1048576 bytes)
make[1]: *** [compiler/ghc.mk:291: compiler/stage2/doc/html/ghc/ghc.haddock] Error 251
make[1]: *** Deleting file 'compiler/stage2/doc/html/ghc/ghc.haddock'
make: *** [Makefile:128: all] Error 2

You can find the full build.log attached in the mentioned ghc ticket.

Environment

GHC version used: 8.10.4
Operating System: Fedora 33+
System Architecture: armv7hl

For now I have worked around it by disabling haddock on armv7hl.

[Kleidukos]

Hi, and thank you for the report. :)
I have tested on my machine, and Haddock's components (api, library) typically need about 512M of heap each. The build crashes with 128M. I don't doubt your builder machine has more than that in RAM, but maybe it wasn't fully done with GHC (which can be quite demanding) when it attempted to build Haddock (and thus failed)

[juhp]

Okay, thanks, but 8.10 and 9.0 are fine FWIW, so somehow it would appear more memory is needed for 9.2 somehow.
I am going to test again with 9.2.1 RC1.

[juhp]

Just noting that I ran into this again with 9.2.1, while moving the Fedora ghc9.2 package to use Hadrian.

At the point when it gives up I think the buildserver is completely swapped out, sometimes it hangs for some hours.
Anyway I am fine with disabling the haddock generation for ghc on armv7hl.

[Kleidukos]

Thank you for keeping us updated. :)

[juhp]

Just another datapoint: my ghc-9.2.1 build on my x86_64 laptop also just got OOM'ed while trying to generate ghc.haddock.

[juhp]

Similar issue now with ghc-9.2.5 and i686 Fedora 38

Warning: GHC.Iface.Load: could not find link destinations for:
	- GHC.Iface.Load.computeInterface
Warning: GHC.IfaceToCore: could not find link destinations for:
	- GHC.IfaceToCore.mergeIfaceDecl
Warning: GHC.Iface.Rename: could not find link destinations for:
	- GHC.Iface.Rename.ShIfM
Warning: GHC.Types.Name.Shape: could not find link destinations for:
	- GHC.Types.Name.Shape.uAvailInfos
haddock: out of memory (requested 1048576 bytes)
Warning: GHC.Tc.Gen.Splice: could not find link

Also noting that the buildserver in question had 128GB of ram :-)

https://koji.fedoraproject.org/koji/taskinfo?taskID=96875914