High memory usage during compilation using GHC 8.0.2-rc2

[domenkozar]

I'm building stack using GHC 8.0.2-rc2 and noticed store uses 5.17GB of (RSS) memory during Data.Store.Internal compilation step. Using GHC 8.0.1 memory goes up to only 1.6GB.

I haven't dug deeper, but something suspicious is going on.

You should be able to reproduce using installed Nix and executing nix-build -A haskell.packages.ghc802.store '<nixpkgs>' using recent channel.

Building haddock for store also goes up to 5.8GB.

[domenkozar]

I've submitted GHC bug report, maybe leave this open until it's resolved: https://ghc.haskell.org/trac/ghc/ticket/13059

[mgsloan]

Wow, good catch! Not sure why so much memory is needed to compile this module. A few possibilities:

1. Lots of TH use? Seems unlikely
2. Lots of instances, lots of them with INLINE

[domenkozar]

@mgsloan FYI GHC 8.0.2 has just been released so I expect to see this to pop up :)

[mitchty]

Yep, just hit this trying to build on an arm box with 2g of ram:

Configuring store-0.3...
Building store-0.3...
Preprocessing library store-0.3...
[ 1 of 11] Compiling System.IO.ByteBuffer ( src/System/IO/ByteBuffer.hs, dist/build/System/IO/ByteBuffer.o )
[ 2 of 11] Compiling Data.Store.Streaming.Internal ( src/Data/Store/Streaming/Internal.hs, dist/build/Data/Store/Streaming/Internal.o )
[ 3 of 11] Compiling Data.Store.Impl  ( src/Data/Store/Impl.hs, dist/build/Data/Store/Impl.o )
[ 4 of 11] Compiling Data.Store.TH    ( src/Data/Store/TH.hs, dist/build/Data/Store/TH.o )
[ 5 of 11] Compiling Data.Store.TH.Internal ( src/Data/Store/TH/Internal.hs, dist/build/Data/Store/TH/Internal.o )
[ 6 of 11] Compiling Data.Store.Internal ( src/Data/Store/Internal.hs, dist/build/Data/Store/Internal.o )
ghc: out of memory (requested 1048576 bytes)
cabal: Leaving directory '/tmp/cabal-tmp-31399/store-0.3'
cabal: Error: some packages failed to install:
stack-1.3.2 depends on store-0.3 which failed to install.
store-0.3 failed during the building phase. The exception was:
ExitFailure 251

[sjakobi]

A likely culprit for the excessive memory usage has been identified in https://ghc.haskell.org/trac/ghc/ticket/13059#comment:20:

Essentially, GHCs up to 8.0.2 had a bug that caused the INLINE pragma on DefaultSignatures method implementations to be ignored. Now that this bug has been fixed, the additional INLINE pragmas kick in and cause the extra memory usage.

The fix is to remove one "layer" of INLINE pragmas as shown in the comment. Apparently this shouldn't much affect performance, but it might be better to run some benchmarks to be safe.