This repository was archived by the owner on Aug 3, 2024. It is now read-only.
Reduce memory usage #1594
Merged
Reduce memory usage #1594
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Change logic for accumulation of names for which link warnings will be generated - Change handling of `--ignore-link-symbol` to allow qualified and unqualified names. Added to CHANGES.md - Some formatting changes and comments here and there
- ghc-debug dependency and instrumentation - cabal.project custom with-compiler - hie.yaml files - traces and such
GRE closures should never be necessary to Haddock, so we never want to keep them on the heap. Despite that, they are retained by a lot of the data structures that Haddock makes use of. - Attempt to fix that situation by adding strictness to various computations and pruning the `ifaceInstances` field of `Interface` to a much thinner data type. - Removes the `ifaceFamInstances` field, as it was never used. - Move some of the attach instances types (e.g. `SimpleType`) to the types module
- Refactor `ifaceDeclMap` to drastically reduce memory footprint. We no longer store all declarations associated with a given name, since we only cared to determine if the only declaration associated with a name was a value declaration. Change the `DeclMap` type to better reflect this. - Drop pre-renaming export items after the renaming step. Since the Hoogle backend used the pre-renamed export items, this isn't trivial. We now generate Hoogle output for exported declarations during the renaming step (if Hoogle output /should/ be generated), and store that with the renamed export item. - Slightly refactor Hoogle backend to handle the above change and allow for early generation of Hoogle output. - Remove the `ifaceRnDocMap` and `ifaceRnArgMap` fields of the `Interface` type, as they were never used. - Remove some unnecessary strictness - Remove a lot of dead code from `Syb` module
- Delete ErrMsgM, stop accumulating warnings in a writer - Make IfM a state monad, print warnings directly to stdout, move IfM type into types module - Drop ErrMsg = String synonym - Unset IORefs from plugin after they are read, preventing unnecessary retention of interfaces
The strictness introduced in this commit was motivated by observing thunk leaks in the eventlog2html output. - Refactor attach instances list comprehension to avoid large intermediate thunks - Refactor some HTML backend list comprehensions to avoid large intermediate thunks - Avoid thunks accumulating in documentation types or documentation parser - A lot of orphan NFData instances to allow us to force documentation values
Removes a potential retainer of `ModSummary`s
|
That's some serious work, thanks a bunch @FinleyMcIlwaine |
hubot
pushed a commit
to ghc/ghc
that referenced
this pull request
May 17, 2024
…hc-9.6-mem-fixes Reduce memory usage
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR does several things to drastically reduce Haddock's memory usage. The high-level changes are described in the commit messages.
I have been testing these changes on the Agda codebase with a version of Haddock that is compiled with a GHC version that has been patched to further reduce Haddock's memory usage. With a non-patched GHC and haddock 9.6, the maximum residency is about 1.1GB with the following profile:
With these patches and the patched GHC, the maximum residency is about 580MB with the following profile:
Note the reduction in running time as well.
On the Agda codebase, there are no diffs in the generated HTML documentation.
I'll do my best to annotate this PR with comments describing the changes.