Skip to content
This repository has been archived by the owner on Jan 2, 2021. It is now read-only.

Cache a ghc session per file of interest #630

Merged
merged 7 commits into from
Jun 17, 2020

Conversation

pepeiborra
Copy link
Collaborator

@pepeiborra pepeiborra commented Jun 11, 2020

I recently discovered that the .hi files PR had regressed the performance of GetSpanInfo quite noticeably. This is due mostly to a change in the extraction of spanInfoDocs. Prior to the interfaces change, these were mined out of parsed ASTs. .hi files don't come with ASTs but instead bundle the haddocks for identifiers. But to be able to use them, we need to:

  1. set up a GHC session with all the transitive .hi files
  2. rely on getDocs to extract them

Step 1 can be sped up easily by caching a GHC session per module for files of interest, which makes GetSpanInfo as speedy as it used to be, as seen in the "hover after edit" experiment. I was hoping it would also speed up typechecking, but the edit experiment suggests otherwise so I might have missed something.

Benchmark results (hover becomes much faster, all else stays the same):

version              name                         success   samples   startup              setup                 experiment            maxResidency
upstream             hover                        True      100       7.567142767000001    0.0                   0.479372591           218MB
upstream             edit                         True      100       7.953766959          0.0                   25.364602476          217MB
upstream             getDefinition                True      100       8.692988179          0.0                   0.46926602100000003   219MB
upstream             hover after edit             True      100       8.31115918           0.0                   66.021477715          237MB
upstream             completions after edit       True      100       9.410036817          0.0                   33.817070408          222MB
upstream             code actions                 True      100       10.039016802         0.44827246800000004   0.576492419           242MB
upstream             code actions after edit      False     100       7.954712423          0.0                   1.5715171460000001    245MB
upstream             documentSymbols after edit   True      100       9.284543272          0.0                   1.883872325           218MB
module-ghc-session   hover                        True      100       7.223818533          0.0                   0.13876206500000002   228MB
module-ghc-session   edit                         True      100       7.344935921          0.0                   24.669866416          219MB
module-ghc-session   getDefinition                True      100       7.237144228          0.0                   0.119550469           216MB
module-ghc-session   hover after edit             True      100       7.276310433000001    0.0                   29.694541188000002    218MB
module-ghc-session   completions after edit       True      100       7.3100683150000005   0.0                   30.394427346          222MB
module-ghc-session   code actions                 True      100       7.236867787          0.304327993           0.407078469           243MB
module-ghc-session   code actions after edit      False     100       7.397161018          0.0                   0.445487738           242MB
module-ghc-session   documentSymbols after edit   True      100       7.867092457000001    0.0                   1.7937549000000002    216MB

@pepeiborra
Copy link
Collaborator Author

pepeiborra commented Jun 11, 2020

Unfortunately github doesn't allow SVG attachments in PRs, so I have uploaded the benchmarks here: https://github.com/pepeiborra/ghcide/tree/module-ghc-session-benchmarks/bench-hist/module-ghc-session

hover after edit

We set up a GHC session (load deps, setup finder cache) every time we want to:

- typecheck a module
- get the span infos

This is very expensive, and can be cached.
This allows to bypass work when a module imports & pragmas haven't changed,
e.g. GetDependencies, GetDependencyInformation, GetLocatedImports, etc.
@pepeiborra
Copy link
Collaborator Author

pepeiborra commented Jun 13, 2020

Added early cut-off for the GetModSummary rule, to bypass work when a module imports & pragmas haven't changed,e.g. GetDependencies, GetDependencyInformation, GetLocatedImports, etc.

This results in further perf improvements for hover after edit and for edit. Rendered in the charts below as "modsummary-ghc-session"

edit

hover after edit

Not sure where that came from
Copy link
Collaborator

@cocreature cocreature left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thank you! A few small comments.

src/Development/IDE/Core/Rules.hs Show resolved Hide resolved
src/Development/IDE/Core/Compile.hs Show resolved Hide resolved
src/Development/IDE/Core/Rules.hs Show resolved Hide resolved
src/Development/IDE/Core/Rules.hs Show resolved Hide resolved
@pepeiborra
Copy link
Collaborator Author

Windows 10 build is failing as usual, please retry

Copy link
Collaborator

@cocreature cocreature left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thank you!

@cocreature cocreature merged commit 8de10e9 into haskell:master Jun 17, 2020
pepeiborra added a commit to pepeiborra/ghcide that referenced this pull request Jun 21, 2020
* Cache a GHC session per module

We set up a GHC session (load deps, setup finder cache) every time we want to:

- typecheck a module
- get the span infos

This is very expensive, and can be cached.

* cache the Ghc session for files of interest only

* hlint

* fix 8.4 build

* Early cut-off for ModSummary rule

This allows to bypass work when a module imports & pragmas haven't changed,
e.g. GetDependencies, GetDependencyInformation, GetLocatedImports, etc.

* remove extraneous reverse

Not sure where that came from

* review feedback
pepeiborra added a commit to pepeiborra/ghcide that referenced this pull request Jun 30, 2020
The regression was introduced in haskell#630 - it introduced `GhcSessionDeps` with the idea of reusing the GHC session built for typechecking for computing the SpanInfo, instead of rebuilding it from scratch.

But I forgot to actually reuse it, or maybe the change got lost during the merge.
pepeiborra added a commit to pepeiborra/ghcide that referenced this pull request Jun 30, 2020
The regression was introduced in haskell#630 - it introduced `GhcSessionDeps` with the idea of reusing the GHC session built for typechecking for computing the SpanInfo, instead of rebuilding it from scratch.

But I forgot to actually reuse it, or maybe the change got lost during the merge.
pepeiborra added a commit to pepeiborra/ghcide that referenced this pull request Jun 30, 2020
The regression was introduced in haskell#630.

I added `GhcSessionDeps` with the idea of reusing the typecheck GHC session
for computing the SpanInfo, instead of rebuilding it from scratch.

But I forgot to actually reuse it, or maybe the change got lost during the merge.
cocreature pushed a commit that referenced this pull request Jul 1, 2020
* Fix regression in SpanInfo haddocks for local modules

The regression was introduced in #630.

I added `GhcSessionDeps` with the idea of reusing the typecheck GHC session
for computing the SpanInfo, instead of rebuilding it from scratch.

But I forgot to actually reuse it, or maybe the change got lost during the merge.

* Add test
pepeiborra added a commit to pepeiborra/ide that referenced this pull request Dec 29, 2020
* Cache a GHC session per module

We set up a GHC session (load deps, setup finder cache) every time we want to:

- typecheck a module
- get the span infos

This is very expensive, and can be cached.

* cache the Ghc session for files of interest only

* hlint

* fix 8.4 build

* Early cut-off for ModSummary rule

This allows to bypass work when a module imports & pragmas haven't changed,
e.g. GetDependencies, GetDependencyInformation, GetLocatedImports, etc.

* remove extraneous reverse

Not sure where that came from

* review feedback
pepeiborra added a commit to pepeiborra/ide that referenced this pull request Dec 29, 2020
* Fix regression in SpanInfo haddocks for local modules

The regression was introduced in haskell/ghcide#630.

I added `GhcSessionDeps` with the idea of reusing the typecheck GHC session
for computing the SpanInfo, instead of rebuilding it from scratch.

But I forgot to actually reuse it, or maybe the change got lost during the merge.

* Add test
pepeiborra added a commit to pepeiborra/ide that referenced this pull request Dec 29, 2020
* Cache a GHC session per module

We set up a GHC session (load deps, setup finder cache) every time we want to:

- typecheck a module
- get the span infos

This is very expensive, and can be cached.

* cache the Ghc session for files of interest only

* hlint

* fix 8.4 build

* Early cut-off for ModSummary rule

This allows to bypass work when a module imports & pragmas haven't changed,
e.g. GetDependencies, GetDependencyInformation, GetLocatedImports, etc.

* remove extraneous reverse

Not sure where that came from

* review feedback
pepeiborra added a commit to pepeiborra/ide that referenced this pull request Dec 29, 2020
* Fix regression in SpanInfo haddocks for local modules

The regression was introduced in haskell/ghcide#630.

I added `GhcSessionDeps` with the idea of reusing the typecheck GHC session
for computing the SpanInfo, instead of rebuilding it from scratch.

But I forgot to actually reuse it, or maybe the change got lost during the merge.

* Add test
pepeiborra added a commit to pepeiborra/ide that referenced this pull request Dec 29, 2020
* Cache a GHC session per module

We set up a GHC session (load deps, setup finder cache) every time we want to:

- typecheck a module
- get the span infos

This is very expensive, and can be cached.

* cache the Ghc session for files of interest only

* hlint

* fix 8.4 build

* Early cut-off for ModSummary rule

This allows to bypass work when a module imports & pragmas haven't changed,
e.g. GetDependencies, GetDependencyInformation, GetLocatedImports, etc.

* remove extraneous reverse

Not sure where that came from

* review feedback
pepeiborra added a commit to pepeiborra/ide that referenced this pull request Dec 29, 2020
* Fix regression in SpanInfo haddocks for local modules

The regression was introduced in haskell/ghcide#630.

I added `GhcSessionDeps` with the idea of reusing the typecheck GHC session
for computing the SpanInfo, instead of rebuilding it from scratch.

But I forgot to actually reuse it, or maybe the change got lost during the merge.

* Add test
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants