Skip to content

Commit

Permalink
Start writing docs
Browse files Browse the repository at this point in the history
  • Loading branch information
safesparrow committed Oct 18, 2022
1 parent 672e3a2 commit 5ba78e8
Show file tree
Hide file tree
Showing 2 changed files with 55 additions and 0 deletions.
54 changes: 54 additions & 0 deletions tests/FSharp.Compiler.Service.Tests2/Docs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Parallel Type-Checking in F#

This document describes the idea and implementation details for parallel type-checking of files in the F# compiler.

## Context and the current state of the compiler

Performance of F# compilation and code analysis is not very high.

Recent improvements not-withstanding, there are still low-hanging fruit that can be introduced.

One idea for such an improvement was originally described in https://github.com/dotnet/fsharp/discussions/11634 by @kerams .
This is going to be the main topic of this page.

### Current state of type-checking

One of the main phases of compilation is type-checking. Depending on the project in question, it can take up to and exceed 50% of the total compilation time.
Currently, by default all files in a project are type-checked in sequence, one-by-one, leading to increased compilation wall-clock time.

### Recent addition - "Parallel type checking for impl files with backing sig files"

A recent [change](https://github.com/dotnet/fsharp/pull/13737) introduced in the compiler allows for parallel type-checking of all `.fs` files backed by `.fsi` files. Such `.fs` files by definition cannot be depended upon by any other files w.r.t. type-checking, since all the type-checking information is exposed by the corresponding `.fsi` files.

The new feature, when enabled, allows partial parallelisation of type-checking as follows:
1. All `.fsi` files and `.fs` files without backing `.fsi` files are type-checked in sequence, as before.
2. Then all `.fs` files with backing `.fsi` files are type-checked in parallel.

For a project that uses `.fsi` files throughout, such as the `FSharp.Compiler.Service` project, this presents a major speedup.

Some data points:
- [Fantomas](https://github.com/fsprojects/fantomas) solution: total build time 17.49s -> 14.28s - [link](https://github.com/dotnet/fsharp/pull/13737#issuecomment-1223637818)
- F# codebase build time: 112s -> 92s - [link](https://github.com/dotnet/fsharp/pull/13737#issuecomment-1223386853)

#### Enabling the feature

The feature is opt-in and can be enabled in the compiler via a CLI arg & MSBuild property.

## The importance of using Server GC for parallel work

By default .NET processes use Workstation GC, which is single-threaded. What this means is it can become a bottleneck for highly-parallel operations, due to increased GC pressure and the cost of GC pauses being multiplied by the number of threads waiting.
That is why when increasing parallelisation of the compiler and the compiler service it is important to note the GC mode being used and consider enabling Server GC.

Below is an example showing the difference it can make for a parallel workflow.

### Parallel projects analysis results for a synthetic solution
| GC Mode | Processing Mode | Time |
|-------------|-----------------|-----------------------------------------|
| Workstation | Sequential | 16005ms |
| Workstation | Parallel | 10849ms |
| Server | Sequential | 14594ms (-9% vs Workstation Sequential) |
| Server | Parallel | 2659ms (-75% vs Workstation Parallel) |

For more details see https://github.com/dotnet/fsharp/pull/13521

###
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
<None Include="Big.fs">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
<Content Include="Docs.md" />
</ItemGroup>

<ItemGroup>
Expand Down

0 comments on commit 5ba78e8

Please sign in to comment.