Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: micro & macrobenchmarks #50

Merged
merged 3 commits into from
Jul 23, 2022
Merged

chore: micro & macrobenchmarks #50

merged 3 commits into from
Jul 23, 2022

Conversation

leonardochaia
Copy link
Owner

@leonardochaia leonardochaia commented Jul 23, 2022

The macrobenchmark does a complete I/O benchmark. It is the same as running dotnet affected command.
It discovers csproj files, creates an MSBuild project graph and then applies the detection algorithms.

The microbenchmark runs the detection algorithm only. The Projects are already discovered and the ProjectGraph is already built.

According to my profiling, the most time consumer method is the building of the ProjectGraph.
Instantiating ProjectGraph for 1k SDK based projects takes ~5s and 3GB ram 😮

BenchmarkDotNet=v0.13.1, OS=manjaro 
11th Gen Intel Core i7-1165G7 2.80GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK=6.0.200
  [Host]     : .NET 6.0.2 (6.0.222.6406), X64 RyuJIT
  Job-NDREBB : .NET 6.0.2 (6.0.222.6406), X64 RyuJIT

Toolchain=.NET 6.0  WarmupCount=1  
Method TotalProjects ChildrenPerProject Mean Error StdDev Gen 0 Gen 1 Allocated
AffectedAlgorithm 500 20 57.42 μs 0.366 μs 0.324 μs 3.1128 0.1221 19 KB
AffectedAlgorithm 1000 20 925.54 μs 18.312 μs 19.593 μs - - 67 KB
Method TotalProjects ChildrenPerProject Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
MacroBenchmark 500 20 2.654 s 0.0531 s 0.1261 s 266000.0000 53000.0000 1000.0000 1 GB
MacroBenchmark 1000 20 5.687 s 0.1134 s 0.2185 s 546000.0000 107000.0000 1000.0000 3 GB

Gonna research if there are ways to improve this by configuring the ProjectGraph, but now we have a way to measure it.
We could add a separate benchmark for ProjectGraph instantiation only in order to measure this.

I did notice that if the project has no Sdk defined, it takes significantly less time/resources.
However, we will need to have the Sdk defined in order to use Predictions in PR #46

@leonardochaia leonardochaia merged commit 4997bab into main Jul 23, 2022
@leonardochaia leonardochaia deleted the chore/macrobenchmark branch July 23, 2022 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant