.NET 7.0 Preview 4 Microbenchmarks Performance Study Report #69396
Labels
area-Meta
tenet-performance
Performance related issue
tracking
This issue is tracking the completion of other related issues.
Milestone
Comments
dakersnar
added
area-Meta
tenet-performance
ghost
added
the
|
Tagging subscribers to this area: @dotnet/area-meta Issue DetailsDataThis time we have covered following configs:
Most of the benchmarks were run on bare-metal machines, some were executed via WSL. This would not be possible without the help from: @adsitnik @AndyAyersMS @carlossanlop @danmoseley @jeffhandley @kunalspathak @mrsharm and @tannergooding who contributed their results and time. The full report generated by the tool is available here. The full report contains improvements, so if you read it from the end you can see the biggest perf improvements. There are plenty of them! Again, the full historical data turned out to be extremely useful. For details about methodology please read #41871. Preview3 report can be found here. Notable Improvements
RegressionsBy design
Investigation in progress
Already solved
Noise, flaky or multimodalThe following benchmarks showed up in the report generated by the tool, but were not actual regressions:
Noisy benchmarks from previous months:
Statistics
Total: 104158 Statistics per Architecture
Statistics per Operating System
Statistics per Namespace
Big thanks to everyone involved!
|
|
TODO: ensure all the "in investigation" regressions get addressed/solved. If they do not, we should make standalone issues for instances where I only left a comment. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Data
This time we have covered following configs:
Most of the benchmarks were run on bare-metal machines, some were executed via WSL.
This would not be possible without the help from: @adamsitnik @AndyAyersMS @carlossanlop @danmoseley @jeffhandley @kunalspathak @mrsharm and @tannergooding who contributed their results and time.
The full report generated by the tool is available here. The full report contains improvements, so if you read it from the end you can see the biggest perf improvements. There are plenty of them!
Again, the full historical data turned out to be extremely useful. For details about methodology please read #41871. Preview3 report can be found here.
Notable Improvements
System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False)),System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase),System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase), etcSystem.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: None), and othersExceptions.Handling.ThrowAndCatch*System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled)Regressions
By design
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(*, options: NonBacktracking)Investigation in progress
FractalPerf.Launch.TestSystem.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterBE,System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterLEBenchstone.BenchI.Array2.Test,SciMark2.kernel.benchSparseMult,System.Collections.CopyTo<Int32>.ReadOnlySpan(Size: 2048),LinqBenchmarks.Count00ForX,System.Collections.CopyTo<Int32>.Memory(Size: 2048)System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom|Sawyer|Huckleberry|Finn", Options: NonBacktracking),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer|[A-Za-z]inn)\s", Options: NonBacktracking),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: NonBacktracking),System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: NonBacktracking)System.Collections.ContainsTrue<Int32>.Span(Size: 512)System.Collections.IterateForEach<Int32>.List(Size: 512)System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomConverterAlready solved
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(*, Options: NonBacktracking)System.Net.Security.Tests.SslStreamTests.HandshakeRSA2048CertAsync(protocol: Tls12),System.Net.Security.Tests.SslStreamTests.HandshakeECDSA256CertAsync(protocol: Tls12),System.Net.Security.Tests.SslStreamTests.HandshakeRSA4096CertAsync(protocol: Tls12),System.Net.Security.Tests.SslStreamTests.HandshakeECDSA256CertAsync(protocol: Tls13),System.Net.Security.Tests.SslStreamTests.HandshakeRSA2048CertAsync(protocol: Tls13)System.Text.Json.Tests.Perf_Get.GetBooleanSystem.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: None)Noise, flaky or multimodal
The following benchmarks showed up in the report generated by the tool, but were not actual regressions:
System.Numerics.Tests.Perf_VectorOf<*>.*System.Numerics.Tests.Perf_Matrix4x4.IdentityBenchmarkSystem.Numerics.Tests.Perf_Matrix4x4.TranslationBenchmarkSystem.Numerics.Tests.Perf_Vector3.EqualityOperatorBenchmarkSystem.Numerics.Tests.Perf_Vector4.AddOperatorBenchmarkSystem.Numerics.Tests.Perf_Vector4.AddFunctionBenchmarkSystem.Numerics.Tests.Perf_Vector4.ClampBenchmarkSystem.Numerics.Tests.Perf_Vector4.InequalityOperatorBenchmarkSystem.Numerics.Tests.Constructor.SpanCastBenchmark_SingleSystem.Numerics.Tests.Constructor.ConstructorBenchmark_SingleNoisy benchmarks from previous months:
System.Collections.Concurrent.Count<String>.Dictionary(Size: 512)System.Collections.TryAddDefaultSize<String>.ConcurrentDictionary(Count: 512)PerfLabTests.CastingPerf.CheckArrayIsInterfaceNoStatistics
Total: 104158
Same: 82.88 %
Slower: 3.43 %
Faster: 4.97 %
Noise: 8.67 %
Unknown: 0.05 %
Statistics per Architecture
Statistics per Operating System
Statistics per Namespace
Big thanks to everyone involved!
The text was updated successfully, but these errors were encountered: