[Perf] Windows/x64: 5 Regressions on 10/17/2024 5:07:04 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.22621
Queue	TigerWindows
Baseline	2830cf5d4b6c7853c6f187b8646ee7cfda7a9ae7
Compare	e377401c233637e761d597115ab42b1e23b757b4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Perf_SingleCharFrozenDictionary

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
TryGetValue_False_FrozenDictionary - Duration of single invocation ADX - Test Multi Config Graph	41.55 μs	47.14 μs	1.13	0.54	False
TryGetValue_False_FrozenDictionary - Duration of single invocation ADX - Test Multi Config Graph	385.41 ns	506.43 ns	1.31	0.38	False
TryGetValue_False_FrozenDictionary - Duration of single invocation ADX - Test Multi Config Graph	3.79 μs	5.01 μs	1.32	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_SingleCharFrozenDictionary*'

System.Collections.Perf_SingleCharFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 10000)

ETL Files

Histogram

JIT Disasms

System.Collections.Perf_SingleCharFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 100)

ETL Files

Histogram

JIT Disasms

System.Collections.Perf_SingleCharFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 1000)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.22621
Queue	TigerWindows
Baseline	2830cf5d4b6c7853c6f187b8646ee7cfda7a9ae7
Compare	e377401c233637e761d597115ab42b1e23b757b4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Count - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	867.42 ms	995.35 ms	1.15	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer|[A-Za-z]inn)\s", Options: None)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.22621
Queue	TigerWindows
Baseline	2830cf5d4b6c7853c6f187b8646ee7cfda7a9ae7
Compare	e377401c233637e761d597115ab42b1e23b757b4
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<IntClass>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Array_ComparerStruct - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	58.28 μs	62.54 μs	1.07	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<IntClass>*'

System.Collections.Sort<IntClass>.Array_ComparerStruct(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[DrewScoggins]

Seems related to #108903

Potentially related regression:

• [Perf] Linux/arm64: 1 Regression on 10/17/2024 11:07:08 PM perf-autofiling-issues#43609
• [Perf] Linux/x64: 11 Regressions on 10/17/2024 5:07:04 PM perf-autofiling-issues#43411

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-collections
See info in area-owners.md if you want to be subscribed.

[amanasifkhalid]

Notes	Recent Score	Orig Score	Ampere Ubuntu	Tiger Ubuntu	Tiger Windows	Benchmark
	1.32	1.32			1.32 1.32	System.Collections.Perf_SingleCharFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 1000)
	1.32	1.31			1.32 1.31	System.Collections.Perf_SingleCharFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 100)
	1.25	1.28		1.25 1.28		System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties)
	1.21	1.13			1.21 1.13	System.Collections.Perf_SingleCharFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 10000)
	1.12	1.29		1.12 1.29		System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings)
	1.11	1.27		1.11 1.27		System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties)
	1.10	1.15			1.10 1.15	System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer
	1.10	1.12		1.10 1.12		System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom
	1.08	1.10		1.08 1.10		System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom
	1.06	1.07			1.06 1.07	System.Collections.Sort(IntClass).Array_ComparerStruct(Size: 512)
	1.04	1.09		1.04 1.09		System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock
	1.02	1.11		1.02 1.11		System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+
	1.00	1.06	1.00 1.06			System.Collections.CreateAddAndClear(Int32).Stack(Size: 512)
	0.99	1.08		0.99 1.08		System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable)
	0.81	1.19		0.81 1.19		System.Buffers.Tests.ReadOnlySequenceTests(Byte).IterateTryGetTenSegments
	0.66	1.16		0.66 1.16		System.Buffers.Tests.ReadOnlySequenceTests(Char).IterateTryGetTenSegments
	0.55	1.21		0.55 1.21		System.Buffers.Tests.ReadOnlySequenceTests(Char).IterateForEachTenSegments

#116017 seems to have helped the Linux arm64 regression return to its baseline. The regressions on Tiger machines are quite spikey -- considering these are layout-sensitive benchmarks, I think it's likely the JCC erratum is driving most of the churn here. Thus, I'm going to consider this resolved.