[Perf] Linux/x64: 6 Regressions on 1/12/2023 3:42:01 AM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5385f045f9aee91c22922d21552c4317198710f1
Compare	870825a840fa501c686421ab4d85f59917a8e37f
Diff	Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SerializeToString - Duration of single invocation	851.95 ns	920.40 ns	1.08	0.14	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 920.3959360699695 > 899.8422544055065.
IsChangePoint: Marked as a change because one of 1/11/2023 11:00:48 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -11.43125921980275 (T) = (0 -933.8644253757811) / Math.Sqrt((478.8518246508526 / (31)) + (475.94281667066355 / (26))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (26) - 2, .025) and -0.07655969816532687 = (867.4525221102675 - 933.8644253757811) / 867.4525221102675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5385f045f9aee91c22922d21552c4317198710f1
Compare	870825a840fa501c686421ab4d85f59917a8e37f
Diff	Diff

Regressions in System.IO.Tests.Perf_Path

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetTempPath - Duration of single invocation	166.40 ns	179.77 ns	1.08	0.17	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetTempPath

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 179.7657054686494 > 171.69850868621342.
IsChangePoint: Marked as a change because one of 1/11/2023 11:00:48 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -7.8311900195225395 (T) = (0 -180.96303054213482) / Math.Sqrt((28.794410139524505 / (31)) + (39.06723875817004 / (26))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (26) - 2, .025) and -0.0723620642621025 = (168.75180181486218 - 180.96303054213482) / 168.75180181486218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5385f045f9aee91c22922d21552c4317198710f1
Compare	870825a840fa501c686421ab4d85f59917a8e37f
Diff	Diff

Regressions in ByteMark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BenchNumericSortRectangular - Duration of single invocation	635.61 ms	670.80 ms	1.06	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

Histogram

ByteMark.BenchNumericSortRectangular

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 670.7977693076923 > 668.527900204231.
IsChangePoint: Marked as a change because one of 12/13/2022 2:17:54 AM, 12/14/2022 11:14:18 PM, 1/11/2023 11:00:48 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -17.577438902386923 (T) = (0 -675523460.7422655) / Math.Sqrt((26466525799419.324 / (31)) + (89694376495133.31 / (26))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (26) - 2, .025) and -0.057059456393689284 = (639059096.1145281 - 675523460.7422655) / 639059096.1145281 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5385f045f9aee91c22922d21552c4317198710f1
Compare	870825a840fa501c686421ab4d85f59917a8e37f
Diff	Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Span - Duration of single invocation	2.85 ms	3.20 ms	1.12	0.31	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.Span(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.1987580321428566 > 3.014869194741587.
IsChangePoint: Marked as a change because one of 12/10/2022 1:39:11 AM, 1/11/2023 11:00:48 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -4.26484207927623 (T) = (0 -3365204.43748327) / Math.Sqrt((11813925920.546385 / (31)) + (274889782797.2209 / (26))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (26) - 2, .025) and -0.15292339361461602 = (2918844.787191599 - 3365204.43748327) / 2918844.787191599 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5385f045f9aee91c22922d21552c4317198710f1
Compare	870825a840fa501c686421ab4d85f59917a8e37f
Diff	Diff

Regressions in System.Tests.Perf_Byte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	12.77 ns	15.37 ns	1.20	0.57	False
ToString - Duration of single invocation	60.74 ns	153.73 ns	2.53	0.36	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.374555924731487 > 13.871340597901343.
IsChangePoint: Marked as a change because one of 11/21/2022 6:40:27 PM, 11/22/2022 6:55:51 AM, 12/1/2022 8:17:56 AM, 12/9/2022 1:52:32 PM, 1/11/2023 11:00:48 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -7.637189202615617 (T) = (0 -16.118933843081955) / Math.Sqrt((2.1154240154433146 / (31)) + (0.7489914394713282 / (26))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (26) - 2, .025) and -0.17315888802219678 = (13.73977046728642 - 16.118933843081955) / 13.73977046728642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Byte.ToString(value: 255)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 153.72557812147727 > 63.98279241850975.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -85.67227262356207 (T) = (0 -153.94390079268214) / Math.Sqrt((6.137269265008336 / (47)) + (9.94876989101981 / (10))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (10) - 2, .025) and -1.4413733611923796 = (63.05627121182937 - 153.94390079268214) / 63.05627121182937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[lewing]

looks like dotnet/runtime#80668 again