Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 21 Improvements on 2/16/2023 6:27:39 AM #13318

Open
performanceautofiler bot opened this issue Feb 21, 2023 · 2 comments
Open

[Perf] Linux/x64: 21 Improvements on 2/16/2023 6:27:39 AM #13318

performanceautofiler bot opened this issue Feb 21, 2023 · 2 comments
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement untriaged

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS ubuntu 18.04
Baseline e56db76481726cc451b8fcf499500f9f3218de63
Compare f2f734a7e4f95d22a56994c127fe63d6aa97c780
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 23.14 μs 20.21 μs 0.87 0.05 False
SerializeToString - Duration of single invocation 12.29 μs 10.77 μs 0.88 0.03 False
SerializeToString - Duration of single invocation 18.43 μs 16.52 μs 0.90 0.06 False
SerializeObjectProperty - Duration of single invocation 25.62 μs 23.46 μs 0.92 0.15 False
SerializeToUtf8Bytes - Duration of single invocation 9.75 μs 8.56 μs 0.88 0.09 False
SerializeToWriter - Duration of single invocation 12.79 μs 11.35 μs 0.89 0.03 False
SerializeToUtf8Bytes - Duration of single invocation 15.92 μs 13.60 μs 0.85 0.09 False
SerializeToWriter - Duration of single invocation 7.72 μs 6.22 μs 0.81 0.12 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.210343583593463 < 21.47001691075655.
IsChangePoint: Marked as a change because one of 2/16/2023 3:28:45 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 20.161641339409314 (T) = (0 -20227.57322056809) / Math.Sqrt((236787.11903617776 / (35)) + (109360.03052376884 / (28))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .975) and 0.09335219448838204 = (22310.28752025021 - 20227.57322056809) / 22310.28752025021 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;.SerializeToString(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.76892269865765 < 11.797645310467443.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 22.735362483397097 (T) = (0 -10736.000270362261) / Math.Sqrt((71055.28135009317 / (31)) + (64165.093019083135 / (32))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (32) - 2, .975) and 0.12189894241098535 = (12226.383486930186 - 10736.000270362261) / 12226.383486930186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;.SerializeToString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.520739956523364 < 17.49570607810362.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 22.695996196317378 (T) = (0 -16309.58915901024) / Math.Sqrt((83164.24629586413 / (31)) + (147292.6789359768 / (32))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (32) - 2, .975) and 0.10616834209587796 = (18246.824236739787 - 16309.58915901024) / 18246.824236739787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;.SerializeObjectProperty(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.45533716334894 < 24.76371225775809.
IsChangePoint: Marked as a change because one of 2/16/2023 3:28:45 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 15.758835315331924 (T) = (0 -23769.421377689192) / Math.Sqrt((424091.0921809214 / (35)) + (193226.28107816857 / (28))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .975) and 0.08377034378849038 = (25942.645729207954 - 23769.421377689192) / 25942.645729207954 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;.SerializeToUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.563450335692252 < 9.444945229359574.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 28.739889466107098 (T) = (0 -8588.467100964936) / Math.Sqrt((35929.305542325106 / (31)) + (30362.285163633904 / (32))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (32) - 2, .975) and 0.13317395835141202 = (9907.947717665256 - 8588.467100964936) / 9907.947717665256 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.346061730519478 < 12.214402362702835.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 22.759580460517622 (T) = (0 -11443.783160100831) / Math.Sqrt((50713.55001978762 / (31)) + (75937.26560645056 / (32))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (32) - 2, .975) and 0.1118410460150791 = (12884.836783726354 - 11443.783160100831) / 12884.836783726354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;.SerializeToUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.595949101703342 < 15.063684552722462.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 23.780375563269196 (T) = (0 -13950.50058789241) / Math.Sqrt((71997.9361048211 / (30)) + (106663.0323341907 / (32))) is greater than 2.0002978220134553 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (32) - 2, .975) and 0.11431539050766888 = (15751.092926734664 - 13950.50058789241) / 15751.092926734664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.218886426717252 < 7.206855090614957.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 23.91982072480428 (T) = (0 -6375.0372743031085) / Math.Sqrt((32104.666786305716 / (30)) + (44611.59359471907 / (32))) is greater than 2.0002978220134553 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (32) - 2, .975) and 0.15701434525355534 = (7562.450485851516 - 6375.0372743031085) / 7562.450485851516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline e56db76481726cc451b8fcf499500f9f3218de63
Compare f2f734a7e4f95d22a56994c127fe63d6aa97c780
Diff Diff

Improvements in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FromBase64Chars - Duration of single invocation 1.28 μs 1.01 μs 0.79 0.06 False
ChangeType - Duration of single invocation 708.88 ns 509.41 ns 0.72 0.09 False
ToDateTime_String - Duration of single invocation 10.54 μs 9.65 μs 0.92 0.06 False
ToDateTime_String - Duration of single invocation 7.19 μs 5.85 μs 0.81 0.04 False
ToDateTime_String - Duration of single invocation 11.04 μs 9.19 μs 0.83 0.06 False
ToDateTime_String - Duration of single invocation 5.40 μs 4.39 μs 0.81 0.06 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.FromBase64Chars

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0051209044795657 < 1.1506823999857698.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/16/2023 3:28:45 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 47.83178597678201 (T) = (0 -1021.2558463156803) / Math.Sqrt((24223.525872396764 / (299)) + (306.4266658814064 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.30993421816983036 = (1479.9398451653976 - 1021.2558463156803) / 1479.9398451653976 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ChangeType

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 509.40956625500075 < 672.2828365324032.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 4.979634555383605 (T) = (0 -511.8941036447248) / Math.Sqrt((33835.37769323189 / (299)) + (241.29288331907972 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.09655537962876244 = (566.6026362904022 - 511.8941036447248) / 566.6026362904022 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "Thursday, February 26, 2009")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.65413215012869 < 10.033792582570216.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 38.92683395808854 (T) = (0 -9509.115672963311) / Math.Sqrt((719288.2764011706 / (299)) + (49373.36923457061 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.20562433590634188 = (11970.552602228474 - 9509.115672963311) / 11970.552602228474 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "February 26, 2009")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.850814491346494 < 6.795895197135093.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 44.98517359707882 (T) = (0 -6137.364666919675) / Math.Sqrt((318977.16020903434 / (299)) + (28213.955919374617 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.2444545823056162 = (8123.091641066935 - 6137.364666919675) / 8123.091641066935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "12/12/1999 11:59:59 PM")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.186536323744514 < 10.180001308128581.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 39.23538987227544 (T) = (0 -9381.927390880526) / Math.Sqrt((1010183.7949698779 / (299)) + (72419.3651209772 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.23903165682908914 = (12328.932570028577 - 9381.927390880526) / 12328.932570028577 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "12/12/1999")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.394418356142896 < 5.136147373447279.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.86714026585762 (T) = (0 -4467.720354157776) / Math.Sqrt((198827.5597143293 / (299)) + (18708.501408330598 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.278838032232629 = (6195.169121285319 - 4467.720354157776) / 6195.169121285319 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline e56db76481726cc451b8fcf499500f9f3218de63
Compare f2f734a7e4f95d22a56994c127fe63d6aa97c780
Diff Diff

Improvements in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Slice_Start - Duration of single invocation 40.17 ns 36.31 ns 0.90 0.12 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySequence*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySequence.Slice_Start(Segment: Single)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.31435002618232 < 38.622826608023885.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/16/2023 3:28:45 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 33.015482201033606 (T) = (0 -37.40257270091299) / Math.Sqrt((8.054683823976452 / (299)) + (1.0272773494256688 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.18210956870419523 = (45.730541976943215 - 37.40257270091299) / 45.730541976943215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline e56db76481726cc451b8fcf499500f9f3218de63
Compare f2f734a7e4f95d22a56994c127fe63d6aa97c780
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 826.70 μs 769.96 μs 0.93 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;TreeRecord&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 769.9606222527474 < 786.8606972355769.
IsChangePoint: Marked as a change because one of 2/16/2023 3:28:45 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 14.475627786552092 (T) = (0 -772942.1309853453) / Math.Sqrt((208515089.9837153 / (35)) + (105665608.26457335 / (28))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .975) and 0.05519732035064128 = (818099.0037752694 - 772942.1309853453) / 818099.0037752694 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline e56db76481726cc451b8fcf499500f9f3218de63
Compare f2f734a7e4f95d22a56994c127fe63d6aa97c780
Diff Diff

Improvements in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CtorIdnHostPathAndQuery - Duration of single invocation 35.43 μs 30.48 μs 0.86 0.02 False
UriBuilderReplacePort - Duration of single invocation 5.11 μs 4.15 μs 0.81 0.10 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://höst.with.ünicode/path/with?key=value#fragment")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.477357896611185 < 33.76828618866831.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/16/2023 3:28:45 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 34.08381219113493 (T) = (0 -30881.186769881264) / Math.Sqrt((10229495.004842605 / (196)) + (232298.10863251108 / (28))) is greater than 1.9707073953201992 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (196) + (28) - 2, .975) and 0.21349552599405083 = (39263.84119926529 - 30881.186769881264) / 39263.84119926529 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Uri.UriBuilderReplacePort

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.153406562722579 < 4.766567554726219.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 31.085214520612737 (T) = (0 -4203.796525404898) / Math.Sqrt((93993.267056324 / (195)) + (22655.67493397851 / (31))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (195) + (31) - 2, .975) and 0.20478546872896086 = (5286.367841751731 - 4203.796525404898) / 5286.367841751731 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline e56db76481726cc451b8fcf499500f9f3218de63
Compare f2f734a7e4f95d22a56994c127fe63d6aa97c780
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 11.44 ms 10.60 ms 0.93 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.JsonNet_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.5978231884058 < 10.830960775821994.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/16/2023 3:28:45 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.43320007832086 (T) = (0 -10498232.92191157) / Math.Sqrt((442542398349.17413 / (299)) + (10753512537.91674 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1485832181018247 = (12330310.072707845 - 10498232.92191157) / 12330310.072707845 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline e56db76481726cc451b8fcf499500f9f3218de63
Compare f2f734a7e4f95d22a56994c127fe63d6aa97c780
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 19.85 ms 17.56 ms 0.88 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.JsonNet_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.55596215909091 < 18.73537425179843.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/16/2023 3:28:45 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 29.26998656074804 (T) = (0 -17912381.730407022) / Math.Sqrt((1185519632285.2278 / (299)) + (99590409563.48065 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.12412787209149527 = (20450909.624422006 - 17912381.730407022) / 20450909.624422006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline e56db76481726cc451b8fcf499500f9f3218de63
Compare f2f734a7e4f95d22a56994c127fe63d6aa97c780
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 1.72 ms 1.63 ms 0.94 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.JsonNet_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6250794961622805 < 1.6348657480694129.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/16/2023 3:28:45 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 38.16759961306432 (T) = (0 -1589773.5508045573) / Math.Sqrt((8461924852.163627 / (299)) + (716450246.4342784 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14983400675581335 = (1869956.6478048232 - 1589773.5508045573) / 1869956.6478048232 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@radekdoulik
Copy link
Member

I think this is dotnet/runtime#82093 /cc @BrzVlad @lewing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@radekdoulik