-
Notifications
You must be signed in to change notification settings - Fork 147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: Disable shared clock on version conflict #4459
base: master
Are you sure you want to change the base?
Conversation
Datadog ReportBranch report: ✅ |
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4459) - mean (3,028ms) : 2966, 3089
. : milestone, 3028,
master - mean (3,013ms) : 2909, 3117
. : milestone, 3013,
section CallTarget+Inlining+NGEN
This PR (4459) - mean (3,837ms) : 3722, 3952
. : milestone, 3837,
master - mean (3,829ms) : 3755, 3903
. : milestone, 3829,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4459) - mean (3,138ms) : 3030, 3246
. : milestone, 3138,
master - mean (3,126ms) : 3003, 3250
. : milestone, 3126,
section CallTarget+Inlining+NGEN
This PR (4459) - mean (3,615ms) : 3532, 3697
. : milestone, 3615,
master - mean (3,608ms) : 3564, 3653
. : milestone, 3608,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4459) - mean (3,110ms) : 2987, 3234
. : milestone, 3110,
master - mean (3,096ms) : 2995, 3197
. : milestone, 3096,
section CallTarget+Inlining+NGEN
This PR (4459) - mean (3,584ms) : 3509, 3660
. : milestone, 3584,
master - mean (3,583ms) : 3521, 3644
. : milestone, 3583,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4459) - mean (192ms) : 186, 197
. : milestone, 192,
master - mean (191ms) : 188, 194
. : milestone, 191,
section CallTarget+Inlining+NGEN
This PR (4459) - mean (1,124ms) : 1097, 1151
. : milestone, 1124,
master - mean (1,123ms) : 1097, 1149
. : milestone, 1123,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4459) - mean (380ms) : 370, 389
. : milestone, 380,
master - mean (372ms) : 367, 378
. : milestone, 372,
section CallTarget+Inlining+NGEN
This PR (4459) - mean (1,180ms) : 1150, 1210
. : milestone, 1180,
master - mean (1,175ms) : 1159, 1191
. : milestone, 1175,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (4459) - mean (364ms) : 358, 370
. : milestone, 364,
master - mean (360ms) : 357, 363
. : milestone, 360,
section CallTarget+Inlining+NGEN
This PR (4459) - mean (1,140ms) : 1112, 1168
. : milestone, 1140,
master - mean (1,127ms) : 1099, 1154
. : milestone, 1127,
|
Benchmarks Report 🐌Benchmarks for #4459 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.SpanBenchmark - Slower
|
Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 | 1.354 | 371.46 | 503.05 | |
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 | 1.149 | 450.42 | 517.64 | |
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 | 1.113 | 718.03 | 799.25 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | StartFinishSpan |
net6.0 | 372ns | 0.179ns | 0.622ns | 0.0076 | 0 | 0 | 536 B |
master | StartFinishSpan |
netcoreapp3.1 | 509ns | 0.228ns | 0.855ns | 0.00709 | 0 | 0 | 536 B |
master | StartFinishSpan |
net472 | 549ns | 0.129ns | 0.481ns | 0.0853 | 0 | 0 | 538 B |
master | StartFinishScope |
net6.0 | 450ns | 0.0998ns | 0.373ns | 0.0091 | 0 | 0 | 656 B |
master | StartFinishScope |
netcoreapp3.1 | 718ns | 0.343ns | 1.33ns | 0.00881 | 0 | 0 | 656 B |
master | StartFinishScope |
net472 | 751ns | 0.16ns | 0.597ns | 0.0979 | 0 | 0 | 618 B |
#4459 | StartFinishSpan |
net6.0 | 503ns | 0.151ns | 0.566ns | 0.0075 | 0 | 0 | 536 B |
#4459 | StartFinishSpan |
netcoreapp3.1 | 505ns | 0.27ns | 1.01ns | 0.00736 | 0 | 0 | 536 B |
#4459 | StartFinishSpan |
net472 | 602ns | 0.127ns | 0.459ns | 0.0854 | 0 | 0 | 538 B |
#4459 | StartFinishScope |
net6.0 | 518ns | 0.205ns | 0.793ns | 0.00922 | 0 | 0 | 656 B |
#4459 | StartFinishScope |
netcoreapp3.1 | 799ns | 0.403ns | 1.51ns | 0.00888 | 0 | 0 | 656 B |
#4459 | StartFinishScope |
net472 | 806ns | 0.275ns | 1.03ns | 0.098 | 0 | 0 | 618 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | RunOnMethodBegin |
net6.0 | 554ns | 0.143ns | 0.535ns | 0.00916 | 0 | 0 | 656 B |
master | RunOnMethodBegin |
netcoreapp3.1 | 786ns | 0.352ns | 1.32ns | 0.00873 | 0 | 0 | 656 B |
master | RunOnMethodBegin |
net472 | 916ns | 0.343ns | 1.28ns | 0.0979 | 0 | 0 | 618 B |
#4459 | RunOnMethodBegin |
net6.0 | 615ns | 0.209ns | 0.78ns | 0.00909 | 0 | 0 | 656 B |
#4459 | RunOnMethodBegin |
netcoreapp3.1 | 791ns | 1.5ns | 5.8ns | 0.00906 | 0 | 0 | 656 B |
#4459 | RunOnMethodBegin |
net472 | 934ns | 0.17ns | 0.658ns | 0.098 | 0 | 0 | 618 B |
Throughput/Crank Report:zap:Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4459) (11.211M) : 0, 11211012
master (10.877M) : 0, 10877040
benchmarks/2.34.0 (11.172M) : 0, 11172386
benchmarks/2.9.0 (11.226M) : 0, 11225900
section Automatic
This PR (4459) (7.843M) : 0, 7842583
master (7.669M) : 0, 7669221
benchmarks/2.34.0 (7.859M) : 0, 7858718
benchmarks/2.9.0 (8.128M) : 0, 8127670
section Trace stats
master (7.809M) : 0, 7808796
benchmarks/2.34.0 (7.905M) : 0, 7904802
section Manual
This PR (4459) (9.990M) : 0, 9990303
master (9.823M) : 0, 9822723
benchmarks/2.34.0 (9.986M) : 0, 9986137
section Manual + Automatic
This PR (4459) (7.542M) : 0, 7542458
master (7.406M) : 0, 7405833
benchmarks/2.34.0 (7.570M) : 0, 7570487
section Version Conflict
master (6.770M) : 0, 6769692
benchmarks/2.34.0 (6.749M) : 0, 6749094
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4459) (9.688M) : 0, 9688320
master (9.574M) : 0, 9574337
benchmarks/2.34.0 (9.689M) : 0, 9689253
benchmarks/2.9.0 (9.659M) : 0, 9658728
section Automatic
This PR (4459) (6.679M) : 0, 6679458
master (6.813M) : 0, 6812599
benchmarks/2.34.0 (6.670M) : 0, 6670396
section Trace stats
master (6.818M) : 0, 6817725
benchmarks/2.34.0 (6.743M) : 0, 6743225
section Manual
This PR (4459) (8.533M) : 0, 8532615
master (8.437M) : 0, 8437210
benchmarks/2.34.0 (8.614M) : 0, 8614349
section Manual + Automatic
This PR (4459) (6.542M) : 0, 6541500
master (6.566M) : 0, 6566202
benchmarks/2.34.0 (6.532M) : 0, 6532340
section Version Conflict
master (5.892M) : 0, 5892214
benchmarks/2.34.0 (5.854M) : 0, 5853936
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4459) (10.227M) : 0, 10226916
master (10.238M) : 0, 10238267
benchmarks/2.34.0 (10.187M) : 0, 10187020
benchmarks/2.9.0 (10.240M) : 0, 10239951
section Automatic
This PR (4459) (7.479M) : 0, 7478741
master (7.591M) : 0, 7591432
benchmarks/2.34.0 (7.402M) : 0, 7401872
benchmarks/2.9.0 (7.435M) : 0, 7435116
section Trace stats
master (7.326M) : 0, 7326243
benchmarks/2.34.0 (7.426M) : 0, 7426314
section Manual
This PR (4459) (9.208M) : 0, 9207993
master (9.424M) : 0, 9423734
benchmarks/2.34.0 (9.173M) : 0, 9173438
section Manual + Automatic
This PR (4459) (7.196M) : 0, 7195627
master (7.437M) : 0, 7437421
benchmarks/2.34.0 (7.218M) : 0, 7218183
section Version Conflict
master (6.561M) : 0, 6561453
benchmarks/2.34.0 (6.597M) : 0, 6597265
gantt
title Throughput Linux x64 (ASM) (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (4459) (7.319M) : 0, 7319272
master (7.353M) : 0, 7353218
benchmarks/2.34.0 (7.612M) : 0, 7611651
benchmarks/2.9.0 (7.840M) : 0, 7839825
section No attack
This PR (4459) (2.089M) : 0, 2089171
master (2.138M) : 0, 2137647
benchmarks/2.34.0 (2.156M) : 0, 2156076
benchmarks/2.9.0 (3.181M) : 0, 3180750
section Attack
This PR (4459) (1.812M) : 0, 1812053
master (1.816M) : 0, 1816342
benchmarks/2.9.0 (2.474M) : 0, 2474287
section Blocking
This PR (4459) (3.676M) : 0, 3675527
master (3.656M) : 0, 3655676
|
Summary of changes
This PR disabled the shared clock across multiple traces instances when a version conflict is detected.
Reason for change
See #4450, this is not a complete fix for the issue, but reduces the frequency of happening due to the (15ms) precision of the UtcNow.
For a complete fix we should create a
ITraceClock
interface and share it between the tracers, this can be done in a new IAutomaticTracer2, IManualTracer2 initiative.