Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lazy initialize the AggregatedMetrics. #4602

Merged
merged 2 commits into from
Sep 11, 2023
Merged

Lazy initialize the AggregatedMetrics. #4602

merged 2 commits into from
Sep 11, 2023

Conversation

andrewlock
Copy link
Member

Summary of changes

Removes the allocation of the metric tags array from the immediate startup path, and delays it till the first aggregation (10s later).

Reason for change

Tony was sad about startup allocations

Implementation details

Lazy<T> FTW

Test coverage

Nah, YOLO

@andrewlock andrewlock added type:performance Performance, speed, latency, resource usage (CPU, memory) area:telemetry labels Sep 8, 2023
@andrewlock andrewlock requested a review from a team as a code owner September 8, 2023 09:35
@github-actions github-actions bot added the area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) label Sep 8, 2023
@andrewlock
Copy link
Member Author

andrewlock commented Sep 8, 2023 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4602) - mean (75ms)  : 63, 87
     .   : milestone, 75,
    master - mean (70ms)  : 61, 79
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (4602) - mean (1,015ms)  : 962, 1069
     .   : milestone, 1015,
    master - mean (1,023ms)  : 980, 1067
     .   : milestone, 1023,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4602) - mean (107ms)  : 95, 119
     .   : milestone, 107,
    master - mean (106ms)  : 97, 116
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (4602) - mean (695ms)  : 662, 728
     .   : milestone, 695,
    master - mean (700ms)  : 675, 725
     .   : milestone, 700,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4602) - mean (90ms)  : 83, 97
     .   : milestone, 90,
    master - mean (89ms)  : 87, 92
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (4602) - mean (661ms)  : 625, 697
     .   : milestone, 661,
    master - mean (669ms)  : 633, 705
     .   : milestone, 669,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4602) - mean (190ms)  : 184, 196
     .   : milestone, 190,
    master - mean (189ms)  : 182, 195
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (4602) - mean (1,088ms)  : 1036, 1141
     .   : milestone, 1088,
    master - mean (1,103ms)  : 1067, 1139
     .   : milestone, 1103,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4602) - mean (273ms)  : 266, 280
     .   : milestone, 273,
    master - mean (270ms)  : 262, 278
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (4602) - mean (1,053ms)  : 1014, 1091
     .   : milestone, 1053,
    master - mean (1,057ms)  : 1017, 1097
     .   : milestone, 1057,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4602) - mean (284ms)  : 218, 349
     .   : milestone, 284,
    master - mean (258ms)  : 248, 269
     .   : milestone, 258,

    section CallTarget+Inlining+NGEN
    This PR (4602) - mean (1,004ms)  : 965, 1042
     .   : milestone, 1004,
    master - mean (1,012ms)  : 968, 1055
     .   : milestone, 1012,
Loading

@andrewlock
Copy link
Member Author

andrewlock commented Sep 8, 2023 •
edited
Loading

Benchmarks Report 🐌

Benchmarks for #4602 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.129
  • 1 benchmarks are slower, with geometric mean 1.343
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.26μs 44ns 220ns 0.0236 0.0118 0 7.3 KB
master StartStopWithChild netcoreapp3.1 10.1μs 52.5ns 257ns 0.0261 0.0105 0 7.39 KB
master StartStopWithChild net472 15.8μs 61.5ns 238ns 1.29 0.34 0.108 7.67 KB
#4602 StartStopWithChild net6.0 8.21μs 43ns 219ns 0.0233 0.0117 0 7.3 KB
#4602 StartStopWithChild netcoreapp3.1 10.3μs 54.9ns 275ns 0.0249 0.00997 0 7.39 KB
#4602 StartStopWithChild net472 15.7μs 49.1ns 190ns 1.28 0.338 0.102 7.67 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 484μs 475ns 1.84μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 659μs 499ns 1.87μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 812μs 360ns 1.4μs 0.401 0 0 3.3 KB
#4602 WriteAndFlushEnrichedTraces net6.0 467μs 266ns 997ns 0 0 0 2.7 KB
#4602 WriteAndFlushEnrichedTraces netcoreapp3.1 628μs 351ns 1.36μs 0 0 0 2.7 KB
#4602 WriteAndFlushEnrichedTraces net472 805μs 402ns 1.56μs 0.401 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 39.3μs 18.7ns 72.4ns 0.0197 0 0 1.69 KB
master AllCycleSimpleBody netcoreapp3.1 41.4μs 26.2ns 98ns 0.0206 0 0 1.67 KB
master AllCycleSimpleBody net472 42.8μs 19.1ns 71.5ns 0.256 0 0 1.73 KB
master AllCycleMoreComplexBody net6.0 230μs 224ns 839ns 0.114 0 0 9.26 KB
master AllCycleMoreComplexBody netcoreapp3.1 237μs 97ns 376ns 0.12 0 0 9.16 KB
master AllCycleMoreComplexBody net472 249μs 57.3ns 214ns 1.48 0 0 9.33 KB
master ObjectExtractorSimpleBody net6.0 126ns 0.0913ns 0.354ns 0.00395 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 165ns 0.0607ns 0.235ns 0.00377 0 0 272 B
master ObjectExtractorSimpleBody net472 144ns 0.227ns 0.848ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.09μs 0.93ns 3.48ns 0.0537 0 0 3.88 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.04μs 1.14ns 4.11ns 0.0507 0 0 3.78 KB
master ObjectExtractorMoreComplexBody net472 4.1μs 5.36ns 20.8ns 0.618 0.00612 0 3.89 KB
#4602 AllCycleSimpleBody net6.0 38.4μs 66.3ns 257ns 0.0193 0 0 1.69 KB
#4602 AllCycleSimpleBody netcoreapp3.1 40.8μs 60.3ns 233ns 0.0204 0 0 1.67 KB
#4602 AllCycleSimpleBody net472 44.3μs 52.6ns 204ns 0.271 0 0 1.73 KB
#4602 AllCycleMoreComplexBody net6.0 233μs 45.2ns 157ns 0.116 0 0 9.26 KB
#4602 AllCycleMoreComplexBody netcoreapp3.1 240μs 273ns 1.02μs 0.12 0 0 9.16 KB
#4602 AllCycleMoreComplexBody net472 250μs 286ns 1.11μs 1.37 0 0 9.33 KB
#4602 ObjectExtractorSimpleBody net6.0 120ns 0.122ns 0.473ns 0.00396 0 0 280 B
#4602 ObjectExtractorSimpleBody netcoreapp3.1 178ns 0.114ns 0.425ns 0.00368 0 0 272 B
#4602 ObjectExtractorSimpleBody net472 149ns 0.0969ns 0.375ns 0.0446 0 0 281 B
#4602 ObjectExtractorMoreComplexBody net6.0 3.12μs 0.801ns 3ns 0.054 0 0 3.88 KB
#4602 ObjectExtractorMoreComplexBody netcoreapp3.1 4.03μs 1.01ns 3.77ns 0.0524 0 0 3.78 KB
#4602 ObjectExtractorMoreComplexBody net472 4.21μs 2.5ns 9.36ns 0.617 0.00634 0 3.89 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 152ms 195μs 754μs 0 0 0 33.94 MB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 165ms 225μs 811μs 0 0 0 33.94 MB
master RunWaf(args=NestedMap (10)) net472 241ms 1.3ms 6.73ms 6E+03 3E+03 2E+03 35.2 MB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 167ms 194μs 673μs 0 0 0 33.95 MB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 186ms 172μs 666μs 0 0 0 33.95 MB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 254ms 257μs 963μs 5E+03 2E+03 1E+03 35.2 MB
master RunWaf(args=NestedMap (100)) net6.0 304ms 697μs 2.7ms 0 0 0 73.6 MB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 346ms 1.02ms 3.94ms 0 0 0 74.91 MB
master RunWaf(args=NestedMap (100)) net472 508ms 1.02ms 3.82ms 1.1E+04 5E+03 2E+03 78.67 MB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 328ms 891μs 3.45ms 0 0 0 73.6 MB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 368ms 546μs 2.04ms 0 0 0 74.91 MB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 540ms 538μs 2.08ms 1.1E+04 5E+03 2E+03 78.7 MB
master RunWaf(args=NestedMap (1000)) net6.0 305ms 712μs 2.57ms 0 0 0 73.6 MB
master RunWaf(args=NestedMap (1000)) netcoreapp3.1 347ms 1.07ms 4.13ms 0 0 0 74.99 MB
master RunWaf(args=NestedMap (1000)) net472 503ms 1.15ms 4.32ms 1.1E+04 5E+03 2E+03 78.66 MB
master RunWafWithAttack(args=Neste(...)tack) [24]) net6.0 323ms 692μs 2.59ms 0 0 0 73.6 MB
master RunWafWithAttack(args=Neste(...)tack) [24]) netcoreapp3.1 372ms 672μs 2.42ms 0 0 0 74.91 MB
master RunWafWithAttack(args=Neste(...)tack) [24]) net472 545ms 1.8ms 6.95ms 1.1E+04 5E+03 2E+03 78.66 MB
#4602 RunWaf(args=NestedMap (10)) net6.0 152ms 159μs 614μs 0 0 0 33.94 MB
#4602 RunWaf(args=NestedMap (10)) netcoreapp3.1 164ms 276μs 957μs 0 0 0 33.94 MB
#4602 RunWaf(args=NestedMap (10)) net472 246ms 1.22ms 5.58ms 6E+03 3E+03 2E+03 35.19 MB
#4602 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 167ms 403μs 1.56ms 0 0 0 33.95 MB
#4602 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 185ms 267μs 1ms 0 0 0 33.95 MB
#4602 RunWafWithAttack(args=Neste(...)tack) [22]) net472 250ms 182μs 656μs 5E+03 2E+03 1E+03 35.2 MB
#4602 RunWaf(args=NestedMap (100)) net6.0 302ms 710μs 2.75ms 0 0 0 73.68 MB
#4602 RunWaf(args=NestedMap (100)) netcoreapp3.1 337ms 958μs 3.58ms 0 0 0 74.91 MB
#4602 RunWaf(args=NestedMap (100)) net472 503ms 1.89ms 7.31ms 1.1E+04 5E+03 2E+03 78.67 MB
#4602 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 327ms 672μs 2.51ms 0 0 0 73.69 MB
#4602 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 371ms 982μs 3.8ms 0 0 0 74.91 MB
#4602 RunWafWithAttack(args=Neste(...)tack) [23]) net472 541ms 2.01ms 7.52ms 1.1E+04 5E+03 2E+03 78.67 MB
#4602 RunWaf(args=NestedMap (1000)) net6.0 292ms 341μs 1.23ms 0 0 0 73.68 MB
#4602 RunWaf(args=NestedMap (1000)) netcoreapp3.1 342ms 992μs 3.71ms 0 0 0 74.91 MB
#4602 RunWaf(args=NestedMap (1000)) net472 507ms 552μs 2.06ms 1.1E+04 5E+03 2E+03 78.72 MB
#4602 RunWafWithAttack(args=Neste(...)tack) [24]) net6.0 327ms 1.06ms 3.96ms 0 0 0 73.69 MB
#4602 RunWafWithAttack(args=Neste(...)tack) [24]) netcoreapp3.1 369ms 511μs 1.84ms 0 0 0 74.91 MB
#4602 RunWafWithAttack(args=Neste(...)tack) [24]) net472 536ms 1.76ms 6.6ms 1.1E+04 5E+03 2E+03 78.67 MB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 171μs 183ns 709ns 0.256 0 0 18.09 KB
master SendRequest netcoreapp3.1 189μs 193ns 696ns 0.187 0 0 20.25 KB
master SendRequest net472 0.000762ns 0.000267ns 0.001ns 0 0 0 0 b
#4602 SendRequest net6.0 169μs 94.8ns 367ns 0.255 0 0 18.09 KB
#4602 SendRequest netcoreapp3.1 190μs 295ns 1.14μs 0.187 0 0 20.25 KB
#4602 SendRequest net472 0.00018ns 0.000103ns 0.000397ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 523μs 810ns 3.14μs 0.514 0 0 41.48 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 623μs 747ns 2.89μs 0.314 0 0 41.86 KB
master WriteAndFlushEnrichedTraces net472 778μs 3.9μs 17μs 8.1 2.31 0.386 53.23 KB
#4602 WriteAndFlushEnrichedTraces net6.0 543μs 1.19μs 4.6μs 0.532 0 0 41.55 KB
#4602 WriteAndFlushEnrichedTraces netcoreapp3.1 612μs 1.43μs 5.53μs 0.308 0 0 41.78 KB
#4602 WriteAndFlushEnrichedTraces net472 802μs 3.32μs 12.9μs 8.33 2.65 0.379 53.25 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.04μs 0.372ns 1.39ns 0.0109 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.3μs 0.45ns 1.69ns 0.00994 0 0 768 B
master ExecuteNonQuery net472 1.59μs 1.15ns 4.3ns 0.116 0.000789 0 730 B
#4602 ExecuteNonQuery net6.0 1μs 0.801ns 3.1ns 0.011 0 0 768 B
#4602 ExecuteNonQuery netcoreapp3.1 1.31μs 1.23ns 4.78ns 0.0103 0 0 768 B
#4602 ExecuteNonQuery net472 1.59μs 0.55ns 2.06ns 0.116 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.21μs 0.405ns 1.57ns 0.014 0 0 992 B
master CallElasticsearch netcoreapp3.1 1.4μs 0.727ns 2.62ns 0.0134 0 0 992 B
master CallElasticsearch net472 2.33μs 0.723ns 2.8ns 0.159 0.00116 0 1 KB
master CallElasticsearchAsync net6.0 1.25μs 0.547ns 2.12ns 0.0137 0 0 968 B
master CallElasticsearchAsync netcoreapp3.1 1.65μs 1.62ns 6.29ns 0.014 0 0 1.04 KB
master CallElasticsearchAsync net472 2.57μs 0.431ns 1.67ns 0.168 0.00128 0 1.06 KB
#4602 CallElasticsearch net6.0 1.24μs 0.533ns 1.99ns 0.0134 0 0 992 B
#4602 CallElasticsearch netcoreapp3.1 1.51μs 0.64ns 2.4ns 0.0129 0 0 992 B
#4602 CallElasticsearch net472 2.31μs 0.827ns 3.2ns 0.159 0.00115 0 1 KB
#4602 CallElasticsearchAsync net6.0 1.28μs 1.59ns 6.16ns 0.0136 0 0 968 B
#4602 CallElasticsearchAsync netcoreapp3.1 1.57μs 3.46ns 12.5ns 0.0139 0 0 1.04 KB
#4602 CallElasticsearchAsync net472 2.47μs 0.72ns 2.69ns 0.168 0 0 1.06 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.12μs 0.501ns 1.87ns 0.0128 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.48μs 1.27ns 4.38ns 0.0118 0 0 912 B
master ExecuteAsync net472 1.72μs 0.5ns 1.87ns 0.139 0.000856 0 875 B
#4602 ExecuteAsync net6.0 1.23μs 0.676ns 2.62ns 0.013 0 0 912 B
#4602 ExecuteAsync netcoreapp3.1 1.5μs 0.731ns 2.73ns 0.0121 0 0 912 B
#4602 ExecuteAsync net472 1.81μs 1.03ns 4ns 0.139 0.000902 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 3.64μs 1.62ns 6.06ns 0.0274 0 0 1.94 KB
master SendAsync netcoreapp3.1 4.51μs 1.18ns 4.55ns 0.0339 0 0 2.48 KB
master SendAsync net472 7.2μs 2.22ns 8.61ns 0.483 0 0 3.05 KB
#4602 SendAsync net6.0 3.76μs 0.907ns 3.4ns 0.0262 0 0 1.94 KB
#4602 SendAsync netcoreapp3.1 4.46μs 1.33ns 5.16ns 0.0335 0 0 2.48 KB
#4602 SendAsync net472 7.17μs 1.7ns 6.59ns 0.484 0 0 3.05 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.34μs 0.673ns 2.52ns 0.0227 0 0 1.62 KB
master EnrichedLog netcoreapp3.1 1.99μs 0.908ns 3.4ns 0.0219 0 0 1.62 KB
master EnrichedLog net472 2.33μs 1.78ns 6.67ns 0.244 0 0 1.54 KB
#4602 EnrichedLog net6.0 1.29μs 0.903ns 3.38ns 0.0228 0 0 1.62 KB
#4602 EnrichedLog netcoreapp3.1 1.97μs 0.668ns 2.5ns 0.0218 0 0 1.62 KB
#4602 EnrichedLog net472 2.4μs 2.15ns 8.34ns 0.244 0 0 1.54 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 112μs 134ns 500ns 0.0566 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 119μs 247ns 955ns 0.0594 0 0 4.21 KB
master EnrichedLog net472 147μs 90.3ns 338ns 0.659 0.22 0 4.38 KB
#4602 EnrichedLog net6.0 111μs 187ns 725ns 0.0557 0 0 4.21 KB
#4602 EnrichedLog netcoreapp3.1 118μs 271ns 1.05μs 0.0586 0 0 4.21 KB
#4602 EnrichedLog net472 149μs 142ns 551ns 0.674 0.225 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.83μs 0.664ns 2.39ns 0.0312 0 0 2.18 KB
master EnrichedLog netcoreapp3.1 3.81μs 1.46ns 5.67ns 0.0286 0 0 2.18 KB
master EnrichedLog net472 4.44μs 1.49ns 5.36ns 0.316 0 0 1.99 KB
#4602 EnrichedLog net6.0 2.79μs 1.76ns 6.8ns 0.0305 0 0 2.18 KB
#4602 EnrichedLog netcoreapp3.1 3.9μs 1.41ns 5.09ns 0.0293 0 0 2.18 KB
#4602 EnrichedLog net472 4.44μs 1.32ns 5.1ns 0.315 0 0 1.99 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.36μs 0.764ns 2.96ns 0.0164 0 0 1.16 KB
master SendReceive netcoreapp3.1 1.61μs 1.46ns 5.46ns 0.016 0 0 1.16 KB
master SendReceive net472 2.09μs 0.861ns 3.22ns 0.184 0.00104 0 1.16 KB
#4602 SendReceive net6.0 1.3μs 0.619ns 2.31ns 0.0162 0 0 1.16 KB
#4602 SendReceive netcoreapp3.1 1.64μs 0.608ns 2.19ns 0.0156 0 0 1.16 KB
#4602 SendReceive net472 2.04μs 1.65ns 6.39ns 0.184 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.56μs 0.721ns 2.7ns 0.0205 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.66μs 1.15ns 4.32ns 0.0202 0 0 1.58 KB
master EnrichedLog net472 3.95μs 1.62ns 6.05ns 0.31 0 0 1.96 KB
#4602 EnrichedLog net6.0 2.6μs 1ns 3.76ns 0.0208 0 0 1.53 KB
#4602 EnrichedLog netcoreapp3.1 3.66μs 2.2ns 8.24ns 0.0219 0 0 1.58 KB
#4602 EnrichedLog net472 3.96μs 1.15ns 4.47ns 0.311 0 0 1.96 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4602

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.343 489.31 656.98

Faster 🎉 in #4602

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 1.140 610.97 536.02

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 464ns 0.13ns 0.505ns 0.00746 0 0 536 B
master StartFinishSpan netcoreapp3.1 609ns 1.32ns 5.1ns 0.00733 0 0 536 B
master StartFinishSpan net472 607ns 0.133ns 0.479ns 0.0854 0 0 538 B
master StartFinishScope net6.0 489ns 0.136ns 0.509ns 0.00925 0 0 656 B
master StartFinishScope netcoreapp3.1 715ns 0.313ns 1.13ns 0.00898 0 0 656 B
master StartFinishScope net472 838ns 0.277ns 0.998ns 0.0979 0 0 618 B
#4602 StartFinishSpan net6.0 450ns 0.112ns 0.418ns 0.00754 0 0 536 B
#4602 StartFinishSpan netcoreapp3.1 536ns 0.258ns 0.966ns 0.00721 0 0 536 B
#4602 StartFinishSpan net472 666ns 0.378ns 1.46ns 0.0852 0 0 538 B
#4602 StartFinishScope net6.0 657ns 0.215ns 0.834ns 0.00925 0 0 656 B
#4602 StartFinishScope netcoreapp3.1 664ns 0.231ns 0.799ns 0.00872 0 0 656 B
#4602 StartFinishScope net472 805ns 0.451ns 1.75ns 0.098 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #4602

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 1.119 1,060.73 948.10

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 558ns 0.13ns 0.487ns 0.00922 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 800ns 0.342ns 1.33ns 0.00874 0 0 656 B
master RunOnMethodBegin net472 1.06μs 0.356ns 1.38ns 0.0978 0 0 618 B
#4602 RunOnMethodBegin net6.0 594ns 0.181ns 0.7ns 0.00918 0 0 656 B
#4602 RunOnMethodBegin netcoreapp3.1 825ns 0.258ns 0.966ns 0.00904 0 0 656 B
#4602 RunOnMethodBegin net472 948ns 0.317ns 1.23ns 0.0981 0 0 618 B

@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Sep 8, 2023 •
edited
Loading

Datadog Report

Branch report: andrew/telemetry/lazy-initialize-tags
Commit report: c022db3

✅ dd-trace-dotnet: 0 Failed, 0 New Flaky, 301626 Passed, 1115 Skipped, 21m 43.36s Wall Time

@andrewlock
Copy link
Member Author

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4602) (11.225M)   : 0, 11224517
    master (11.190M)   : 0, 11190428
    benchmarks/2.37.0 (11.539M)   : 0, 11539240
    benchmarks/2.9.0 (11.212M)   : 0, 11211637

    section Automatic
    This PR (4602) (7.817M)   : 0, 7816731
    master (7.830M)   : 0, 7829725
    benchmarks/2.37.0 (7.743M)   : 0, 7743046
    benchmarks/2.9.0 (8.139M)   : 0, 8139111

    section Trace stats
    master (7.824M)   : 0, 7824456
    benchmarks/2.37.0 (7.822M)   : 0, 7822046

    section Manual
    This PR (4602) (9.977M)   : 0, 9976999
    master (9.958M)   : 0, 9957707
    benchmarks/2.37.0 (9.930M)   : 0, 9930410

    section Manual + Automatic
    This PR (4602) (7.492M)   : 0, 7491961
    master (7.503M)   : 0, 7502753
    benchmarks/2.37.0 (7.420M)   : 0, 7419660

    section Version Conflict
    master (6.780M)   : 0, 6779967
    benchmarks/2.37.0 (6.715M)   : 0, 6714611
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4602) (9.746M)   : 0, 9746108
    master (9.257M)   : 0, 9256536
    benchmarks/2.37.0 (9.762M)   : 0, 9762293
    benchmarks/2.9.0 (9.520M)   : 0, 9520167

    section Automatic
    This PR (4602) (6.735M)   : 0, 6735265
    master (6.679M)   : 0, 6679060
    benchmarks/2.37.0 (6.804M)   : 0, 6804305

    section Trace stats
    master (6.785M)   : 0, 6785175
    benchmarks/2.37.0 (6.664M)   : 0, 6663814

    section Manual
    This PR (4602) (8.546M)   : 0, 8546202
    master (8.687M)   : 0, 8686994
    benchmarks/2.37.0 (8.413M)   : 0, 8412763

    section Manual + Automatic
    This PR (4602) (6.575M)   : 0, 6575060
    master (6.493M)   : 0, 6492737
    benchmarks/2.37.0 (6.481M)   : 0, 6480877

    section Version Conflict
    master (5.875M)   : 0, 5874843
    benchmarks/2.37.0 (5.967M)   : 0, 5967147
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4602) (10.250M)   : 0, 10250373
    master (10.281M)   : 0, 10281269
    benchmarks/2.37.0 (10.740M)   : 0, 10739727
    benchmarks/2.9.0 (10.546M)   : 0, 10546222

    section Automatic
    This PR (4602) (7.403M)   : 0, 7402860
    master (7.576M)   : 0, 7575616
    benchmarks/2.37.0 (7.737M)   : 0, 7737095
    benchmarks/2.9.0 (7.729M)   : 0, 7728832

    section Trace stats
    master (7.240M)   : 0, 7239702
    benchmarks/2.37.0 (7.556M)   : 0, 7556168

    section Manual
    This PR (4602) (9.359M)   : 0, 9359348
    master (9.199M)   : 0, 9198947
    benchmarks/2.37.0 (9.693M)   : 0, 9693280

    section Manual + Automatic
    This PR (4602) (7.301M)   : 0, 7300740
    master (7.133M)   : 0, 7132686
    benchmarks/2.37.0 (7.343M)   : 0, 7343098

    section Version Conflict
    master (6.756M)   : 0, 6755701
    benchmarks/2.37.0 (6.804M)   : 0, 6803632
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4602) (6.969M)   : 0, 6969241
    master (7.057M)   : 0, 7056686
    benchmarks/2.37.0 (7.625M)   : 0, 7624599
    benchmarks/2.9.0 (7.810M)   : 0, 7809514

    section No attack
    This PR (4602) (2.058M)   : 0, 2057509
    master (2.050M)   : 0, 2049627
    benchmarks/2.37.0 (2.206M)   : 0, 2206160
    benchmarks/2.9.0 (3.318M)   : 0, 3317796

    section Attack
    This PR (4602) (1.639M)   : 0, 1638505
    master (1.623M)   : 0, 1623242
    benchmarks/2.37.0 (1.704M)   : 0, 1703946
    benchmarks/2.9.0 (2.513M)   : 0, 2512948

    section Blocking
    This PR (4602) (2.971M)   : 0, 2970870
    master (2.984M)   : 0, 2983775
    benchmarks/2.37.0 (3.186M)   : 0, 3185505
Loading

@andrewlock
Lazy initialize the AggregatedMetrics.
e40713b 
This removes the allocation of the metric tags from the immediate startup path, and delays it till the first aggregation (10s later).
@andrewlock
Update MetricsTelemetryCollector.cs
c022db3
@andrewlock andrewlock force-pushed the andrew/telemetry/lazy-initialize-tags branch from 7ac7f5b to c022db3 Compare September 11, 2023 09:27
@andrewlock andrewlock merged commit 9d9e796 into master Sep 11, 2023
49 of 51 checks passed
@andrewlock andrewlock deleted the andrew/telemetry/lazy-initialize-tags branch September 11, 2023 10:41
@github-actions github-actions bot added this to the vNext milestone Sep 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tonyredondo tonyredondo tonyredondo approved these changes

@kevingosse kevingosse kevingosse approved these changes

Assignees
No one assigned
Labels
area:telemetry area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) type:performance Performance, speed, latency, resource usage (CPU, memory)
Projects
None yet
Milestone
2.38.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@andrewlock @tonyredondo @kevingosse