Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ASM] Fix IAST Grpc DotNet flaky test #6772

Merged
merged 1 commit into from
Mar 17, 2025
Merged

[ASM] Fix IAST Grpc DotNet flaky test #6772

merged 1 commit into from
Mar 17, 2025
+5 −12

Conversation

e-n-0
Copy link
Member

@e-n-0 e-n-0 commented Mar 17, 2025

Summary of changes

Disabling SpanMetaStructs from Agent used in related test and snapshot file.

Reason for change

The test started to be flaky in master.

Implementation details

Test coverage

Other details

@e-n-0 e-n-0 requested a review from a team as a code owner March 17, 2025 10:42
@github-actions GitHub Actions
Copy link
Contributor

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

4 occurrences of :

-    },
-    MetaStruct: {
-      iast:

@github-actions github-actions bot added area:tests unit tests, integration tests area:asm labels Mar 17, 2025
andrewlock
andrewlock approved these changes Mar 17, 2025
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@andrewlock andrewlock mentioned this pull request Mar 17, 2025
Closed
@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Mar 17, 2025 •
edited
Loading

Datadog Report

Branch report: flavien/asm/fix-fllaky-grpc-iast
Commit report: 8412282
Test service: dd-trace-dotnet

✅ 0 Failed, 250475 Passed, 2419 Skipped, 20h 15m 28.42s Total Time

@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6772) - mean (69ms)  : 65, 73
     .   : milestone, 69,
    master - mean (69ms)  : 67, 72
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6772) - mean (1,004ms)  : 980, 1029
     .   : milestone, 1004,
    master - mean (1,005ms)  : 983, 1027
     .   : milestone, 1005,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6772) - mean (102ms)  : 100, 104
     .   : milestone, 102,
    master - mean (103ms)  : 101, 105
     .   : milestone, 103,

    section CallTarget+Inlining+NGEN
    This PR (6772) - mean (684ms)  : 669, 698
     .   : milestone, 684,
    master - mean (687ms)  : 668, 707
     .   : milestone, 687,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6772) - mean (89ms)  : 87, 91
     .   : milestone, 89,
    master - mean (90ms)  : 88, 91
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (6772) - mean (642ms)  : 625, 658
     .   : milestone, 642,
    master - mean (644ms)  : 624, 664
     .   : milestone, 644,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6772) - mean (190ms)  : 187, 194
     .   : milestone, 190,
    master - mean (190ms)  : 187, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6772) - mean (1,108ms)  : 1082, 1133
     .   : milestone, 1108,
    master - mean (1,106ms)  : 1080, 1133
     .   : milestone, 1106,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6772) - mean (270ms)  : 266, 274
     .   : milestone, 270,
    master - mean (270ms)  : 265, 274
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (6772) - mean (877ms)  : 842, 911
     .   : milestone, 877,
    master - mean (874ms)  : 851, 897
     .   : milestone, 874,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6772) - mean (261ms)  : 257, 266
     .   : milestone, 261,
    master - mean (262ms)  : 259, 266
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (6772) - mean (858ms)  : 822, 894
     .   : milestone, 858,
    master - mean (856ms)  : 825, 887
     .   : milestone, 856,

@andrewlock
Copy link
Member

Benchmarks Report for tracer 🐌

Benchmarks for #6772 compared to master:

  • 3 benchmarks are faster, with geometric mean 1.204
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.02μs 43.7ns 255ns 0.0152 0.0076 0 5.61 KB
master StartStopWithChild netcoreapp3.1 10.1μs 53.5ns 268ns 0.0185 0.00926 0 5.8 KB
master StartStopWithChild net472 15.9μs 34.8ns 125ns 1.05 0.31 0.103 6.21 KB
#6772 StartStopWithChild net6.0 7.97μs 46.2ns 372ns 0.0175 0.00876 0 5.6 KB
#6772 StartStopWithChild netcoreapp3.1 10.3μs 58.7ns 407ns 0.0205 0.0102 0 5.8 KB
#6772 StartStopWithChild net472 16.1μs 53.8ns 208ns 1.03 0.306 0.0967 6.21 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 513μs 639ns 2.47μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 648μs 632ns 2.45μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 865μs 901ns 3.49μs 0.431 0 0 3.3 KB
#6772 WriteAndFlushEnrichedTraces net6.0 529μs 363ns 1.36μs 0 0 0 2.7 KB
#6772 WriteAndFlushEnrichedTraces netcoreapp3.1 673μs 835ns 3.23μs 0 0 0 2.7 KB
#6772 WriteAndFlushEnrichedTraces net472 860μs 891ns 3.21μs 0.425 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 132μs 527ns 2.04μs 0.133 0 0 14.47 KB
master SendRequest netcoreapp3.1 150μs 329ns 1.27μs 0.15 0 0 17.27 KB
master SendRequest net472 0.000368ns 0.000142ns 0.000548ns 0 0 0 0 b
#6772 SendRequest net6.0 132μs 513ns 1.99μs 0.132 0 0 14.47 KB
#6772 SendRequest netcoreapp3.1 147μs 335ns 1.25μs 0.218 0 0 17.27 KB
#6772 SendRequest net472 0.00101ns 0.000286ns 0.00107ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 592μs 3.4μs 28.8μs 0.541 0 0 41.6 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 726μs 4.08μs 37.4μs 0.353 0 0 41.77 KB
master WriteAndFlushEnrichedTraces net472 872μs 4.34μs 19.9μs 8.08 2.55 0.425 53.28 KB
#6772 WriteAndFlushEnrichedTraces net6.0 576μs 3.2μs 19.7μs 0.331 0 0 41.78 KB
#6772 WriteAndFlushEnrichedTraces netcoreapp3.1 729μs 4.21μs 33.7μs 0.343 0 0 41.82 KB
#6772 WriteAndFlushEnrichedTraces net472 880μs 4.21μs 16.3μs 8.13 2.57 0.428 53.3 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.34μs 1.46ns 5.66ns 0.0145 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.85μs 2.48ns 9.6ns 0.0131 0 0 1.02 KB
master ExecuteNonQuery net472 2.03μs 1.02ns 3.66ns 0.156 0.00101 0 987 B
#6772 ExecuteNonQuery net6.0 1.41μs 1.63ns 6.3ns 0.0145 0 0 1.02 KB
#6772 ExecuteNonQuery netcoreapp3.1 1.8μs 1.76ns 6.35ns 0.0136 0 0 1.02 KB
#6772 ExecuteNonQuery net472 2.09μs 3.29ns 12.7ns 0.157 0.00107 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.18μs 0.505ns 1.89ns 0.0135 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.62μs 1.83ns 6.35ns 0.0132 0 0 976 B
master CallElasticsearch net472 2.57μs 1.65ns 6.4ns 0.157 0 0 995 B
master CallElasticsearchAsync net6.0 1.31μs 0.742ns 2.78ns 0.0131 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.66μs 0.987ns 3.69ns 0.0137 0 0 1.02 KB
master CallElasticsearchAsync net472 2.71μs 0.944ns 3.53ns 0.166 0 0 1.05 KB
#6772 CallElasticsearch net6.0 1.26μs 0.49ns 1.9ns 0.0133 0 0 976 B
#6772 CallElasticsearch netcoreapp3.1 1.54μs 0.671ns 2.51ns 0.0132 0 0 976 B
#6772 CallElasticsearch net472 2.66μs 1.46ns 5.67ns 0.157 0 0 995 B
#6772 CallElasticsearchAsync net6.0 1.3μs 0.858ns 3.32ns 0.0135 0 0 952 B
#6772 CallElasticsearchAsync netcoreapp3.1 1.68μs 0.666ns 2.58ns 0.0136 0 0 1.02 KB
#6772 CallElasticsearchAsync net472 2.84μs 1.92ns 7.42ns 0.166 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6772

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 1.221 1,393.44 1,141.52

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.39μs 0.442ns 1.71ns 0.0135 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.62μs 0.508ns 1.83ns 0.013 0 0 952 B
master ExecuteAsync net472 1.85μs 0.519ns 2.01ns 0.144 0 0 915 B
#6772 ExecuteAsync net6.0 1.14μs 1.38ns 5.15ns 0.0136 0 0 952 B
#6772 ExecuteAsync netcoreapp3.1 1.69μs 0.583ns 2.18ns 0.0127 0 0 952 B
#6772 ExecuteAsync net472 1.77μs 0.298ns 1.07ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.31μs 2.33ns 8.41ns 0.0324 0 0 2.31 KB
master SendAsync netcoreapp3.1 5.51μs 2.65ns 10.2ns 0.0385 0 0 2.85 KB
master SendAsync net472 7.47μs 2.4ns 8.97ns 0.493 0 0 3.12 KB
#6772 SendAsync net6.0 4.52μs 2.83ns 10.6ns 0.0318 0 0 2.31 KB
#6772 SendAsync netcoreapp3.1 5.31μs 3.7ns 14.3ns 0.0372 0 0 2.85 KB
#6772 SendAsync net472 7.47μs 2.18ns 8.43ns 0.493 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.52μs 0.618ns 2.23ns 0.0228 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.11μs 0.772ns 2.78ns 0.0222 0 0 1.64 KB
master EnrichedLog net472 2.47μs 0.796ns 3.08ns 0.249 0 0 1.57 KB
#6772 EnrichedLog net6.0 1.45μs 0.593ns 2.22ns 0.0232 0 0 1.64 KB
#6772 EnrichedLog netcoreapp3.1 2.26μs 1.13ns 4.25ns 0.0225 0 0 1.64 KB
#6772 EnrichedLog net472 2.55μs 1.03ns 3.73ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 111μs 126ns 471ns 0.0555 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 117μs 243ns 941ns 0.0578 0 0 4.28 KB
master EnrichedLog net472 149μs 149ns 578ns 0.671 0.224 0 4.46 KB
#6772 EnrichedLog net6.0 111μs 131ns 507ns 0.0556 0 0 4.28 KB
#6772 EnrichedLog netcoreapp3.1 115μs 202ns 782ns 0 0 0 4.28 KB
#6772 EnrichedLog net472 148μs 79.5ns 308ns 0.662 0.221 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.98μs 0.767ns 2.97ns 0.0314 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.33μs 1.7ns 6.58ns 0.0282 0 0 2.2 KB
master EnrichedLog net472 4.75μs 1.28ns 4.79ns 0.319 0 0 2.02 KB
#6772 EnrichedLog net6.0 3.19μs 2.46ns 9.51ns 0.0303 0 0 2.2 KB
#6772 EnrichedLog netcoreapp3.1 4.23μs 1.65ns 6.39ns 0.0296 0 0 2.2 KB
#6772 EnrichedLog net472 4.73μs 2.56ns 9.92ns 0.32 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.35μs 0.981ns 3.8ns 0.0163 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.75μs 1.07ns 4.16ns 0.0158 0 0 1.14 KB
master SendReceive net472 2.17μs 2.23ns 8.65ns 0.183 0 0 1.16 KB
#6772 SendReceive net6.0 1.45μs 0.804ns 3.01ns 0.016 0 0 1.14 KB
#6772 SendReceive netcoreapp3.1 1.81μs 0.946ns 3.67ns 0.0154 0 0 1.14 KB
#6772 SendReceive net472 2.13μs 0.762ns 2.95ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.75μs 0.676ns 2.62ns 0.022 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 4.02μs 1.56ns 6.05ns 0.0221 0 0 1.65 KB
master EnrichedLog net472 4.25μs 2.35ns 9.1ns 0.322 0 0 2.04 KB
#6772 EnrichedLog net6.0 2.85μs 1.01ns 3.79ns 0.0228 0 0 1.6 KB
#6772 EnrichedLog netcoreapp3.1 3.89μs 4.59ns 17.8ns 0.0215 0 0 1.65 KB
#6772 EnrichedLog net472 4.48μs 3.65ns 14.2ns 0.324 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6772

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.214 795.94 655.90
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.177 470.08 399.25

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 470ns 0.0811ns 0.314ns 0.008 0 0 576 B
master StartFinishSpan netcoreapp3.1 558ns 0.324ns 1.26ns 0.00777 0 0 576 B
master StartFinishSpan net472 649ns 0.193ns 0.747ns 0.0917 0 0 578 B
master StartFinishScope net6.0 487ns 0.119ns 0.446ns 0.0098 0 0 696 B
master StartFinishScope netcoreapp3.1 795ns 0.799ns 3.1ns 0.0091 0 0 696 B
master StartFinishScope net472 781ns 0.377ns 1.46ns 0.104 0 0 658 B
#6772 StartFinishSpan net6.0 399ns 0.13ns 0.504ns 0.00816 0 0 576 B
#6772 StartFinishSpan netcoreapp3.1 586ns 2.67ns 10.3ns 0.00768 0 0 576 B
#6772 StartFinishSpan net472 647ns 0.215ns 0.834ns 0.0917 0 0 578 B
#6772 StartFinishScope net6.0 475ns 0.159ns 0.616ns 0.00976 0 0 696 B
#6772 StartFinishScope netcoreapp3.1 656ns 0.319ns 1.19ns 0.00953 0 0 696 B
#6772 StartFinishScope net472 812ns 0.209ns 0.811ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 640ns 0.206ns 0.798ns 0.00961 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 899ns 0.407ns 1.41ns 0.00932 0 0 696 B
master RunOnMethodBegin net472 1.17μs 0.405ns 1.57ns 0.104 0 0 658 B
#6772 RunOnMethodBegin net6.0 642ns 0.303ns 1.18ns 0.00961 0 0 696 B
#6772 RunOnMethodBegin netcoreapp3.1 932ns 0.782ns 3.03ns 0.00946 0 0 696 B
#6772 RunOnMethodBegin net472 1.08μs 0.278ns 1.08ns 0.104 0 0 658 B

@andrewlock
Copy link
Member

Benchmarks Report for appsec 🐌

Benchmarks for #6772 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.142
  • 2 benchmarks are slower, with geometric mean 1.129
  • 2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6772

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net6.0 1.138 190,066.12 216,210.75
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net6.0 1.120 195,643.81 219,161.56

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 190μs 72.1ns 270ns 2.75 0 0 196.14 KB
master AllCycleSimpleBody netcoreapp3.1 303μs 217ns 810ns 2.69 0 0 203.7 KB
master AllCycleSimpleBody net472 264μs 71.1ns 275ns 37.2 2.11 0 234.72 KB
master AllCycleMoreComplexBody net6.0 196μs 125ns 484ns 2.84 0 0 199.64 KB
master AllCycleMoreComplexBody netcoreapp3.1 302μs 155ns 558ns 2.72 0 0 207.12 KB
master AllCycleMoreComplexBody net472 276μs 137ns 492ns 37.7 2.21 0 238.24 KB
master ObjectExtractorSimpleBody net6.0 136ns 0.0659ns 0.238ns 0.00391 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 199ns 0.2ns 0.777ns 0.00368 0 0 272 B
master ObjectExtractorSimpleBody net472 165ns 0.0976ns 0.365ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 2.84μs 2.53ns 9.78ns 0.0526 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.68μs 3.33ns 12.9ns 0.0494 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 4.32μs 2.68ns 10.4ns 0.602 0.00649 0 3.8 KB
#6772 AllCycleSimpleBody net6.0 219μs 1.17μs 8.12μs 2.76 0 0 195.99 KB
#6772 AllCycleSimpleBody netcoreapp3.1 319μs 137ns 532ns 2.71 0 0 203.56 KB
#6772 AllCycleSimpleBody net472 287μs 171ns 662ns 37.2 2.14 0 234.56 KB
#6772 AllCycleMoreComplexBody net6.0 219μs 146ns 548ns 2.74 0 0 199.5 KB
#6772 AllCycleMoreComplexBody netcoreapp3.1 319μs 74.4ns 278ns 2.71 0 0 206.98 KB
#6772 AllCycleMoreComplexBody net472 287μs 86.6ns 335ns 37.7 2.15 0 238.08 KB
#6772 ObjectExtractorSimpleBody net6.0 137ns 0.0916ns 0.343ns 0.00392 0 0 280 B
#6772 ObjectExtractorSimpleBody netcoreapp3.1 195ns 0.09ns 0.337ns 0.00365 0 0 272 B
#6772 ObjectExtractorSimpleBody net472 167ns 0.105ns 0.407ns 0.0446 0 0 281 B
#6772 ObjectExtractorMoreComplexBody net6.0 3.01μs 1.68ns 6.49ns 0.053 0 0 3.78 KB
#6772 ObjectExtractorMoreComplexBody netcoreapp3.1 3.86μs 1.83ns 7.08ns 0.0502 0 0 3.69 KB
#6772 ObjectExtractorMoreComplexBody net472 4.38μs 10.3ns 39.9ns 0.601 0.00651 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6772

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs‑net6.0 1.142 80,301.81 70,312.28

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 36.9μs 61.1ns 229ns 0.457 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 54.1μs 17.7ns 68.7ns 0.432 0 0 32.4 KB
master EncodeArgs net472 64.9μs 36.8ns 138ns 5.14 0.0646 0 32.5 KB
master EncodeLegacyArgs net6.0 79.2μs 433ns 2.63μs 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 111μs 359ns 1.39μs 0 0 0 2.14 KB
master EncodeLegacyArgs net472 151μs 81.7ns 316ns 0.304 0 0 2.15 KB
#6772 EncodeArgs net6.0 37.3μs 23.8ns 92.2ns 0.465 0 0 32.4 KB
#6772 EncodeArgs netcoreapp3.1 54μs 21.6ns 80.9ns 0.43 0 0 32.4 KB
#6772 EncodeArgs net472 65.3μs 54.9ns 205ns 5.15 0.0651 0 32.5 KB
#6772 EncodeLegacyArgs net6.0 70.5μs 97.8ns 379ns 0.0354 0 0 2.14 KB
#6772 EncodeLegacyArgs netcoreapp3.1 108μs 360ns 1.4μs 0 0 0 2.15 KB
#6772 EncodeLegacyArgs net472 150μs 58.8ns 212ns 0.3 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 173μs 77.5ns 280ns 0 0 0 2.54 KB
master RunWafRealisticBenchmark netcoreapp3.1 188μs 205ns 766ns 0 0 0 2.49 KB
master RunWafRealisticBenchmark net472 201μs 484ns 1.88μs 0.405 0 0 2.55 KB
master RunWafRealisticBenchmarkWithAttack net6.0 116μs 64.8ns 251ns 0 0 0 1.57 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 124μs 357ns 1.38μs 0 0 0 1.55 KB
master RunWafRealisticBenchmarkWithAttack net472 132μs 115ns 447ns 0.197 0 0 1.58 KB
#6772 RunWafRealisticBenchmark net6.0 174μs 124ns 448ns 0 0 0 2.54 KB
#6772 RunWafRealisticBenchmark netcoreapp3.1 186μs 351ns 1.36μs 0 0 0 2.49 KB
#6772 RunWafRealisticBenchmark net472 198μs 73.6ns 285ns 0.396 0 0 2.55 KB
#6772 RunWafRealisticBenchmarkWithAttack net6.0 116μs 96.8ns 349ns 0 0 0 1.57 KB
#6772 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 125μs 143ns 534ns 0 0 0 1.55 KB
#6772 RunWafRealisticBenchmarkWithAttack net472 132μs 14.1ns 51ns 0.198 0 0 1.58 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6772

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 252.49 KB 254.02 KB 1.54 KB 0.61%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 59.07 KB 59.42 KB 344 B 0.58%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 60.7μs 986ns 9.86μs 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 61.1μs 742ns 7.38μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 36.9μs 172ns 643ns 0 0 0 59.07 KB
master StringConcatAspectBenchmark net6.0 301μs 5.75μs 56.9μs 0 0 0 254.9 KB
master StringConcatAspectBenchmark netcoreapp3.1 338μs 1.87μs 11.4μs 0 0 0 252.49 KB
master StringConcatAspectBenchmark net472 293μs 7.07μs 69.6μs 0 0 0 278.53 KB
#6772 StringConcatBenchmark net6.0 56.2μs 605ns 5.99μs 0 0 0 43.44 KB
#6772 StringConcatBenchmark netcoreapp3.1 59.5μs 718ns 7.11μs 0 0 0 42.64 KB
#6772 StringConcatBenchmark net472 36.9μs 119ns 446ns 0 0 0 59.42 KB
#6772 StringConcatAspectBenchmark net6.0 316μs 1.77μs 11.7μs 0 0 0 255.92 KB
#6772 StringConcatAspectBenchmark netcoreapp3.1 341μs 1.88μs 11.3μs 0 0 0 254.02 KB
#6772 StringConcatAspectBenchmark net472 289μs 6.19μs 60.7μs 0 0 0 278.53 KB

@e-n-0 e-n-0 merged commit 119382a into master Mar 17, 2025
129 of 131 checks passed
@e-n-0 e-n-0 deleted the flavien/asm/fix-fllaky-grpc-iast branch March 17, 2025 12:37
@github-actions github-actions bot added this to the vNext-v3 milestone Mar 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewlock andrewlock

@NachoEchevarria NachoEchevarria

Assignees
No one assigned
Labels
area:asm area:tests unit tests, integration tests
Projects
None yet
Milestone
3.13.0
Development

Successfully merging this pull request may close these issues.
3 participants
@e-n-0 @andrewlock @NachoEchevarria