Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flush pending traces synchronously before terminating child process #6866

Merged
merged 1 commit into from Apr 2, 2024
Merged

Flush pending traces synchronously before terminating child process #6866

merged 1 commit into from Apr 2, 2024

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Apr 2, 2024 •
edited by jira bot

What Does This Do

Adds a synchronous flush of pending traces to TestFrameworkSession#end invocation.
TestFrameworkSession is used by CI Vis children processes (JVMs that a build system forks for tests execution).
The end method is invoked by a shutdown hook when a child process terminates.

Motivation

As soon as all the tests have executed, the build system can terminate the child process that it previously forked to run the tests.
We need to ensure that all the pending traces have been flushed before the process shuts down, otherwise test events may be lost.

Jira ticket: CIVIS-9567

@nikita-tkachenko-datadog nikita-tkachenko-datadog added type: bug comp: ci visibility Continuous Integration Visibility labels Apr 2, 2024
@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review April 2, 2024 11:58
@nikita-tkachenko-datadog nikita-tkachenko-datadog requested a review from a team as a code owner April 2, 2024 11:58
@pr-commenter
Copy link

pr-commenter bot commented Apr 2, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/flush-tests-in-child-process
git_commit_date 1712016923 1712058298
git_commit_sha ab1bf09 cab3068
release_version 1.32.0-SNAPSHOT~ab1bf09ac8 1.32.0-SNAPSHOT~cab30685b7
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1712061697 1712061697
ci_job_id 475174491 475174491
ci_pipeline_id 31250005 31250005
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 11 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~cab30685b7, baseline=1.32.0-SNAPSHOT~ab1bf09ac8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.075 s) : 0, 1075112
Total [baseline] (8.559 s) : 0, 8558589
Agent [candidate] (1.077 s) : 0, 1076794
Total [candidate] (8.577 s) : 0, 8576709
section iast
Agent [baseline] (1.206 s) : 0, 1205844
Total [baseline] (9.098 s) : 0, 9097535
Agent [candidate] (1.2 s) : 0, 1199514
Total [candidate] (9.1 s) : 0, 9100312
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.208 s) : 0, 1208178
Total [baseline] (9.099 s) : 0, 9098504
Agent [candidate] (1.201 s) : 0, 1200532
Total [candidate] (9.053 s) : 0, 9053166
section iast_TELEMETRY_OFF
Agent [baseline] (1.197 s) : 0, 1196706
Total [baseline] (9.098 s) : 0, 9098410
Agent [candidate] (1.206 s) : 0, 1206114
Total [candidate] (9.115 s) : 0, 9115462
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.075 s -
Agent iast 1.206 s 130.732 ms (12.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.208 s 133.065 ms (12.4%)
Agent iast_TELEMETRY_OFF 1.197 s 121.594 ms (11.3%)
Total tracing 8.559 s -
Total iast 9.098 s 538.946 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.099 s 539.916 ms (6.3%)
Total iast_TELEMETRY_OFF 9.098 s 539.821 ms (6.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent iast 1.2 s 122.72 ms (11.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.201 s 123.739 ms (11.5%)
Agent iast_TELEMETRY_OFF 1.206 s 129.32 ms (12.0%)
Total tracing 8.577 s -
Total iast 9.1 s 523.603 ms (6.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.053 s 476.457 ms (5.6%)
Total iast_TELEMETRY_OFF 9.115 s 538.753 ms (6.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~cab30685b7, baseline=1.32.0-SNAPSHOT~ab1bf09ac8

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (673.453 ms) : 0, 673453
BytebuddyAgent [candidate] (673.96 ms) : 0, 673960
GlobalTracer [baseline] (309.623 ms) : 0, 309623
GlobalTracer [candidate] (310.585 ms) : 0, 310585
AppSec [baseline] (49.49 ms) : 0, 49490
AppSec [candidate] (49.559 ms) : 0, 49559
Remote Config [baseline] (661.178 µs) : 0, 661
Remote Config [candidate] (670.93 µs) : 0, 671
Telemetry [baseline] (7.546 ms) : 0, 7546
Telemetry [candidate] (7.694 ms) : 0, 7694
section iast
BytebuddyAgent [baseline] (798.767 ms) : 0, 798767
BytebuddyAgent [candidate] (794.655 ms) : 0, 794655
GlobalTracer [baseline] (290.286 ms) : 0, 290286
GlobalTracer [candidate] (288.786 ms) : 0, 288786
AppSec [baseline] (50.695 ms) : 0, 50695
AppSec [candidate] (50.258 ms) : 0, 50258
IAST [baseline] (24.397 ms) : 0, 24397
IAST [candidate] (22.821 ms) : 0, 22821
Remote Config [baseline] (586.498 µs) : 0, 586
Remote Config [candidate] (578.961 µs) : 0, 579
Telemetry [baseline] (6.58 ms) : 0, 6580
Telemetry [candidate] (8.126 ms) : 0, 8126
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.381 ms) : 0, 800381
BytebuddyAgent [candidate] (794.469 ms) : 0, 794469
GlobalTracer [baseline] (290.555 ms) : 0, 290555
GlobalTracer [candidate] (289.983 ms) : 0, 289983
AppSec [baseline] (50.001 ms) : 0, 50001
AppSec [candidate] (48.941 ms) : 0, 48941
IAST [baseline] (25.509 ms) : 0, 25509
IAST [candidate] (23.921 ms) : 0, 23921
Remote Config [baseline] (572.708 µs) : 0, 573
Remote Config [candidate] (584.33 µs) : 0, 584
Telemetry [baseline] (6.572 ms) : 0, 6572
Telemetry [candidate] (8.16 ms) : 0, 8160
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (791.629 ms) : 0, 791629
BytebuddyAgent [candidate] (798.723 ms) : 0, 798723
GlobalTracer [baseline] (288.771 ms) : 0, 288771
GlobalTracer [candidate] (291.251 ms) : 0, 291251
AppSec [baseline] (49.894 ms) : 0, 49894
AppSec [candidate] (49.534 ms) : 0, 49534
IAST [baseline] (24.816 ms) : 0, 24816
IAST [candidate] (23.32 ms) : 0, 23320
Remote Config [baseline] (576.18 µs) : 0, 576
Remote Config [candidate] (582.378 µs) : 0, 582
Telemetry [baseline] (6.599 ms) : 0, 6599
Telemetry [candidate] (8.093 ms) : 0, 8093
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~cab30685b7, baseline=1.32.0-SNAPSHOT~ab1bf09ac8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.092 s) : 0, 1091766
Total [baseline] (10.476 s) : 0, 10476121
Agent [candidate] (1.083 s) : 0, 1083482
Total [candidate] (10.431 s) : 0, 10430846
section appsec
Agent [baseline] (1.197 s) : 0, 1196832
Total [baseline] (10.521 s) : 0, 10521360
Agent [candidate] (1.197 s) : 0, 1196576
Total [candidate] (10.502 s) : 0, 10502149
section iast
Agent [baseline] (1.2 s) : 0, 1199739
Total [baseline] (10.813 s) : 0, 10813059
Agent [candidate] (1.201 s) : 0, 1200573
Total [candidate] (10.761 s) : 0, 10761289
section profiling
Agent [baseline] (1.267 s) : 0, 1267457
Total [baseline] (10.635 s) : 0, 10634525
Agent [candidate] (1.269 s) : 0, 1269476
Total [candidate] (10.626 s) : 0, 10625603
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.092 s -
Agent appsec 1.197 s 105.066 ms (9.6%)
Agent iast 1.2 s 107.972 ms (9.9%)
Agent profiling 1.267 s 175.691 ms (16.1%)
Total tracing 10.476 s -
Total appsec 10.521 s 45.239 ms (0.4%)
Total iast 10.813 s 336.938 ms (3.2%)
Total profiling 10.635 s 158.403 ms (1.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent appsec 1.197 s 113.093 ms (10.4%)
Agent iast 1.201 s 117.091 ms (10.8%)
Agent profiling 1.269 s 185.993 ms (17.2%)
Total tracing 10.431 s -
Total appsec 10.502 s 71.303 ms (0.7%)
Total iast 10.761 s 330.443 ms (3.2%)
Total profiling 10.626 s 194.757 ms (1.9%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~cab30685b7, baseline=1.32.0-SNAPSHOT~ab1bf09ac8

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (683.519 ms) : 0, 683519
BytebuddyAgent [candidate] (678.697 ms) : 0, 678697
GlobalTracer [baseline] (314.503 ms) : 0, 314503
GlobalTracer [candidate] (312.057 ms) : 0, 312057
AppSec [baseline] (50.366 ms) : 0, 50366
AppSec [candidate] (49.768 ms) : 0, 49768
Remote Config [baseline] (680.971 µs) : 0, 681
Remote Config [candidate] (664.166 µs) : 0, 664
Telemetry [baseline] (7.762 ms) : 0, 7762
Telemetry [candidate] (7.663 ms) : 0, 7663
section appsec
BytebuddyAgent [baseline] (694.7 ms) : 0, 694700
BytebuddyAgent [candidate] (694.775 ms) : 0, 694775
GlobalTracer [baseline] (291.456 ms) : 0, 291456
GlobalTracer [candidate] (292.026 ms) : 0, 292026
AppSec [baseline] (150.152 ms) : 0, 150152
AppSec [candidate] (149.225 ms) : 0, 149225
IAST [baseline] (18.705 ms) : 0, 18705
IAST [candidate] (18.809 ms) : 0, 18809
Remote Config [baseline] (605.575 µs) : 0, 606
Remote Config [candidate] (605.85 µs) : 0, 606
Telemetry [baseline] (6.766 ms) : 0, 6766
Telemetry [candidate] (6.794 ms) : 0, 6794
section iast
BytebuddyAgent [baseline] (794.93 ms) : 0, 794930
BytebuddyAgent [candidate] (795.492 ms) : 0, 795492
GlobalTracer [baseline] (288.535 ms) : 0, 288535
GlobalTracer [candidate] (289.429 ms) : 0, 289429
AppSec [baseline] (51.918 ms) : 0, 51918
AppSec [candidate] (49.617 ms) : 0, 49617
IAST [baseline] (22.912 ms) : 0, 22912
IAST [candidate] (24.472 ms) : 0, 24472
Remote Config [baseline] (575.901 µs) : 0, 576
Remote Config [candidate] (576.698 µs) : 0, 577
Telemetry [baseline] (6.579 ms) : 0, 6579
Telemetry [candidate] (6.58 ms) : 0, 6580
section profiling
BytebuddyAgent [baseline] (677.049 ms) : 0, 677049
BytebuddyAgent [candidate] (677.257 ms) : 0, 677257
GlobalTracer [baseline] (380.252 ms) : 0, 380252
GlobalTracer [candidate] (381.557 ms) : 0, 381557
AppSec [baseline] (50.366 ms) : 0, 50366
AppSec [candidate] (50.299 ms) : 0, 50299
Remote Config [baseline] (691.44 µs) : 0, 691
Remote Config [candidate] (690.245 µs) : 0, 690
Telemetry [baseline] (7.455 ms) : 0, 7455
Telemetry [candidate] (7.48 ms) : 0, 7480
ProfilingAgent [baseline] (95.277 ms) : 0, 95277
ProfilingAgent [candidate] (95.928 ms) : 0, 95928
Profiling [baseline] (95.301 ms) : 0, 95301
Profiling [candidate] (95.951 ms) : 0, 95951

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-04-02T12:13:47 2024-04-02T12:35:51
git_branch master nikita-tkachenko/flush-tests-in-child-process
git_commit_date 1712016923 1712058298
git_commit_sha ab1bf09 cab3068
release_version 1.32.0-SNAPSHOT~ab1bf09ac8 1.32.0-SNAPSHOT~cab30685b7
start_time 2024-04-02T12:13:34 2024-04-02T12:35:38
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1712061697 1712061697
ci_job_id 475174491 475174491
ci_pipeline_id 31250005 31250005
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~cab30685b7, baseline=1.32.0-SNAPSHOT~ab1bf09ac8
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.258 µs) : 351, 393
.   : milestone, 372,
iast (481.433 µs) : 461, 502
.   : milestone, 481,
iast_FULL (547.51 µs) : 527, 568
.   : milestone, 548,
iast_GLOBAL (512.038 µs) : 490, 534
.   : milestone, 512,
iast_HARDCODED_SECRET_DISABLED (481.349 µs) : 461, 502
.   : milestone, 481,
iast_INACTIVE (448.988 µs) : 428, 470
.   : milestone, 449,
iast_TELEMETRY_OFF (470.933 µs) : 450, 491
.   : milestone, 471,
tracing (452.836 µs) : 432, 474
.   : milestone, 453,
section candidate
no_agent (367.206 µs) : 347, 387
.   : milestone, 367,
iast (475.977 µs) : 455, 497
.   : milestone, 476,
iast_FULL (550.577 µs) : 530, 571
.   : milestone, 551,
iast_GLOBAL (496.382 µs) : 476, 517
.   : milestone, 496,
iast_HARDCODED_SECRET_DISABLED (479.03 µs) : 458, 500
.   : milestone, 479,
iast_INACTIVE (452.49 µs) : 431, 474
.   : milestone, 452,
iast_TELEMETRY_OFF (479.46 µs) : 458, 501
.   : milestone, 479,
tracing (443.705 µs) : 423, 464
.   : milestone, 444,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.258 µs [351.336 µs, 393.18 µs] -
iast 481.433 µs [460.809 µs, 502.057 µs] 109.175 µs (29.3%)
iast_FULL 547.51 µs [526.749 µs, 568.271 µs] 175.252 µs (47.1%)
iast_GLOBAL 512.038 µs [490.297 µs, 533.78 µs] 139.781 µs (37.5%)
iast_HARDCODED_SECRET_DISABLED 481.349 µs [460.629 µs, 502.068 µs] 109.091 µs (29.3%)
iast_INACTIVE 448.988 µs [428.407 µs, 469.569 µs] 76.731 µs (20.6%)
iast_TELEMETRY_OFF 470.933 µs [450.37 µs, 491.497 µs] 98.676 µs (26.5%)
tracing 452.836 µs [431.966 µs, 473.706 µs] 80.578 µs (21.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.206 µs [347.048 µs, 387.365 µs] -
iast 475.977 µs [455.377 µs, 496.577 µs] 108.771 µs (29.6%)
iast_FULL 550.577 µs [529.917 µs, 571.236 µs] 183.37 µs (49.9%)
iast_GLOBAL 496.382 µs [476.14 µs, 516.625 µs] 129.176 µs (35.2%)
iast_HARDCODED_SECRET_DISABLED 479.03 µs [458.296 µs, 499.764 µs] 111.823 µs (30.5%)
iast_INACTIVE 452.49 µs [431.278 µs, 473.702 µs] 85.284 µs (23.2%)
iast_TELEMETRY_OFF 479.46 µs [457.959 µs, 500.96 µs] 112.253 µs (30.6%)
tracing 443.705 µs [423.077 µs, 464.333 µs] 76.499 µs (20.8%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~cab30685b7, baseline=1.32.0-SNAPSHOT~ab1bf09ac8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.349 ms) : 1330, 1369
.   : milestone, 1349,
appsec (1.723 ms) : 1698, 1747
.   : milestone, 1723,
appsec_no_iast (1.736 ms) : 1712, 1759
.   : milestone, 1736,
iast (1.498 ms) : 1475, 1521
.   : milestone, 1498,
profiling (1.496 ms) : 1472, 1520
.   : milestone, 1496,
tracing (1.487 ms) : 1462, 1512
.   : milestone, 1487,
section candidate
no_agent (1.353 ms) : 1333, 1373
.   : milestone, 1353,
appsec (1.734 ms) : 1710, 1758
.   : milestone, 1734,
appsec_no_iast (1.727 ms) : 1704, 1751
.   : milestone, 1727,
iast (1.506 ms) : 1483, 1529
.   : milestone, 1506,
profiling (1.516 ms) : 1491, 1541
.   : milestone, 1516,
tracing (1.473 ms) : 1449, 1498
.   : milestone, 1473,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.349 ms [1.33 ms, 1.369 ms] -
appsec 1.723 ms [1.698 ms, 1.747 ms] 373.353 µs (27.7%)
appsec_no_iast 1.736 ms [1.712 ms, 1.759 ms] 386.216 µs (28.6%)
iast 1.498 ms [1.475 ms, 1.521 ms] 148.548 µs (11.0%)
profiling 1.496 ms [1.472 ms, 1.52 ms] 146.461 µs (10.9%)
tracing 1.487 ms [1.462 ms, 1.512 ms] 137.772 µs (10.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.333 ms, 1.373 ms] -
appsec 1.734 ms [1.71 ms, 1.758 ms] 380.924 µs (28.2%)
appsec_no_iast 1.727 ms [1.704 ms, 1.751 ms] 374.06 µs (27.6%)
iast 1.506 ms [1.483 ms, 1.529 ms] 152.584 µs (11.3%)
profiling 1.516 ms [1.491 ms, 1.541 ms] 162.468 µs (12.0%)
tracing 1.473 ms [1.449 ms, 1.498 ms] 120.11 µs (8.9%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 329acac into master Apr 2, 2024
79 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/flush-tests-in-child-process branch April 2, 2024 13:31
@github-actions github-actions bot added this to the 1.32.0 milestone Apr 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anmarchenko anmarchenko anmarchenko approved these changes

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility type: bug
Projects
None yet
Milestone
1.32.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@nikita-tkachenko-datadog @anmarchenko