Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proactively cleanup dangling JFR repositories on startup #6441

Merged
merged 2 commits into from
Jan 15, 2024
Merged

Proactively cleanup dangling JFR repositories on startup #6441

merged 2 commits into from
Jan 15, 2024

Conversation

jbachorik
Copy link
Contributor

@jbachorik jbachorik commented Jan 4, 2024 •
edited by jira bot
Loading

What Does This Do

It runs a cleanup of JFR repository locations not belonging to any of the currently running JVMs.

Motivation

This is supposed to prevent accumulation of JFR artifacts due to unnoticed VM crashes. These can fill up the disk and render the whole host inoperable.

Additional Notes

Jira ticket: PROF-8848

@pr-commenter
Copy link

pr-commenter bot commented Jan 4, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jb/repo_cleanup
git_commit_date 1705317556 1705324986
git_commit_sha 1fae4ff 6405e01
release_version 1.28.0-SNAPSHOT~1fae4ff17e 1.28.0-SNAPSHOT~6405e013ce
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1705327635 1705327635
ci_job_id 408477928 408477928
ci_pipeline_id 26471374 26471374
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 41 metrics, 10 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:profiling:Agent.start worse
[+124.004ms; +167.350ms] or [+9.735%; +13.138%]		 1.419s 1.274s
scenario:startup:petclinic:profiling:ProfilingAgent worse
[+126.883ms; +131.437ms] or [+103.735%; +107.458%]		 251.474ms 122.314ms
scenario:startup:petclinic:profiling:Profiling worse
[+126.886ms; +131.439ms] or [+103.717%; +107.439%]		 251.501ms 122.339ms
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.28.0-SNAPSHOT~6405e013ce, baseline=1.28.0-SNAPSHOT~1fae4ff17e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1060488
Total [baseline] (8.735 s) : 0, 8734520
Agent [candidate] (1.056 s) : 0, 1055705
Total [candidate] (8.743 s) : 0, 8742862
section iast
Agent [baseline] (1.182 s) : 0, 1181887
Total [baseline] (9.294 s) : 0, 9294364
Agent [candidate] (1.205 s) : 0, 1204660
Total [candidate] (9.347 s) : 0, 9346561
section iast_TELEMETRY_OFF
Agent [baseline] (1.185 s) : 0, 1184919
Total [baseline] (9.286 s) : 0, 9286496
Agent [candidate] (1.17 s) : 0, 1170463
Total [candidate] (9.273 s) : 0, 9272880
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent iast 1.182 s 121.4 ms (11.4%)
Agent iast_TELEMETRY_OFF 1.185 s 124.431 ms (11.7%)
Total tracing 8.735 s -
Total iast 9.294 s 559.844 ms (6.4%)
Total iast_TELEMETRY_OFF 9.286 s 551.975 ms (6.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.056 s -
Agent iast 1.205 s 148.955 ms (14.1%)
Agent iast_TELEMETRY_OFF 1.17 s 114.758 ms (10.9%)
Total tracing 8.743 s -
Total iast 9.347 s 603.699 ms (6.9%)
Total iast_TELEMETRY_OFF 9.273 s 530.018 ms (6.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.28.0-SNAPSHOT~6405e013ce, baseline=1.28.0-SNAPSHOT~1fae4ff17e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (669.969 ms) : 0, 669969
BytebuddyAgent [candidate] (666.598 ms) : 0, 666598
GlobalTracer [baseline] (297.059 ms) : 0, 297059
GlobalTracer [candidate] (296.183 ms) : 0, 296183
AppSec [baseline] (50.973 ms) : 0, 50973
AppSec [candidate] (50.813 ms) : 0, 50813
Remote Config [baseline] (670.084 µs) : 0, 670
Remote Config [candidate] (657.153 µs) : 0, 657
Telemetry [baseline] (7.273 ms) : 0, 7273
Telemetry [candidate] (7.172 ms) : 0, 7172
section iast
BytebuddyAgent [baseline] (778.04 ms) : 0, 778040
BytebuddyAgent [candidate] (793.975 ms) : 0, 793975
GlobalTracer [baseline] (287.336 ms) : 0, 287336
GlobalTracer [candidate] (292.843 ms) : 0, 292843
AppSec [baseline] (53.943 ms) : 0, 53943
AppSec [candidate] (53.0 ms) : 0, 53000
IAST [baseline] (19.97 ms) : 0, 19970
IAST [candidate] (20.924 ms) : 0, 20924
Remote Config [baseline] (609.782 µs) : 0, 610
Remote Config [candidate] (607.529 µs) : 0, 608
Telemetry [baseline] (7.433 ms) : 0, 7433
Telemetry [candidate] (8.096 ms) : 0, 8096
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (778.851 ms) : 0, 778851
BytebuddyAgent [candidate] (768.998 ms) : 0, 768998
GlobalTracer [baseline] (289.67 ms) : 0, 289670
GlobalTracer [candidate] (286.597 ms) : 0, 286597
AppSec [baseline] (54.516 ms) : 0, 54516
AppSec [candidate] (55.104 ms) : 0, 55104
IAST [baseline] (20.101 ms) : 0, 20101
IAST [candidate] (18.445 ms) : 0, 18445
Remote Config [baseline] (585.41 µs) : 0, 585
Remote Config [candidate] (575.971 µs) : 0, 576
Telemetry [baseline] (6.421 ms) : 0, 6421
Telemetry [candidate] (6.388 ms) : 0, 6388
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.28.0-SNAPSHOT~6405e013ce, baseline=1.28.0-SNAPSHOT~1fae4ff17e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1053736
Total [baseline] (9.347 s) : 0, 9347276
Agent [candidate] (1.057 s) : 0, 1056776
Total [candidate] (9.355 s) : 0, 9354918
section appsec
Agent [baseline] (1.15 s) : 0, 1150381
Total [baseline] (9.474 s) : 0, 9473585
Agent [candidate] (1.155 s) : 0, 1154523
Total [candidate] (9.493 s) : 0, 9492779
section iast
Agent [baseline] (1.184 s) : 0, 1183704
Total [baseline] (9.716 s) : 0, 9715579
Agent [candidate] (1.177 s) : 0, 1176750
Total [candidate] (9.601 s) : 0, 9601370
section profiling
Agent [baseline] (1.274 s) : 0, 1273735
Total [baseline] (9.643 s) : 0, 9643245
Agent [candidate] (1.419 s) : 0, 1419412
Total [candidate] (9.738 s) : 0, 9737755
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent appsec 1.15 s 96.645 ms (9.2%)
Agent iast 1.184 s 129.968 ms (12.3%)
Agent profiling 1.274 s 219.999 ms (20.9%)
Total tracing 9.347 s -
Total appsec 9.474 s 126.309 ms (1.4%)
Total iast 9.716 s 368.303 ms (3.9%)
Total profiling 9.643 s 295.969 ms (3.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.155 s 97.747 ms (9.2%)
Agent iast 1.177 s 119.974 ms (11.4%)
Agent profiling 1.419 s 362.636 ms (34.3%)
Total tracing 9.355 s -
Total appsec 9.493 s 137.861 ms (1.5%)
Total iast 9.601 s 246.452 ms (2.6%)
Total profiling 9.738 s 382.837 ms (4.1%) 
gantt
    title petclinic - break down per module: candidate=1.28.0-SNAPSHOT~6405e013ce, baseline=1.28.0-SNAPSHOT~1fae4ff17e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (666.681 ms) : 0, 666681
BytebuddyAgent [candidate] (667.215 ms) : 0, 667215
GlobalTracer [baseline] (294.282 ms) : 0, 294282
GlobalTracer [candidate] (296.434 ms) : 0, 296434
AppSec [baseline] (50.491 ms) : 0, 50491
AppSec [candidate] (50.807 ms) : 0, 50807
Remote Config [baseline] (657.704 µs) : 0, 658
Remote Config [candidate] (657.452 µs) : 0, 657
Telemetry [baseline] (7.214 ms) : 0, 7214
Telemetry [candidate] (7.214 ms) : 0, 7214
section appsec
BytebuddyAgent [baseline] (665.115 ms) : 0, 665115
BytebuddyAgent [candidate] (667.539 ms) : 0, 667539
GlobalTracer [baseline] (295.336 ms) : 0, 295336
GlobalTracer [candidate] (296.291 ms) : 0, 296291
AppSec [baseline] (148.225 ms) : 0, 148225
AppSec [candidate] (148.779 ms) : 0, 148779
Remote Config [baseline] (639.208 µs) : 0, 639
Remote Config [candidate] (645.592 µs) : 0, 646
Telemetry [baseline] (6.875 ms) : 0, 6875
Telemetry [candidate] (6.879 ms) : 0, 6879
section iast
BytebuddyAgent [baseline] (781.655 ms) : 0, 781655
BytebuddyAgent [candidate] (775.128 ms) : 0, 775128
GlobalTracer [baseline] (285.889 ms) : 0, 285889
GlobalTracer [candidate] (286.34 ms) : 0, 286340
AppSec [baseline] (52.94 ms) : 0, 52940
AppSec [candidate] (54.931 ms) : 0, 54931
Remote Config [baseline] (603.821 µs) : 0, 604
Remote Config [candidate] (582.666 µs) : 0, 583
Telemetry [baseline] (8.14 ms) : 0, 8140
Telemetry [candidate] (7.303 ms) : 0, 7303
IAST [baseline] (19.876 ms) : 0, 19876
IAST [candidate] (18.132 ms) : 0, 18132
section profiling
ProfilingAgent [baseline] (122.314 ms) : 0, 122314
ProfilingAgent [candidate] (251.474 ms) : 0, 251474
BytebuddyAgent [baseline] (661.86 ms) : 0, 661860
BytebuddyAgent [candidate] (672.055 ms) : 0, 672055
GlobalTracer [baseline] (375.921 ms) : 0, 375921
GlobalTracer [candidate] (381.071 ms) : 0, 381071
AppSec [baseline] (51.139 ms) : 0, 51139
AppSec [candidate] (51.444 ms) : 0, 51444
Remote Config [baseline] (980.763 µs) : 0, 981
Remote Config [candidate] (994.633 µs) : 0, 995
Telemetry [baseline] (7.296 ms) : 0, 7296
Telemetry [candidate] (7.309 ms) : 0, 7309
Profiling [baseline] (122.339 ms) : 0, 122339
Profiling [candidate] (251.501 ms) : 0, 251501
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-01-15T13:46:31 2024-01-15T14:03:01
git_branch master jb/repo_cleanup
git_commit_date 1705317556 1705324986
git_commit_sha 1fae4ff 6405e01
release_version 1.28.0-SNAPSHOT~1fae4ff17e 1.28.0-SNAPSHOT~6405e013ce
start_time 2024-01-15T13:46:18 2024-01-15T14:02:48
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1705327635 1705327635
ci_job_id 408477928 408477928
ci_pipeline_id 26471374 26471374
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.28.0-SNAPSHOT~6405e013ce, baseline=1.28.0-SNAPSHOT~1fae4ff17e
    dateFormat X
    axisFormat %s
section baseline
no_agent (375.19 µs) : 355, 396
.   : milestone, 375,
iast (480.238 µs) : 459, 501
.   : milestone, 480,
iast_FULL (543.177 µs) : 523, 564
.   : milestone, 543,
iast_INACTIVE (453.158 µs) : 432, 474
.   : milestone, 453,
iast_TELEMETRY_OFF (477.956 µs) : 457, 499
.   : milestone, 478,
tracing (449.43 µs) : 428, 470
.   : milestone, 449,
section candidate
no_agent (368.542 µs) : 348, 389
.   : milestone, 369,
iast (478.833 µs) : 458, 500
.   : milestone, 479,
iast_FULL (541.941 µs) : 521, 563
.   : milestone, 542,
iast_INACTIVE (447.387 µs) : 427, 468
.   : milestone, 447,
iast_TELEMETRY_OFF (470.957 µs) : 451, 491
.   : milestone, 471,
tracing (443.625 µs) : 423, 464
.   : milestone, 444,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 375.19 µs [354.674 µs, 395.707 µs] -
iast 480.238 µs [459.489 µs, 500.986 µs] 105.047 µs (28.0%)
iast_FULL 543.177 µs [522.576 µs, 563.778 µs] 167.987 µs (44.8%)
iast_INACTIVE 453.158 µs [431.976 µs, 474.34 µs] 77.968 µs (20.8%)
iast_TELEMETRY_OFF 477.956 µs [457.06 µs, 498.852 µs] 102.766 µs (27.4%)
tracing 449.43 µs [428.469 µs, 470.391 µs] 74.24 µs (19.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 368.542 µs [348.013 µs, 389.07 µs] -
iast 478.833 µs [458.162 µs, 499.505 µs] 110.292 µs (29.9%)
iast_FULL 541.941 µs [521.312 µs, 562.57 µs] 173.399 µs (47.1%)
iast_INACTIVE 447.387 µs [426.562 µs, 468.211 µs] 78.845 µs (21.4%)
iast_TELEMETRY_OFF 470.957 µs [450.783 µs, 491.13 µs] 102.415 µs (27.8%)
tracing 443.625 µs [423.21 µs, 464.04 µs] 75.083 µs (20.4%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.28.0-SNAPSHOT~6405e013ce, baseline=1.28.0-SNAPSHOT~1fae4ff17e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.349 ms) : 1329, 1368
.   : milestone, 1349,
appsec (1.764 ms) : 1739, 1789
.   : milestone, 1764,
iast (1.526 ms) : 1501, 1550
.   : milestone, 1526,
profiling (1.524 ms) : 1498, 1549
.   : milestone, 1524,
tracing (1.495 ms) : 1470, 1520
.   : milestone, 1495,
section candidate
no_agent (1.356 ms) : 1337, 1375
.   : milestone, 1356,
appsec (1.783 ms) : 1757, 1808
.   : milestone, 1783,
iast (1.516 ms) : 1491, 1540
.   : milestone, 1516,
profiling (1.534 ms) : 1509, 1560
.   : milestone, 1534,
tracing (1.486 ms) : 1461, 1510
.   : milestone, 1486,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.349 ms [1.329 ms, 1.368 ms] -
appsec 1.764 ms [1.739 ms, 1.789 ms] 415.651 µs (30.8%)
iast 1.526 ms [1.501 ms, 1.55 ms] 177.27 µs (13.1%)
profiling 1.524 ms [1.498 ms, 1.549 ms] 175.244 µs (13.0%)
tracing 1.495 ms [1.47 ms, 1.52 ms] 146.41 µs (10.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.356 ms [1.337 ms, 1.375 ms] -
appsec 1.783 ms [1.757 ms, 1.808 ms] 426.959 µs (31.5%)
iast 1.516 ms [1.491 ms, 1.54 ms] 160.075 µs (11.8%)
profiling 1.534 ms [1.509 ms, 1.56 ms] 178.46 µs (13.2%)
tracing 1.486 ms [1.461 ms, 1.51 ms] 129.982 µs (9.6%)

@jbachorik jbachorik force-pushed the jb/repo_cleanup branch 4 times, most recently from 9467bf2 to 9e348dc Compare January 4, 2024 12:57
@jbachorik jbachorik marked this pull request as ready for review January 4, 2024 14:17
@jbachorik jbachorik requested review from a team as code owners January 4, 2024 14:17
@jbachorik jbachorik enabled auto-merge (squash) January 5, 2024 16:26
@jbachorik jbachorik merged commit 479f313 into master Jan 15, 2024
73 checks passed
@jbachorik jbachorik deleted the jb/repo_cleanup branch January 15, 2024 16:36
@github-actions github-actions bot added this to the 1.28.0 milestone Jan 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@am312 am312 am312 left review comments

@richardstartin richardstartin richardstartin approved these changes

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer PerfectSlayer is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: profiling Profiling type: enhancement
Projects
None yet
Milestone
1.28.0
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@jbachorik @am312 @richardstartin @PerfectSlayer