Add caching to JVM info factory #6162

nikita-tkachenko-datadog · 2023-11-07T12:11:56Z

What Does This Do

Introduces a caching decorator for JvmInfoFactory class that retrieves JVM info provided path to the JVM's executable.

Motivation

Forking the JVM is needed to obtain its info, which can be slow, and because of some recent changes in CI Visibility the call to JvmInfoFactory can be made several times per test session.

Jira ticket: CIVIS-7930

pr-commenter · 2023-11-07T12:58:51Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.23.0-SNAPSHOT~2e6162bf24	1.23.0-SNAPSHOT~6523c67598
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.23.0-SNAPSHOT~6523c67598, baseline=1.23.0-SNAPSHOT~2e6162bf24

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.042 s) : 0, 1041870
Total [baseline] (8.798 s) : 0, 8798439
Agent [candidate] (1.037 s) : 0, 1037250
Total [candidate] (8.794 s) : 0, 8793835
section iast
Agent [baseline] (1.153 s) : 0, 1153372
Total [baseline] (9.343 s) : 0, 9343163
Agent [candidate] (1.155 s) : 0, 1155500
Total [candidate] (9.311 s) : 0, 9310700
section iast_TELEMETRY_OFF
Agent [baseline] (1.145 s) : 0, 1144745
Total [baseline] (9.288 s) : 0, 9287778
Agent [candidate] (1.155 s) : 0, 1154535
Total [candidate] (9.331 s) : 0, 9331499

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.042 s	-
Agent	iast	1.153 s	111.501 ms (10.7%)
Agent	iast_TELEMETRY_OFF	1.145 s	102.875 ms (9.9%)
Total	tracing	8.798 s	-
Total	iast	9.343 s	544.724 ms (6.2%)
Total	iast_TELEMETRY_OFF	9.288 s	489.339 ms (5.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.037 s	-
Agent	iast	1.155 s	118.249 ms (11.4%)
Agent	iast_TELEMETRY_OFF	1.155 s	117.285 ms (11.3%)
Total	tracing	8.794 s	-
Total	iast	9.311 s	516.864 ms (5.9%)
Total	iast_TELEMETRY_OFF	9.331 s	537.664 ms (6.1%)

gantt
    title insecure-bank - break down per module: candidate=1.23.0-SNAPSHOT~6523c67598, baseline=1.23.0-SNAPSHOT~2e6162bf24

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (650.25 ms) : 0, 650250
BytebuddyAgent [candidate] (646.817 ms) : 0, 646817
GlobalTracer [baseline] (295.371 ms) : 0, 295371
GlobalTracer [candidate] (295.04 ms) : 0, 295040
AppSec [baseline] (49.416 ms) : 0, 49416
AppSec [candidate] (48.9 ms) : 0, 48900
Remote Config [baseline] (704.635 µs) : 0, 705
Remote Config [candidate] (696.205 µs) : 0, 696
Telemetry [baseline] (11.418 ms) : 0, 11418
Telemetry [candidate] (11.262 ms) : 0, 11262
section iast
BytebuddyAgent [baseline] (766.43 ms) : 0, 766430
BytebuddyAgent [candidate] (767.381 ms) : 0, 767381
GlobalTracer [baseline] (274.212 ms) : 0, 274212
GlobalTracer [candidate] (275.528 ms) : 0, 275528
AppSec [baseline] (46.805 ms) : 0, 46805
AppSec [candidate] (46.808 ms) : 0, 46808
Remote Config [baseline] (784.185 µs) : 0, 784
Remote Config [candidate] (570.289 µs) : 0, 570
Telemetry [baseline] (13.174 ms) : 0, 13174
Telemetry [candidate] (12.523 ms) : 0, 12523
IAST [baseline] (17.512 ms) : 0, 17512
IAST [candidate] (18.134 ms) : 0, 18134
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (760.36 ms) : 0, 760360
BytebuddyAgent [candidate] (765.758 ms) : 0, 765758
GlobalTracer [baseline] (274.533 ms) : 0, 274533
GlobalTracer [candidate] (276.618 ms) : 0, 276618
AppSec [baseline] (46.612 ms) : 0, 46612
AppSec [candidate] (46.556 ms) : 0, 46556
Remote Config [baseline] (566.361 µs) : 0, 566
Remote Config [candidate] (582.068 µs) : 0, 582
Telemetry [baseline] (11.923 ms) : 0, 11923
Telemetry [candidate] (12.699 ms) : 0, 12699
IAST [baseline] (16.171 ms) : 0, 16171
IAST [candidate] (17.627 ms) : 0, 17627

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.23.0-SNAPSHOT~6523c67598, baseline=1.23.0-SNAPSHOT~2e6162bf24

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.036 s) : 0, 1036171
Total [baseline] (9.278 s) : 0, 9278376
Agent [candidate] (1.037 s) : 0, 1037296
Total [candidate] (9.314 s) : 0, 9314444
section appsec
Agent [baseline] (1.121 s) : 0, 1121024
Total [baseline] (9.449 s) : 0, 9449080
Agent [candidate] (1.13 s) : 0, 1130040
Total [candidate] (9.413 s) : 0, 9413228
section iast
Agent [baseline] (1.149 s) : 0, 1148900
Total [baseline] (9.496 s) : 0, 9496403
Agent [candidate] (1.15 s) : 0, 1149501
Total [candidate] (9.446 s) : 0, 9445794
section profiling
Agent [baseline] (1.22 s) : 0, 1220496
Total [baseline] (9.499 s) : 0, 9499369
Agent [candidate] (1.221 s) : 0, 1220587
Total [candidate] (9.566 s) : 0, 9565942

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.036 s	-
Agent	appsec	1.121 s	84.853 ms (8.2%)
Agent	iast	1.149 s	112.729 ms (10.9%)
Agent	profiling	1.22 s	184.325 ms (17.8%)
Total	tracing	9.278 s	-
Total	appsec	9.449 s	170.703 ms (1.8%)
Total	iast	9.496 s	218.027 ms (2.3%)
Total	profiling	9.499 s	220.992 ms (2.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.037 s	-
Agent	appsec	1.13 s	92.744 ms (8.9%)
Agent	iast	1.15 s	112.206 ms (10.8%)
Agent	profiling	1.221 s	183.292 ms (17.7%)
Total	tracing	9.314 s	-
Total	appsec	9.413 s	98.784 ms (1.1%)
Total	iast	9.446 s	131.35 ms (1.4%)
Total	profiling	9.566 s	251.498 ms (2.7%)

gantt
    title petclinic - break down per module: candidate=1.23.0-SNAPSHOT~6523c67598, baseline=1.23.0-SNAPSHOT~2e6162bf24

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (646.414 ms) : 0, 646414
BytebuddyAgent [candidate] (647.215 ms) : 0, 647215
GlobalTracer [baseline] (294.261 ms) : 0, 294261
GlobalTracer [candidate] (294.62 ms) : 0, 294620
AppSec [baseline] (48.981 ms) : 0, 48981
AppSec [candidate] (49.022 ms) : 0, 49022
Remote Config [baseline] (708.061 µs) : 0, 708
Remote Config [candidate] (697.111 µs) : 0, 697
Telemetry [baseline] (11.301 ms) : 0, 11301
Telemetry [candidate] (11.308 ms) : 0, 11308
section appsec
BytebuddyAgent [baseline] (646.494 ms) : 0, 646494
BytebuddyAgent [candidate] (651.561 ms) : 0, 651561
GlobalTracer [baseline] (294.509 ms) : 0, 294509
GlobalTracer [candidate] (297.115 ms) : 0, 297115
AppSec [baseline] (138.039 ms) : 0, 138039
AppSec [candidate] (139.041 ms) : 0, 139041
Remote Config [baseline] (646.751 µs) : 0, 647
Remote Config [candidate] (659.393 µs) : 0, 659
Telemetry [baseline] (6.898 ms) : 0, 6898
Telemetry [candidate] (6.928 ms) : 0, 6928
section iast
BytebuddyAgent [baseline] (764.372 ms) : 0, 764372
BytebuddyAgent [candidate] (764.096 ms) : 0, 764096
GlobalTracer [baseline] (274.189 ms) : 0, 274189
GlobalTracer [candidate] (273.761 ms) : 0, 273761
AppSec [baseline] (46.555 ms) : 0, 46555
AppSec [candidate] (47.512 ms) : 0, 47512
Remote Config [baseline] (573.511 µs) : 0, 574
Remote Config [candidate] (571.399 µs) : 0, 571
Telemetry [baseline] (11.218 ms) : 0, 11218
Telemetry [candidate] (11.832 ms) : 0, 11832
IAST [baseline] (17.601 ms) : 0, 17601
IAST [candidate] (17.407 ms) : 0, 17407
section profiling
BytebuddyAgent [baseline] (656.591 ms) : 0, 656591
BytebuddyAgent [candidate] (657.455 ms) : 0, 657455
GlobalTracer [baseline] (359.06 ms) : 0, 359060
GlobalTracer [candidate] (358.774 ms) : 0, 358774
AppSec [baseline] (49.782 ms) : 0, 49782
AppSec [candidate] (49.19 ms) : 0, 49190
Remote Config [baseline] (660.658 µs) : 0, 661
Remote Config [candidate] (654.156 µs) : 0, 654
Telemetry [baseline] (11.339 ms) : 0, 11339
Telemetry [candidate] (11.344 ms) : 0, 11344
ProfilingAgent [baseline] (88.782 ms) : 0, 88782
ProfilingAgent [candidate] (88.855 ms) : 0, 88855
Profiling [baseline] (88.806 ms) : 0, 88806
Profiling [candidate] (88.879 ms) : 0, 88879

Load

Parameters

	Baseline	Candidate
commit	1.23.0-SNAPSHOT~2e6162bf24	1.23.0-SNAPSHOT~6523c67598
config	baseline	candidate
end_time	2023-11-09T12:24:59	2023-11-09T12:41:25
start_time	2023-11-09T12:24:46	2023-11-09T12:41:13

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~6523c67598, baseline=1.23.0-SNAPSHOT~2e6162bf24
    dateFormat X
    axisFormat %s
section baseline
no_agent (364.594 µs) : 343, 386
.   : milestone, 365,
iast (458.04 µs) : 437, 479
.   : milestone, 458,
iast_FULL (518.69 µs) : 498, 539
.   : milestone, 519,
iast_INACTIVE (431.556 µs) : 411, 452
.   : milestone, 432,
iast_TELEMETRY_OFF (457.016 µs) : 436, 478
.   : milestone, 457,
tracing (440.609 µs) : 419, 462
.   : milestone, 441,
section candidate
no_agent (360.74 µs) : 341, 381
.   : milestone, 361,
iast (457.25 µs) : 437, 478
.   : milestone, 457,
iast_FULL (521.698 µs) : 501, 542
.   : milestone, 522,
iast_INACTIVE (435.916 µs) : 415, 457
.   : milestone, 436,
iast_TELEMETRY_OFF (455.531 µs) : 434, 477
.   : milestone, 456,
tracing (433.19 µs) : 412, 455
.   : milestone, 433,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	364.594 µs [342.752 µs, 386.437 µs]	-
iast	458.04 µs [437.276 µs, 478.804 µs]	93.446 µs (25.6%)
iast_FULL	518.69 µs [497.954 µs, 539.426 µs]	154.096 µs (42.3%)
iast_INACTIVE	431.556 µs [410.701 µs, 452.41 µs]	66.962 µs (18.4%)
iast_TELEMETRY_OFF	457.016 µs [436.173 µs, 477.859 µs]	92.422 µs (25.3%)
tracing	440.609 µs [418.927 µs, 462.29 µs]	76.015 µs (20.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	360.74 µs [340.716 µs, 380.764 µs]	-
iast	457.25 µs [436.529 µs, 477.971 µs]	96.51 µs (26.8%)
iast_FULL	521.698 µs [501.05 µs, 542.346 µs]	160.959 µs (44.6%)
iast_INACTIVE	435.916 µs [414.699 µs, 457.132 µs]	75.176 µs (20.8%)
iast_TELEMETRY_OFF	455.531 µs [434.318 µs, 476.744 µs]	94.792 µs (26.3%)
tracing	433.19 µs [411.758 µs, 454.622 µs]	72.45 µs (20.1%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~6523c67598, baseline=1.23.0-SNAPSHOT~2e6162bf24
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.358 ms) : 1339, 1377
.   : milestone, 1358,
appsec (1.71 ms) : 1685, 1735
.   : milestone, 1710,
iast (1.493 ms) : 1468, 1518
.   : milestone, 1493,
profiling (1.475 ms) : 1450, 1500
.   : milestone, 1475,
tracing (1.436 ms) : 1410, 1461
.   : milestone, 1436,
section candidate
no_agent (1.34 ms) : 1320, 1359
.   : milestone, 1340,
appsec (1.71 ms) : 1686, 1735
.   : milestone, 1710,
iast (1.473 ms) : 1449, 1496
.   : milestone, 1473,
profiling (1.485 ms) : 1458, 1511
.   : milestone, 1485,
tracing (1.462 ms) : 1438, 1486
.   : milestone, 1462,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.358 ms [1.339 ms, 1.377 ms]	-
appsec	1.71 ms [1.685 ms, 1.735 ms]	352.473 µs (26.0%)
iast	1.493 ms [1.468 ms, 1.518 ms]	134.832 µs (9.9%)
profiling	1.475 ms [1.45 ms, 1.5 ms]	117.048 µs (8.6%)
tracing	1.436 ms [1.41 ms, 1.461 ms]	77.642 µs (5.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.34 ms [1.32 ms, 1.359 ms]	-
appsec	1.71 ms [1.686 ms, 1.735 ms]	370.317 µs (27.6%)
iast	1.473 ms [1.449 ms, 1.496 ms]	132.742 µs (9.9%)
profiling	1.485 ms [1.458 ms, 1.511 ms]	144.86 µs (10.8%)
tracing	1.462 ms [1.438 ms, 1.486 ms]	122.274 µs (9.1%)

Add caching to JVM info factory

5b7c227

nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Nov 7, 2023

Add code coverage exclusions

ad8ad20

nikita-tkachenko-datadog and others added 2 commits November 7, 2023 18:16

Add code coverage exclusions

2f9d45c

Merge branch 'master' into nikita-tkachenko/jvm-info-factory-caching

cadb94a

nikita-tkachenko-datadog marked this pull request as ready for review November 8, 2023 11:18

nikita-tkachenko-datadog requested review from a team as code owners November 8, 2023 11:18

nikita-tkachenko-datadog requested review from tonyredondo, PerfectSlayer and nayeem-kamal November 8, 2023 11:18

Merge branch 'master' into nikita-tkachenko/jvm-info-factory-caching

6523c67

juan-fernandez approved these changes Nov 14, 2023

View reviewed changes

nikita-tkachenko-datadog merged commit 76a6744 into master Nov 14, 2023
69 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/jvm-info-factory-caching branch November 14, 2023 11:39

github-actions bot added this to the 1.24.0 milestone Nov 14, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add caching to JVM info factory #6162

Add caching to JVM info factory #6162

nikita-tkachenko-datadog commented Nov 7, 2023 •

edited

Loading

pr-commenter bot commented Nov 7, 2023 •

edited

Loading

Add caching to JVM info factory #6162

Add caching to JVM info factory #6162

Conversation

nikita-tkachenko-datadog commented Nov 7, 2023 • edited Loading

What Does This Do

Motivation

pr-commenter bot commented Nov 7, 2023 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

nikita-tkachenko-datadog commented Nov 7, 2023 •

edited

Loading

pr-commenter bot commented Nov 7, 2023 •

edited

Loading