Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhance 'runtime_version' tag to encode graalvm source #6788

Merged
merged 1 commit into from Mar 8, 2024
Merged

Enhance 'runtime_version' tag to encode graalvm source #6788

merged 1 commit into from Mar 8, 2024

Conversation

jbachorik
Copy link
Contributor

@jbachorik jbachorik commented Mar 7, 2024 •
edited

What Does This Do

It will suffix the 'runtime_version' tag with -graalvm if running as a native image.

Motivation

The SubstrateVM (GraalVM native image runtime) implementation of JFR does not properly trim the stacktraces, leaving several frames of runtime internals there.
We can trim those stacktraces in our backend but doing that for all incoming recordings would be too costly - therefore we need to have a quick way to identify such recordings based on the tags.

Additional Notes

Jira ticket: PROF-9285

@pr-commenter
Copy link

pr-commenter bot commented Mar 7, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jb/graalvm_version
git_commit_date 1709742193 1709832347
git_commit_sha d20df91 75e4313
release_version 1.32.0-SNAPSHOT~d20df91089 1.32.0-SNAPSHOT~75e4313559
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1709835616 1709835616
ci_job_id 454237140 454237140
ci_pipeline_id 29743111 29743111
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 13 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~75e4313559, baseline=1.32.0-SNAPSHOT~d20df91089

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.084 s) : 0, 1084455
Total [baseline] (8.599 s) : 0, 8599083
Agent [candidate] (1.092 s) : 0, 1092259
Total [candidate] (8.633 s) : 0, 8632605
section iast
Agent [baseline] (1.217 s) : 0, 1216804
Total [baseline] (9.07 s) : 0, 9070225
Agent [candidate] (1.21 s) : 0, 1209803
Total [candidate] (9.083 s) : 0, 9082837
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.208 s) : 0, 1208451
Total [baseline] (9.028 s) : 0, 9027702
Agent [candidate] (1.209 s) : 0, 1209456
Total [candidate] (9.042 s) : 0, 9042040
section iast_TELEMETRY_OFF
Agent [baseline] (1.201 s) : 0, 1200624
Total [baseline] (9.06 s) : 0, 9060271
Agent [candidate] (1.204 s) : 0, 1204189
Total [candidate] (9.051 s) : 0, 9051453
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.084 s -
Agent iast 1.217 s 132.348 ms (12.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.208 s 123.996 ms (11.4%)
Agent iast_TELEMETRY_OFF 1.201 s 116.169 ms (10.7%)
Total tracing 8.599 s -
Total iast 9.07 s 471.142 ms (5.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.028 s 428.619 ms (5.0%)
Total iast_TELEMETRY_OFF 9.06 s 461.188 ms (5.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.092 s -
Agent iast 1.21 s 117.545 ms (10.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.209 s 117.198 ms (10.7%)
Agent iast_TELEMETRY_OFF 1.204 s 111.931 ms (10.2%)
Total tracing 8.633 s -
Total iast 9.083 s 450.232 ms (5.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.042 s 409.436 ms (4.7%)
Total iast_TELEMETRY_OFF 9.051 s 418.848 ms (4.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~75e4313559, baseline=1.32.0-SNAPSHOT~d20df91089

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (698.251 ms) : 0, 698251
BytebuddyAgent [candidate] (702.718 ms) : 0, 702718
GlobalTracer [baseline] (292.331 ms) : 0, 292331
GlobalTracer [candidate] (295.311 ms) : 0, 295311
AppSec [baseline] (51.019 ms) : 0, 51019
AppSec [candidate] (51.165 ms) : 0, 51165
Remote Config [baseline] (725.613 µs) : 0, 726
Remote Config [candidate] (740.923 µs) : 0, 741
Telemetry [baseline] (7.835 ms) : 0, 7835
Telemetry [candidate] (7.691 ms) : 0, 7691
section iast
BytebuddyAgent [baseline] (807.886 ms) : 0, 807886
BytebuddyAgent [candidate] (801.878 ms) : 0, 801878
GlobalTracer [baseline] (290.485 ms) : 0, 290485
GlobalTracer [candidate] (289.566 ms) : 0, 289566
AppSec [baseline] (51.703 ms) : 0, 51703
AppSec [candidate] (53.276 ms) : 0, 53276
IAST [baseline] (24.121 ms) : 0, 24121
IAST [candidate] (22.575 ms) : 0, 22575
Remote Config [baseline] (597.173 µs) : 0, 597
Remote Config [candidate] (587.721 µs) : 0, 588
Telemetry [baseline] (7.426 ms) : 0, 7426
Telemetry [candidate] (7.498 ms) : 0, 7498
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (801.149 ms) : 0, 801149
BytebuddyAgent [candidate] (801.931 ms) : 0, 801931
GlobalTracer [baseline] (289.436 ms) : 0, 289436
GlobalTracer [candidate] (289.639 ms) : 0, 289639
AppSec [baseline] (50.567 ms) : 0, 50567
AppSec [candidate] (50.949 ms) : 0, 50949
IAST [baseline] (25.643 ms) : 0, 25643
IAST [candidate] (23.803 ms) : 0, 23803
Remote Config [baseline] (581.808 µs) : 0, 582
Remote Config [candidate] (578.498 µs) : 0, 578
Telemetry [baseline] (6.663 ms) : 0, 6663
Telemetry [candidate] (8.242 ms) : 0, 8242
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (793.837 ms) : 0, 793837
BytebuddyAgent [candidate] (796.586 ms) : 0, 796586
GlobalTracer [baseline] (289.26 ms) : 0, 289260
GlobalTracer [candidate] (290.472 ms) : 0, 290472
AppSec [baseline] (54.375 ms) : 0, 54375
AppSec [candidate] (53.776 ms) : 0, 53776
IAST [baseline] (21.679 ms) : 0, 21679
IAST [candidate] (21.643 ms) : 0, 21643
Remote Config [baseline] (583.678 µs) : 0, 584
Remote Config [candidate] (578.281 µs) : 0, 578
Telemetry [baseline] (6.596 ms) : 0, 6596
Telemetry [candidate] (6.61 ms) : 0, 6610
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~75e4313559, baseline=1.32.0-SNAPSHOT~d20df91089

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.085 s) : 0, 1084862
Total [baseline] (9.214 s) : 0, 9214326
Agent [candidate] (1.098 s) : 0, 1098323
Total [candidate] (9.226 s) : 0, 9226139
section appsec
Agent [baseline] (1.209 s) : 0, 1209167
Total [baseline] (9.412 s) : 0, 9411972
Agent [candidate] (1.208 s) : 0, 1207901
Total [candidate] (9.409 s) : 0, 9408745
section iast
Agent [baseline] (1.209 s) : 0, 1209317
Total [baseline] (9.323 s) : 0, 9323061
Agent [candidate] (1.224 s) : 0, 1223527
Total [candidate] (9.461 s) : 0, 9461367
section profiling
Agent [baseline] (1.287 s) : 0, 1286527
Total [baseline] (9.386 s) : 0, 9386171
Agent [candidate] (1.285 s) : 0, 1284525
Total [candidate] (9.374 s) : 0, 9374040
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent appsec 1.209 s 124.305 ms (11.5%)
Agent iast 1.209 s 124.455 ms (11.5%)
Agent profiling 1.287 s 201.666 ms (18.6%)
Total tracing 9.214 s -
Total appsec 9.412 s 197.646 ms (2.1%)
Total iast 9.323 s 108.735 ms (1.2%)
Total profiling 9.386 s 171.846 ms (1.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.098 s -
Agent appsec 1.208 s 109.577 ms (10.0%)
Agent iast 1.224 s 125.203 ms (11.4%)
Agent profiling 1.285 s 186.201 ms (17.0%)
Total tracing 9.226 s -
Total appsec 9.409 s 182.605 ms (2.0%)
Total iast 9.461 s 235.228 ms (2.5%)
Total profiling 9.374 s 147.901 ms (1.6%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~75e4313559, baseline=1.32.0-SNAPSHOT~d20df91089

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (698.423 ms) : 0, 698423
BytebuddyAgent [candidate] (707.214 ms) : 0, 707214
GlobalTracer [baseline] (292.368 ms) : 0, 292368
GlobalTracer [candidate] (296.332 ms) : 0, 296332
AppSec [baseline] (51.281 ms) : 0, 51281
AppSec [candidate] (51.327 ms) : 0, 51327
Remote Config [baseline] (724.425 µs) : 0, 724
Remote Config [candidate] (739.977 µs) : 0, 740
Telemetry [baseline] (7.633 ms) : 0, 7633
Telemetry [candidate] (7.833 ms) : 0, 7833
section appsec
BytebuddyAgent [baseline] (698.475 ms) : 0, 698475
BytebuddyAgent [candidate] (699.051 ms) : 0, 699051
GlobalTracer [baseline] (293.704 ms) : 0, 293704
GlobalTracer [candidate] (292.68 ms) : 0, 292680
AppSec [baseline] (157.193 ms) : 0, 157193
AppSec [candidate] (156.217 ms) : 0, 156217
IAST [baseline] (17.949 ms) : 0, 17949
IAST [candidate] (17.964 ms) : 0, 17964
Remote Config [baseline] (611.846 µs) : 0, 612
Remote Config [candidate] (619.203 µs) : 0, 619
Telemetry [baseline] (6.906 ms) : 0, 6906
Telemetry [candidate] (6.948 ms) : 0, 6948
section iast
BytebuddyAgent [baseline] (802.96 ms) : 0, 802960
BytebuddyAgent [candidate] (810.294 ms) : 0, 810294
GlobalTracer [baseline] (289.366 ms) : 0, 289366
GlobalTracer [candidate] (293.529 ms) : 0, 293529
AppSec [baseline] (52.981 ms) : 0, 52981
AppSec [candidate] (53.629 ms) : 0, 53629
IAST [baseline] (22.432 ms) : 0, 22432
IAST [candidate] (22.454 ms) : 0, 22454
Remote Config [baseline] (580.103 µs) : 0, 580
Remote Config [candidate] (589.289 µs) : 0, 589
Telemetry [baseline] (6.638 ms) : 0, 6638
Telemetry [candidate] (8.334 ms) : 0, 8334
section profiling
BytebuddyAgent [baseline] (698.659 ms) : 0, 698659
BytebuddyAgent [candidate] (695.543 ms) : 0, 695543
GlobalTracer [baseline] (375.936 ms) : 0, 375936
GlobalTracer [candidate] (376.562 ms) : 0, 376562
AppSec [baseline] (53.133 ms) : 0, 53133
AppSec [candidate] (53.123 ms) : 0, 53123
Remote Config [baseline] (770.292 µs) : 0, 770
Remote Config [candidate] (818.192 µs) : 0, 818
Telemetry [baseline] (7.49 ms) : 0, 7490
Telemetry [candidate] (7.406 ms) : 0, 7406
ProfilingAgent [baseline] (93.598 ms) : 0, 93598
ProfilingAgent [candidate] (94.585 ms) : 0, 94585
Profiling [baseline] (93.622 ms) : 0, 93622
Profiling [candidate] (94.61 ms) : 0, 94610

Load

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~75e4313559, baseline=1.32.0-SNAPSHOT~d20df91089
    dateFormat X
    axisFormat %s
section baseline
no_agent (371.442 µs) : 350, 393
.   : milestone, 371,
iast (476.252 µs) : 456, 497
.   : milestone, 476,
iast_FULL (538.55 µs) : 518, 559
.   : milestone, 539,
iast_GLOBAL (498.643 µs) : 478, 519
.   : milestone, 499,
iast_HARDCODED_SECRET_DISABLED (479.539 µs) : 459, 500
.   : milestone, 480,
iast_INACTIVE (451.795 µs) : 431, 472
.   : milestone, 452,
iast_TELEMETRY_OFF (485.907 µs) : 465, 507
.   : milestone, 486,
tracing (446.746 µs) : 427, 467
.   : milestone, 447,
section candidate
no_agent (367.627 µs) : 348, 387
.   : milestone, 368,
iast (477.332 µs) : 456, 498
.   : milestone, 477,
iast_FULL (546.037 µs) : 526, 567
.   : milestone, 546,
iast_GLOBAL (508.071 µs) : 486, 531
.   : milestone, 508,
iast_HARDCODED_SECRET_DISABLED (478.487 µs) : 458, 499
.   : milestone, 478,
iast_INACTIVE (456.267 µs) : 435, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (477.741 µs) : 457, 498
.   : milestone, 478,
tracing (445.5 µs) : 425, 466
.   : milestone, 445,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 371.442 µs [350.172 µs, 392.712 µs] -
iast 476.252 µs [455.552 µs, 496.952 µs] 104.81 µs (28.2%)
iast_FULL 538.55 µs [518.186 µs, 558.914 µs] 167.108 µs (45.0%)
iast_GLOBAL 498.643 µs [478.288 µs, 518.997 µs] 127.201 µs (34.2%)
iast_HARDCODED_SECRET_DISABLED 479.539 µs [458.823 µs, 500.255 µs] 108.097 µs (29.1%)
iast_INACTIVE 451.795 µs [431.14 µs, 472.451 µs] 80.353 µs (21.6%)
iast_TELEMETRY_OFF 485.907 µs [464.903 µs, 506.911 µs] 114.465 µs (30.8%)
tracing 446.746 µs [426.711 µs, 466.781 µs] 75.304 µs (20.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.627 µs [348.194 µs, 387.059 µs] -
iast 477.332 µs [456.385 µs, 498.279 µs] 109.705 µs (29.8%)
iast_FULL 546.037 µs [525.572 µs, 566.503 µs] 178.411 µs (48.5%)
iast_GLOBAL 508.071 µs [485.607 µs, 530.534 µs] 140.444 µs (38.2%)
iast_HARDCODED_SECRET_DISABLED 478.487 µs [457.99 µs, 498.983 µs] 110.86 µs (30.2%)
iast_INACTIVE 456.267 µs [435.053 µs, 477.481 µs] 88.64 µs (24.1%)
iast_TELEMETRY_OFF 477.741 µs [457.249 µs, 498.234 µs] 110.115 µs (30.0%)
tracing 445.5 µs [425.287 µs, 465.712 µs] 77.873 µs (21.2%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~75e4313559, baseline=1.32.0-SNAPSHOT~d20df91089
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.372 ms) : 1353, 1391
.   : milestone, 1372,
appsec (1.787 ms) : 1763, 1811
.   : milestone, 1787,
iast (1.539 ms) : 1515, 1563
.   : milestone, 1539,
profiling (1.535 ms) : 1512, 1559
.   : milestone, 1535,
tracing (1.524 ms) : 1500, 1548
.   : milestone, 1524,
section candidate
no_agent (1.345 ms) : 1326, 1364
.   : milestone, 1345,
appsec (1.792 ms) : 1769, 1816
.   : milestone, 1792,
iast (1.521 ms) : 1498, 1545
.   : milestone, 1521,
profiling (1.527 ms) : 1504, 1551
.   : milestone, 1527,
tracing (1.497 ms) : 1473, 1520
.   : milestone, 1497,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.372 ms [1.353 ms, 1.391 ms] -
appsec 1.787 ms [1.763 ms, 1.811 ms] 414.999 µs (30.2%)
iast 1.539 ms [1.515 ms, 1.563 ms] 166.941 µs (12.2%)
profiling 1.535 ms [1.512 ms, 1.559 ms] 163.162 µs (11.9%)
tracing 1.524 ms [1.5 ms, 1.548 ms] 152.204 µs (11.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.345 ms [1.326 ms, 1.364 ms] -
appsec 1.792 ms [1.769 ms, 1.816 ms] 447.168 µs (33.2%)
iast 1.521 ms [1.498 ms, 1.545 ms] 175.968 µs (13.1%)
profiling 1.527 ms [1.504 ms, 1.551 ms] 182.262 µs (13.5%)
tracing 1.497 ms [1.473 ms, 1.52 ms] 151.317 µs (11.2%)

@jbachorik jbachorik marked this pull request as ready for review March 8, 2024 11:58
@jbachorik jbachorik requested a review from a team as a code owner March 8, 2024 11:58
@jbachorik jbachorik merged commit 8887043 into master Mar 8, 2024
81 checks passed
@jbachorik jbachorik deleted the jb/graalvm_version branch March 8, 2024 12:33
@github-actions github-actions bot added this to the 1.32.0 milestone Mar 8, 2024
@PerfectSlayer PerfectSlayer added the comp: native-image GraalVM native-image label Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@richardstartin richardstartin richardstartin approved these changes

Assignees
No one assigned
Labels
comp: native-image GraalVM native-image
Projects
None yet
Milestone
1.32.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@jbachorik @richardstartin @PerfectSlayer