Skip to content

Comments

Adds RC warning to release notes#6662

Merged
am312 merged 3 commits intomasterfrom
andrew.munn/add-rc-warning
Feb 14, 2024
Merged

Adds RC warning to release notes#6662
am312 merged 3 commits intomasterfrom
andrew.munn/add-rc-warning

Conversation

@am312
Copy link
Contributor

@am312 am312 commented Feb 12, 2024

What Does This Do

Adds a warning to top of release notes of RC candidates

@am312 am312 requested a review from PerfectSlayer February 12, 2024 04:10
@am312 am312 changed the title add RC warning Adds RC warning to release notes Feb 12, 2024
@pr-commenter
Copy link

pr-commenter bot commented Feb 12, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master andrew.munn/add-rc-warning
git_commit_date 1707923005 1707926779
git_commit_sha 842dde3 f7a1fdb
release_version 1.31.0-SNAPSHOT~842dde3ef2 1.31.0-SNAPSHOT~f7a1fdb6aa
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1707929740 1707929740
ci_job_id 434110362 434110362
ci_pipeline_id 28294588 28294588
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 10 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.31.0-SNAPSHOT~f7a1fdb6aa, baseline=1.31.0-SNAPSHOT~842dde3ef2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.071 s) : 0, 1071403
Total [baseline] (9.386 s) : 0, 9386396
Agent [candidate] (1.067 s) : 0, 1066948
Total [candidate] (9.382 s) : 0, 9381779
section appsec
Agent [baseline] (1.159 s) : 0, 1159431
Total [baseline] (9.549 s) : 0, 9548521
Agent [candidate] (1.165 s) : 0, 1164903
Total [candidate] (9.508 s) : 0, 9508484
section iast
Agent [baseline] (1.186 s) : 0, 1185635
Total [baseline] (9.679 s) : 0, 9678664
Agent [candidate] (1.185 s) : 0, 1184873
Total [candidate] (9.646 s) : 0, 9645646
section profiling
Agent [baseline] (1.273 s) : 0, 1273028
Total [baseline] (9.553 s) : 0, 9553063
Agent [candidate] (1.287 s) : 0, 1286572
Total [candidate] (9.654 s) : 0, 9654467
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.071 s -
Agent appsec 1.159 s 88.029 ms (8.2%)
Agent iast 1.186 s 114.233 ms (10.7%)
Agent profiling 1.273 s 201.625 ms (18.8%)
Total tracing 9.386 s -
Total appsec 9.549 s 162.126 ms (1.7%)
Total iast 9.679 s 292.268 ms (3.1%)
Total profiling 9.553 s 166.668 ms (1.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.067 s -
Agent appsec 1.165 s 97.955 ms (9.2%)
Agent iast 1.185 s 117.925 ms (11.1%)
Agent profiling 1.287 s 219.624 ms (20.6%)
Total tracing 9.382 s -
Total appsec 9.508 s 126.705 ms (1.4%)
Total iast 9.646 s 263.867 ms (2.8%)
Total profiling 9.654 s 272.688 ms (2.9%) 
gantt
    title petclinic - break down per module: candidate=1.31.0-SNAPSHOT~f7a1fdb6aa, baseline=1.31.0-SNAPSHOT~842dde3ef2

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (676.125 ms) : 0, 676125
BytebuddyAgent [candidate] (672.268 ms) : 0, 672268
GlobalTracer [baseline] (300.325 ms) : 0, 300325
GlobalTracer [candidate] (300.03 ms) : 0, 300030
AppSec [baseline] (51.794 ms) : 0, 51794
AppSec [candidate] (51.885 ms) : 0, 51885
Remote Config [baseline] (701.481 µs) : 0, 701
Remote Config [candidate] (686.178 µs) : 0, 686
Telemetry [baseline] (7.74 ms) : 0, 7740
Telemetry [candidate] (7.651 ms) : 0, 7651
section appsec
BytebuddyAgent [baseline] (668.0 ms) : 0, 668000
BytebuddyAgent [candidate] (672.104 ms) : 0, 672104
GlobalTracer [baseline] (298.092 ms) : 0, 298092
GlobalTracer [candidate] (299.151 ms) : 0, 299151
AppSec [baseline] (151.563 ms) : 0, 151563
AppSec [candidate] (151.576 ms) : 0, 151576
Remote Config [baseline] (649.631 µs) : 0, 650
Remote Config [candidate] (657.154 µs) : 0, 657
Telemetry [baseline] (6.842 ms) : 0, 6842
Telemetry [candidate] (6.877 ms) : 0, 6877
section iast
BytebuddyAgent [baseline] (779.101 ms) : 0, 779101
BytebuddyAgent [candidate] (779.634 ms) : 0, 779634
GlobalTracer [baseline] (289.481 ms) : 0, 289481
GlobalTracer [candidate] (288.791 ms) : 0, 288791
AppSec [baseline] (50.91 ms) : 0, 50910
AppSec [candidate] (52.751 ms) : 0, 52751
IAST [baseline] (24.709 ms) : 0, 24709
IAST [candidate] (22.15 ms) : 0, 22150
Remote Config [baseline] (662.013 µs) : 0, 662
Remote Config [candidate] (613.895 µs) : 0, 614
Telemetry [baseline] (6.544 ms) : 0, 6544
Telemetry [candidate] (6.601 ms) : 0, 6601
section profiling
BytebuddyAgent [baseline] (663.716 ms) : 0, 663716
BytebuddyAgent [candidate] (671.384 ms) : 0, 671384
GlobalTracer [baseline] (379.213 ms) : 0, 379213
GlobalTracer [candidate] (383.66 ms) : 0, 383660
AppSec [baseline] (52.371 ms) : 0, 52371
AppSec [candidate] (52.319 ms) : 0, 52319
Remote Config [baseline] (666.492 µs) : 0, 666
Remote Config [candidate] (658.06 µs) : 0, 658
Telemetry [baseline] (12.279 ms) : 0, 12279
Telemetry [candidate] (10.912 ms) : 0, 10912
ProfilingAgent [baseline] (110.291 ms) : 0, 110291
ProfilingAgent [candidate] (112.832 ms) : 0, 112832
Profiling [baseline] (110.315 ms) : 0, 110315
Profiling [candidate] (112.856 ms) : 0, 112856
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-02-14T16:31:40 2024-02-14T16:50:39
git_branch master andrew.munn/add-rc-warning
git_commit_date 1707923005 1707926779
git_commit_sha 842dde3 f7a1fdb
release_version 1.31.0-SNAPSHOT~842dde3ef2 1.31.0-SNAPSHOT~f7a1fdb6aa
start_time 2024-02-14T16:31:27 2024-02-14T16:50:25
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1707929740 1707929740
ci_job_id 434110362 434110362
ci_pipeline_id 28294588 28294588
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 16 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-105.752µs; -49.520µs] or [-6.707%; -3.140%]		 unstable
[-285.380op/s; +759.454op/s] or [-9.632%; +25.632%]		 1.499ms 3200.000op/s 1.577ms 2962.963op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~f7a1fdb6aa, baseline=1.31.0-SNAPSHOT~842dde3ef2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.362 ms) : 1343, 1381
.   : milestone, 1362,
appsec (1.78 ms) : 1755, 1805
.   : milestone, 1780,
iast (1.526 ms) : 1501, 1552
.   : milestone, 1526,
profiling (1.577 ms) : 1551, 1603
.   : milestone, 1577,
tracing (1.507 ms) : 1482, 1531
.   : milestone, 1507,
section candidate
no_agent (1.355 ms) : 1337, 1374
.   : milestone, 1355,
appsec (1.771 ms) : 1746, 1795
.   : milestone, 1771,
iast (1.526 ms) : 1501, 1550
.   : milestone, 1526,
profiling (1.499 ms) : 1473, 1525
.   : milestone, 1499,
tracing (1.516 ms) : 1491, 1541
.   : milestone, 1516,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.362 ms [1.343 ms, 1.381 ms] -
appsec 1.78 ms [1.755 ms, 1.805 ms] 417.906 µs (30.7%)
iast 1.526 ms [1.501 ms, 1.552 ms] 164.007 µs (12.0%)
profiling 1.577 ms [1.551 ms, 1.603 ms] 214.576 µs (15.8%)
tracing 1.507 ms [1.482 ms, 1.531 ms] 144.415 µs (10.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.337 ms, 1.374 ms] -
appsec 1.771 ms [1.746 ms, 1.795 ms] 415.27 µs (30.6%)
iast 1.526 ms [1.501 ms, 1.55 ms] 170.205 µs (12.6%)
profiling 1.499 ms [1.473 ms, 1.525 ms] 143.755 µs (10.6%)
tracing 1.516 ms [1.491 ms, 1.541 ms] 160.522 µs (11.8%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~f7a1fdb6aa, baseline=1.31.0-SNAPSHOT~842dde3ef2
    dateFormat X
    axisFormat %s
section baseline
no_agent (368.719 µs) : 349, 389
.   : milestone, 369,
iast (475.078 µs) : 454, 496
.   : milestone, 475,
iast_FULL (535.246 µs) : 515, 556
.   : milestone, 535,
iast_GLOBAL (495.835 µs) : 475, 516
.   : milestone, 496,
iast_HARDCODED_SECRET_DISABLED (479.053 µs) : 458, 500
.   : milestone, 479,
iast_INACTIVE (453.764 µs) : 432, 476
.   : milestone, 454,
iast_TELEMETRY_OFF (479.776 µs) : 459, 501
.   : milestone, 480,
tracing (450.415 µs) : 429, 472
.   : milestone, 450,
section candidate
no_agent (363.446 µs) : 343, 383
.   : milestone, 363,
iast (471.907 µs) : 451, 493
.   : milestone, 472,
iast_FULL (537.788 µs) : 517, 559
.   : milestone, 538,
iast_GLOBAL (494.878 µs) : 474, 516
.   : milestone, 495,
iast_HARDCODED_SECRET_DISABLED (479.481 µs) : 458, 501
.   : milestone, 479,
iast_INACTIVE (450.535 µs) : 429, 472
.   : milestone, 451,
iast_TELEMETRY_OFF (467.735 µs) : 447, 488
.   : milestone, 468,
tracing (441.813 µs) : 421, 462
.   : milestone, 442,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 368.719 µs [348.814 µs, 388.625 µs] -
iast 475.078 µs [454.207 µs, 495.948 µs] 106.359 µs (28.8%)
iast_FULL 535.246 µs [514.577 µs, 555.914 µs] 166.526 µs (45.2%)
iast_GLOBAL 495.835 µs [475.316 µs, 516.354 µs] 127.116 µs (34.5%)
iast_HARDCODED_SECRET_DISABLED 479.053 µs [457.734 µs, 500.371 µs] 110.334 µs (29.9%)
iast_INACTIVE 453.764 µs [431.821 µs, 475.707 µs] 85.045 µs (23.1%)
iast_TELEMETRY_OFF 479.776 µs [458.575 µs, 500.977 µs] 111.057 µs (30.1%)
tracing 450.415 µs [428.563 µs, 472.267 µs] 81.695 µs (22.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 363.446 µs [343.494 µs, 383.399 µs] -
iast 471.907 µs [451.226 µs, 492.587 µs] 108.46 µs (29.8%)
iast_FULL 537.788 µs [517.03 µs, 558.545 µs] 174.341 µs (48.0%)
iast_GLOBAL 494.878 µs [473.989 µs, 515.767 µs] 131.432 µs (36.2%)
iast_HARDCODED_SECRET_DISABLED 479.481 µs [458.104 µs, 500.858 µs] 116.034 µs (31.9%)
iast_INACTIVE 450.535 µs [429.407 µs, 471.663 µs] 87.088 µs (24.0%)
iast_TELEMETRY_OFF 467.735 µs [447.15 µs, 488.321 µs] 104.289 µs (28.7%)
tracing 441.813 µs [421.356 µs, 462.269 µs] 78.366 µs (21.6%)

@am312 am312 marked this pull request as ready for review February 12, 2024 05:31
@am312 am312 requested a review from a team as a code owner February 12, 2024 05:31
@am312 am312 requested review from mcculls and removed request for mcculls February 12, 2024 05:31
smola
smola approved these changes Feb 12, 2024
View reviewed changes
Copy link
Member

@smola smola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 as discussed offline.

@smola smola added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Feb 12, 2024
PerfectSlayer
PerfectSlayer reviewed Feb 12, 2024
View reviewed changes
@am312 am312 merged commit 2dcbd3b into master Feb 14, 2024
@am312 am312 deleted the andrew.munn/add-rc-warning branch February 14, 2024 17:29
@github-actions github-actions bot added this to the 1.30.0 milestone Feb 14, 2024
jandro996 pushed a commit that referenced this pull request Feb 16, 2024
@am312 @jandro996
Adds RC warning to release notes (#6662)
5d8fe2e 
add RC warning and support recommendation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer left review comments

@smola smola smola approved these changes

Assignees

No one assigned

Labels

comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.30.0

Development

Successfully merging this pull request may close these issues.

3 participants

@am312 @smola @PerfectSlayer