Skip to content

Remove Trivy scan from PR checks#8890

Merged
PerfectSlayer merged 1 commit intomasterfrom
bbujon/ci
May 27, 2025
Merged

Remove Trivy scan from PR checks#8890
PerfectSlayer merged 1 commit intomasterfrom
bbujon/ci

Conversation

@PerfectSlayer
Copy link
Copy Markdown
Contributor

@PerfectSlayer PerfectSlayer commented May 27, 2025

What Does This Do

Trivy check should not be required as it is failing for 3rd party contribution.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: LANGPLAT-528

@PerfectSlayer PerfectSlayer added the type: bug Bug report and fix label May 27, 2025
@PerfectSlayer PerfectSlayer requested a review from a team as a code owner May 27, 2025 04:45
@PerfectSlayer PerfectSlayer added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels May 27, 2025
@PerfectSlayer PerfectSlayer requested review from bric3 and removed request for a team May 27, 2025 04:45
@PerfectSlayer PerfectSlayer enabled auto-merge (squash) May 27, 2025 04:46
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 27, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/ci
git_commit_date 1746789389 1748353352
git_commit_sha ad6d5fe eb09981
release_version 1.50.0-SNAPSHOT~ad6d5fef42 1.50.0-SNAPSHOT~eb09981fbf
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1748355615 1748355615
ci_job_id 955629885 955629885
ci_pipeline_id 66197518 66197518
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-qshkhgbg-project-304-concurrent-1-u7zjbyu1 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-qshkhgbg-project-304-concurrent-1-u7zjbyu1 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 1 performance improvements and 1 performance regressions! Performance is the same for 55 metrics, 14 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:profiling:GlobalTracer better
[-19.684ms; -8.847ms] or [-5.194%; -2.335%]
364.684ms 378.950ms
scenario:startup:petclinic:profiling:AppSec worse
[+5.981ms; +9.494ms] or [+10.943%; +17.370%]
62.392ms 54.655ms
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~eb09981fbf, baseline=1.50.0-SNAPSHOT~ad6d5fef42

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.021 s) : 0, 1020683
Total [baseline] (8.64 s) : 0, 8640459
Agent [candidate] (1.034 s) : 0, 1034100
Total [candidate] (8.682 s) : 0, 8682487
section iast
Agent [baseline] (1.147 s) : 0, 1146914
Total [baseline] (9.227 s) : 0, 9226896
Agent [candidate] (1.158 s) : 0, 1157971
Total [candidate] (9.25 s) : 0, 9249898
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.156 s) : 0, 1155557
Total [baseline] (9.258 s) : 0, 9258347
Agent [candidate] (1.152 s) : 0, 1152347
Total [candidate] (9.195 s) : 0, 9195201
section iast_TELEMETRY_OFF
Agent [baseline] (1.155 s) : 0, 1155219
Total [baseline] (9.243 s) : 0, 9243401
Agent [candidate] (1.154 s) : 0, 1154476
Total [candidate] (9.238 s) : 0, 9238280
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.021 s -
Agent iast 1.147 s 126.231 ms (12.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.156 s 134.874 ms (13.2%)
Agent iast_TELEMETRY_OFF 1.155 s 134.536 ms (13.2%)
Total tracing 8.64 s -
Total iast 9.227 s 586.437 ms (6.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.258 s 617.887 ms (7.2%)
Total iast_TELEMETRY_OFF 9.243 s 602.941 ms (7.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.034 s -
Agent iast 1.158 s 123.871 ms (12.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.152 s 118.246 ms (11.4%)
Agent iast_TELEMETRY_OFF 1.154 s 120.376 ms (11.6%)
Total tracing 8.682 s -
Total iast 9.25 s 567.41 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.195 s 512.714 ms (5.9%)
Total iast_TELEMETRY_OFF 9.238 s 555.793 ms (6.4%)
gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~eb09981fbf, baseline=1.50.0-SNAPSHOT~ad6d5fef42

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.172 ms) : 0, 682172
BytebuddyAgent [candidate] (692.876 ms) : 0, 692876
GlobalTracer [baseline] (239.836 ms) : 0, 239836
GlobalTracer [candidate] (243.114 ms) : 0, 243114
AppSec [baseline] (54.255 ms) : 0, 54255
AppSec [candidate] (56.002 ms) : 0, 56002
Debugger [baseline] (10.407 ms) : 0, 10407
Debugger [candidate] (8.987 ms) : 0, 8987
Remote Config [baseline] (682.798 µs) : 0, 683
Remote Config [candidate] (707.606 µs) : 0, 708
Telemetry [baseline] (9.703 ms) : 0, 9703
Telemetry [candidate] (8.456 ms) : 0, 8456
section iast
BytebuddyAgent [baseline] (800.593 ms) : 0, 800593
BytebuddyAgent [candidate] (808.024 ms) : 0, 808024
GlobalTracer [baseline] (229.813 ms) : 0, 229813
GlobalTracer [candidate] (232.25 ms) : 0, 232250
IAST [baseline] (28.377 ms) : 0, 28377
IAST [candidate] (28.661 ms) : 0, 28661
AppSec [baseline] (50.251 ms) : 0, 50251
AppSec [candidate] (50.869 ms) : 0, 50869
Debugger [baseline] (5.932 ms) : 0, 5932
Debugger [candidate] (5.984 ms) : 0, 5984
Remote Config [baseline] (592.558 µs) : 0, 593
Remote Config [candidate] (590.514 µs) : 0, 591
Telemetry [baseline] (7.878 ms) : 0, 7878
Telemetry [candidate] (7.946 ms) : 0, 7946
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (806.179 ms) : 0, 806179
BytebuddyAgent [candidate] (803.951 ms) : 0, 803951
GlobalTracer [baseline] (232.16 ms) : 0, 232160
GlobalTracer [candidate] (231.771 ms) : 0, 231771
IAST [baseline] (29.599 ms) : 0, 29599
IAST [candidate] (28.343 ms) : 0, 28343
AppSec [baseline] (49.613 ms) : 0, 49613
AppSec [candidate] (50.316 ms) : 0, 50316
Debugger [baseline] (5.943 ms) : 0, 5943
Debugger [candidate] (5.926 ms) : 0, 5926
Remote Config [baseline] (598.149 µs) : 0, 598
Remote Config [candidate] (586.683 µs) : 0, 587
Telemetry [baseline] (7.881 ms) : 0, 7881
Telemetry [candidate] (7.868 ms) : 0, 7868
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (806.381 ms) : 0, 806381
BytebuddyAgent [candidate] (805.338 ms) : 0, 805338
GlobalTracer [baseline] (232.33 ms) : 0, 232330
GlobalTracer [candidate] (232.407 ms) : 0, 232407
IAST [baseline] (22.554 ms) : 0, 22554
IAST [candidate] (23.466 ms) : 0, 23466
AppSec [baseline] (55.879 ms) : 0, 55879
AppSec [candidate] (55.173 ms) : 0, 55173
Debugger [baseline] (5.969 ms) : 0, 5969
Debugger [candidate] (5.978 ms) : 0, 5978
Remote Config [baseline] (607.52 µs) : 0, 608
Remote Config [candidate] (604.187 µs) : 0, 604
Telemetry [baseline] (7.78 ms) : 0, 7780
Telemetry [candidate] (7.829 ms) : 0, 7829
Loading
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~eb09981fbf, baseline=1.50.0-SNAPSHOT~ad6d5fef42

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.029 s) : 0, 1028850
Total [baseline] (10.498 s) : 0, 10497692
Agent [candidate] (1.026 s) : 0, 1025739
Total [candidate] (10.486 s) : 0, 10485509
section appsec
Agent [baseline] (1.17 s) : 0, 1169564
Total [baseline] (10.72 s) : 0, 10719735
Agent [candidate] (1.165 s) : 0, 1165262
Total [candidate] (10.662 s) : 0, 10661648
section iast
Agent [baseline] (1.15 s) : 0, 1149980
Total [baseline] (10.954 s) : 0, 10953903
Agent [candidate] (1.151 s) : 0, 1151484
Total [candidate] (10.87 s) : 0, 10869562
section profiling
Agent [baseline] (1.282 s) : 0, 1282196
Total [baseline] (10.944 s) : 0, 10943932
Agent [candidate] (1.287 s) : 0, 1286506
Total [candidate] (10.891 s) : 0, 10891357
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.029 s -
Agent appsec 1.17 s 140.714 ms (13.7%)
Agent iast 1.15 s 121.13 ms (11.8%)
Agent profiling 1.282 s 253.346 ms (24.6%)
Total tracing 10.498 s -
Total appsec 10.72 s 222.043 ms (2.1%)
Total iast 10.954 s 456.211 ms (4.3%)
Total profiling 10.944 s 446.24 ms (4.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.026 s -
Agent appsec 1.165 s 139.522 ms (13.6%)
Agent iast 1.151 s 125.745 ms (12.3%)
Agent profiling 1.287 s 260.766 ms (25.4%)
Total tracing 10.486 s -
Total appsec 10.662 s 176.138 ms (1.7%)
Total iast 10.87 s 384.052 ms (3.7%)
Total profiling 10.891 s 405.848 ms (3.9%)
gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~eb09981fbf, baseline=1.50.0-SNAPSHOT~ad6d5fef42

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.478 ms) : 0, 688478
BytebuddyAgent [candidate] (685.774 ms) : 0, 685774
GlobalTracer [baseline] (242.22 ms) : 0, 242220
GlobalTracer [candidate] (241.826 ms) : 0, 241826
AppSec [baseline] (54.887 ms) : 0, 54887
AppSec [candidate] (55.66 ms) : 0, 55660
Debugger [baseline] (9.093 ms) : 0, 9093
Debugger [candidate] (9.767 ms) : 0, 9767
Remote Config [baseline] (685.744 µs) : 0, 686
Remote Config [candidate] (708.318 µs) : 0, 708
Telemetry [baseline] (9.759 ms) : 0, 9759
Telemetry [candidate] (8.352 ms) : 0, 8352
section appsec
BytebuddyAgent [baseline] (706.241 ms) : 0, 706241
BytebuddyAgent [candidate] (703.656 ms) : 0, 703656
GlobalTracer [baseline] (238.396 ms) : 0, 238396
GlobalTracer [candidate] (237.244 ms) : 0, 237244
IAST [baseline] (21.674 ms) : 0, 21674
IAST [candidate] (21.674 ms) : 0, 21674
AppSec [baseline] (176.543 ms) : 0, 176543
AppSec [candidate] (175.564 ms) : 0, 175564
Debugger [baseline] (5.912 ms) : 0, 5912
Debugger [candidate] (5.942 ms) : 0, 5942
Remote Config [baseline] (620.749 µs) : 0, 621
Remote Config [candidate] (622.974 µs) : 0, 623
Telemetry [baseline] (7.321 ms) : 0, 7321
Telemetry [candidate] (7.728 ms) : 0, 7728
section iast
BytebuddyAgent [baseline] (803.224 ms) : 0, 803224
BytebuddyAgent [candidate] (803.737 ms) : 0, 803737
GlobalTracer [baseline] (230.632 ms) : 0, 230632
GlobalTracer [candidate] (230.941 ms) : 0, 230941
IAST [baseline] (28.125 ms) : 0, 28125
IAST [candidate] (28.463 ms) : 0, 28463
AppSec [baseline] (49.355 ms) : 0, 49355
AppSec [candidate] (49.765 ms) : 0, 49765
Debugger [baseline] (5.935 ms) : 0, 5935
Debugger [candidate] (5.89 ms) : 0, 5890
Remote Config [baseline] (589.961 µs) : 0, 590
Remote Config [candidate] (590.798 µs) : 0, 591
Telemetry [baseline] (7.842 ms) : 0, 7842
Telemetry [candidate] (7.884 ms) : 0, 7884
section profiling
BytebuddyAgent [baseline] (673.835 ms) : 0, 673835
BytebuddyAgent [candidate] (685.24 ms) : 0, 685240
GlobalTracer [baseline] (378.95 ms) : 0, 378950
GlobalTracer [candidate] (364.684 ms) : 0, 364684
AppSec [baseline] (54.655 ms) : 0, 54655
AppSec [candidate] (62.392 ms) : 0, 62392
Debugger [baseline] (6.106 ms) : 0, 6106
Debugger [candidate] (6.365 ms) : 0, 6365
Remote Config [baseline] (644.024 µs) : 0, 644
Remote Config [candidate] (663.509 µs) : 0, 664
Telemetry [baseline] (8.124 ms) : 0, 8124
Telemetry [candidate] (8.29 ms) : 0, 8290
ProfilingAgent [baseline] (109.281 ms) : 0, 109281
ProfilingAgent [candidate] (107.237 ms) : 0, 107237
Profiling [baseline] (109.306 ms) : 0, 109306
Profiling [candidate] (107.262 ms) : 0, 107262
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-05-27T13:50:35 2025-05-27T13:58:19
git_branch master bbujon/ci
git_commit_date 1746789389 1748353352
git_commit_sha ad6d5fe eb09981
release_version 1.50.0-SNAPSHOT~ad6d5fef42 1.50.0-SNAPSHOT~eb09981fbf
start_time 2025-05-27T13:50:21 2025-05-27T13:58:05
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1748354697 1748354697
ci_job_id 955629886 955629886
ci_pipeline_id 66197518 66197518
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-dtsgutkg-project-304-concurrent-2-p7cj7vwx 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-dtsgutkg-project-304-concurrent-2-p7cj7vwx 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~eb09981fbf, baseline=1.50.0-SNAPSHOT~ad6d5fef42
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.364 ms) : 1345, 1383
.   : milestone, 1364,
appsec (1.735 ms) : 1711, 1759
.   : milestone, 1735,
appsec_no_iast (1.737 ms) : 1713, 1760
.   : milestone, 1737,
code_origins (1.666 ms) : 1636, 1696
.   : milestone, 1666,
iast (1.51 ms) : 1485, 1535
.   : milestone, 1510,
profiling (1.513 ms) : 1489, 1537
.   : milestone, 1513,
tracing (1.508 ms) : 1484, 1532
.   : milestone, 1508,
section candidate
no_agent (1.355 ms) : 1335, 1374
.   : milestone, 1355,
appsec (1.748 ms) : 1725, 1772
.   : milestone, 1748,
appsec_no_iast (1.742 ms) : 1719, 1765
.   : milestone, 1742,
code_origins (1.679 ms) : 1653, 1706
.   : milestone, 1679,
iast (1.503 ms) : 1478, 1527
.   : milestone, 1503,
profiling (1.539 ms) : 1514, 1564
.   : milestone, 1539,
tracing (1.476 ms) : 1451, 1501
.   : milestone, 1476,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.364 ms [1.345 ms, 1.383 ms] -
appsec 1.735 ms [1.711 ms, 1.759 ms] 370.705 µs (27.2%)
appsec_no_iast 1.737 ms [1.713 ms, 1.76 ms] 372.662 µs (27.3%)
code_origins 1.666 ms [1.636 ms, 1.696 ms] 302.054 µs (22.1%)
iast 1.51 ms [1.485 ms, 1.535 ms] 146.023 µs (10.7%)
profiling 1.513 ms [1.489 ms, 1.537 ms] 148.76 µs (10.9%)
tracing 1.508 ms [1.484 ms, 1.532 ms] 144.172 µs (10.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.335 ms, 1.374 ms] -
appsec 1.748 ms [1.725 ms, 1.772 ms] 393.706 µs (29.1%)
appsec_no_iast 1.742 ms [1.719 ms, 1.765 ms] 387.146 µs (28.6%)
code_origins 1.679 ms [1.653 ms, 1.706 ms] 324.967 µs (24.0%)
iast 1.503 ms [1.478 ms, 1.527 ms] 148.041 µs (10.9%)
profiling 1.539 ms [1.514 ms, 1.564 ms] 184.715 µs (13.6%)
tracing 1.476 ms [1.451 ms, 1.501 ms] 121.138 µs (8.9%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~eb09981fbf, baseline=1.50.0-SNAPSHOT~ad6d5fef42
    dateFormat X
    axisFormat %s
section baseline
no_agent (377.01 µs) : 357, 397
.   : milestone, 377,
iast (514.955 µs) : 492, 538
.   : milestone, 515,
iast_FULL (727.181 µs) : 705, 749
.   : milestone, 727,
iast_GLOBAL (554.684 µs) : 533, 576
.   : milestone, 555,
iast_HARDCODED_SECRET_DISABLED (520.678 µs) : 497, 544
.   : milestone, 521,
iast_INACTIVE (464.706 µs) : 443, 486
.   : milestone, 465,
iast_TELEMETRY_OFF (515.128 µs) : 492, 538
.   : milestone, 515,
tracing (463.797 µs) : 441, 486
.   : milestone, 464,
section candidate
no_agent (385.442 µs) : 366, 405
.   : milestone, 385,
iast (528.11 µs) : 504, 552
.   : milestone, 528,
iast_FULL (734.399 µs) : 713, 756
.   : milestone, 734,
iast_GLOBAL (575.275 µs) : 553, 598
.   : milestone, 575,
iast_HARDCODED_SECRET_DISABLED (527.042 µs) : 505, 549
.   : milestone, 527,
iast_INACTIVE (466.164 µs) : 443, 489
.   : milestone, 466,
iast_TELEMETRY_OFF (517.791 µs) : 495, 541
.   : milestone, 518,
tracing (456.766 µs) : 436, 478
.   : milestone, 457,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 377.01 µs [356.765 µs, 397.255 µs] -
iast 514.955 µs [491.505 µs, 538.405 µs] 137.945 µs (36.6%)
iast_FULL 727.181 µs [705.257 µs, 749.105 µs] 350.17 µs (92.9%)
iast_GLOBAL 554.684 µs [533.123 µs, 576.245 µs] 177.674 µs (47.1%)
iast_HARDCODED_SECRET_DISABLED 520.678 µs [496.91 µs, 544.446 µs] 143.668 µs (38.1%)
iast_INACTIVE 464.706 µs [442.914 µs, 486.497 µs] 87.695 µs (23.3%)
iast_TELEMETRY_OFF 515.128 µs [491.874 µs, 538.382 µs] 138.118 µs (36.6%)
tracing 463.797 µs [441.392 µs, 486.201 µs] 86.786 µs (23.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.442 µs [365.921 µs, 404.964 µs] -
iast 528.11 µs [504.028 µs, 552.192 µs] 142.668 µs (37.0%)
iast_FULL 734.399 µs [712.69 µs, 756.109 µs] 348.957 µs (90.5%)
iast_GLOBAL 575.275 µs [552.656 µs, 597.894 µs] 189.832 µs (49.3%)
iast_HARDCODED_SECRET_DISABLED 527.042 µs [504.711 µs, 549.373 µs] 141.6 µs (36.7%)
iast_INACTIVE 466.164 µs [443.249 µs, 489.079 µs] 80.722 µs (20.9%)
iast_TELEMETRY_OFF 517.791 µs [494.725 µs, 540.856 µs] 132.349 µs (34.3%)
tracing 456.766 µs [435.52 µs, 478.012 µs] 71.324 µs (18.5%)

Dacapo

Copy link
Copy Markdown
Member

@smola smola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we could have a non-privileged job to create the SBOM, upload it as an artifact, and use a separate job for the datadog-ci upload. I guess that would let the bulk of the job to be run on forks?

Still, approving to unblock.

@bric3
Copy link
Copy Markdown
Contributor

bric3 commented May 27, 2025

Shouldn't we instead disable this job, if the key is not available ?

Screenshot 2025-05-27 at 11 29 16

@PerfectSlayer
Copy link
Copy Markdown
Contributor Author

After discussing with @smola, we went with running Trivy only on master push, in addition to adding monitors to make sure it keeps running.

@PerfectSlayer PerfectSlayer merged commit 6bf2458 into master May 27, 2025
917 of 921 checks passed
@PerfectSlayer PerfectSlayer deleted the bbujon/ci branch May 27, 2025 15:19
@github-actions github-actions Bot added this to the 1.50.0 milestone May 27, 2025
@PerfectSlayer PerfectSlayer changed the title Add trivy as non required check Remove Trivy scan from PR checks May 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes type: bug Bug report and fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants