Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increase spark listener max collection size #6979

Merged
merged 1 commit into from
May 3, 2024
Merged

Increase spark listener max collection size #6979

merged 1 commit into from
May 3, 2024

Conversation

paul-laffon-dd
Copy link
Contributor

@paul-laffon-dd paul-laffon-dd commented May 2, 2024 •
edited

What Does This Do

Increase the maximum number of retained events for the injected spark listener

Assuming spans are taking up to ~2kB of memory, going from 1000 to 5000 events can increase the memory impact from 2MB up to 10MB. This should be acceptable since only one listener is injected per spark driver

Motivation

Customer with some big spark clusters reaching this limit

Additional Notes

@paul-laffon-dd paul-laffon-dd added the inst: apache spark Apache Spark instrumentation label May 2, 2024
@pr-commenter
Copy link

pr-commenter bot commented May 2, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master paul.laffon/increase-spark-listener-max-event-size
git_commit_date 1714584487 1714640286
git_commit_sha 5c8d55c dbf06bb
release_version 1.34.0-SNAPSHOT~5c8d55ca87 1.34.0-SNAPSHOT~dbf06bb509
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1714643470 1714643470
ci_job_id 502347396 502347396
ci_pipeline_id 33413260 33413260
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 12 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.34.0-SNAPSHOT~dbf06bb509, baseline=1.34.0-SNAPSHOT~5c8d55ca87

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.074 s) : 0, 1073636
Total [baseline] (10.376 s) : 0, 10376371
Agent [candidate] (1.075 s) : 0, 1074709
Total [candidate] (10.364 s) : 0, 10364005
section appsec
Agent [baseline] (1.192 s) : 0, 1191846
Total [baseline] (10.488 s) : 0, 10488474
Agent [candidate] (1.189 s) : 0, 1188807
Total [candidate] (10.512 s) : 0, 10511657
section iast
Agent [baseline] (1.214 s) : 0, 1213527
Total [baseline] (10.765 s) : 0, 10764755
Agent [candidate] (1.209 s) : 0, 1209234
Total [candidate] (10.79 s) : 0, 10789641
section profiling
Agent [baseline] (1.269 s) : 0, 1268883
Total [baseline] (10.609 s) : 0, 10608785
Agent [candidate] (1.27 s) : 0, 1269859
Total [candidate] (10.614 s) : 0, 10614198
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.074 s -
Agent appsec 1.192 s 118.21 ms (11.0%)
Agent iast 1.214 s 139.891 ms (13.0%)
Agent profiling 1.269 s 195.247 ms (18.2%)
Total tracing 10.376 s -
Total appsec 10.488 s 112.104 ms (1.1%)
Total iast 10.765 s 388.384 ms (3.7%)
Total profiling 10.609 s 232.414 ms (2.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.075 s -
Agent appsec 1.189 s 114.098 ms (10.6%)
Agent iast 1.209 s 134.525 ms (12.5%)
Agent profiling 1.27 s 195.15 ms (18.2%)
Total tracing 10.364 s -
Total appsec 10.512 s 147.653 ms (1.4%)
Total iast 10.79 s 425.636 ms (4.1%)
Total profiling 10.614 s 250.193 ms (2.4%) 
gantt
    title petclinic - break down per module: candidate=1.34.0-SNAPSHOT~dbf06bb509, baseline=1.34.0-SNAPSHOT~5c8d55ca87

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (673.372 ms) : 0, 673372
BytebuddyAgent [candidate] (673.189 ms) : 0, 673189
GlobalTracer [baseline] (308.323 ms) : 0, 308323
GlobalTracer [candidate] (309.591 ms) : 0, 309591
AppSec [baseline] (49.433 ms) : 0, 49433
AppSec [candidate] (49.389 ms) : 0, 49389
Remote Config [baseline] (658.718 µs) : 0, 659
Remote Config [candidate] (654.558 µs) : 0, 655
Telemetry [baseline] (7.516 ms) : 0, 7516
Telemetry [candidate] (7.525 ms) : 0, 7525
section appsec
BytebuddyAgent [baseline] (698.558 ms) : 0, 698558
BytebuddyAgent [candidate] (695.979 ms) : 0, 695979
GlobalTracer [baseline] (291.538 ms) : 0, 291538
GlobalTracer [candidate] (291.645 ms) : 0, 291645
AppSec [baseline] (149.109 ms) : 0, 149109
AppSec [candidate] (149.371 ms) : 0, 149371
IAST [baseline] (19.247 ms) : 0, 19247
IAST [candidate] (19.237 ms) : 0, 19237
Remote Config [baseline] (620.455 µs) : 0, 620
Remote Config [candidate] (617.559 µs) : 0, 618
Telemetry [baseline] (7.898 ms) : 0, 7898
Telemetry [candidate] (7.168 ms) : 0, 7168
section iast
BytebuddyAgent [baseline] (802.714 ms) : 0, 802714
BytebuddyAgent [candidate] (801.345 ms) : 0, 801345
GlobalTracer [baseline] (292.658 ms) : 0, 292658
GlobalTracer [candidate] (290.39 ms) : 0, 290390
AppSec [baseline] (53.439 ms) : 0, 53439
AppSec [candidate] (49.85 ms) : 0, 49850
IAST [baseline] (22.675 ms) : 0, 22675
IAST [candidate] (24.991 ms) : 0, 24991
Remote Config [baseline] (592.64 µs) : 0, 593
Remote Config [candidate] (1.391 ms) : 0, 1391
Telemetry [baseline] (6.729 ms) : 0, 6729
Telemetry [candidate] (6.674 ms) : 0, 6674
section profiling
BytebuddyAgent [baseline] (678.156 ms) : 0, 678156
BytebuddyAgent [candidate] (678.663 ms) : 0, 678663
GlobalTracer [baseline] (380.435 ms) : 0, 380435
GlobalTracer [candidate] (380.431 ms) : 0, 380431
AppSec [baseline] (50.348 ms) : 0, 50348
AppSec [candidate] (50.23 ms) : 0, 50230
Remote Config [baseline] (706.749 µs) : 0, 707
Remote Config [candidate] (713.212 µs) : 0, 713
Telemetry [baseline] (7.5 ms) : 0, 7500
Telemetry [candidate] (7.46 ms) : 0, 7460
ProfilingAgent [baseline] (95.356 ms) : 0, 95356
ProfilingAgent [candidate] (95.85 ms) : 0, 95850
Profiling [baseline] (95.379 ms) : 0, 95379
Profiling [candidate] (95.874 ms) : 0, 95874
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.34.0-SNAPSHOT~dbf06bb509, baseline=1.34.0-SNAPSHOT~5c8d55ca87

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.074 s) : 0, 1074445
Total [baseline] (8.535 s) : 0, 8534692
Agent [candidate] (1.079 s) : 0, 1079435
Total [candidate] (8.591 s) : 0, 8590732
section iast
Agent [baseline] (1.209 s) : 0, 1209142
Total [baseline] (9.025 s) : 0, 9025152
Agent [candidate] (1.212 s) : 0, 1212285
Total [candidate] (9.006 s) : 0, 9005611
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.208 s) : 0, 1207898
Total [baseline] (8.986 s) : 0, 8985751
Agent [candidate] (1.208 s) : 0, 1208308
Total [candidate] (8.983 s) : 0, 8983439
section iast_TELEMETRY_OFF
Agent [baseline] (1.205 s) : 0, 1205159
Total [baseline] (9.005 s) : 0, 9005357
Agent [candidate] (1.198 s) : 0, 1198326
Total [candidate] (8.993 s) : 0, 8992879
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.074 s -
Agent iast 1.209 s 134.697 ms (12.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.208 s 133.454 ms (12.4%)
Agent iast_TELEMETRY_OFF 1.205 s 130.715 ms (12.2%)
Total tracing 8.535 s -
Total iast 9.025 s 490.46 ms (5.7%)
Total iast_HARDCODED_SECRET_DISABLED 8.986 s 451.059 ms (5.3%)
Total iast_TELEMETRY_OFF 9.005 s 470.665 ms (5.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.079 s -
Agent iast 1.212 s 132.85 ms (12.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.208 s 128.873 ms (11.9%)
Agent iast_TELEMETRY_OFF 1.198 s 118.891 ms (11.0%)
Total tracing 8.591 s -
Total iast 9.006 s 414.878 ms (4.8%)
Total iast_HARDCODED_SECRET_DISABLED 8.983 s 392.706 ms (4.6%)
Total iast_TELEMETRY_OFF 8.993 s 402.146 ms (4.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.34.0-SNAPSHOT~dbf06bb509, baseline=1.34.0-SNAPSHOT~5c8d55ca87

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (673.184 ms) : 0, 673184
BytebuddyAgent [candidate] (675.976 ms) : 0, 675976
GlobalTracer [baseline] (309.247 ms) : 0, 309247
GlobalTracer [candidate] (311.312 ms) : 0, 311312
AppSec [baseline] (49.527 ms) : 0, 49527
AppSec [candidate] (49.411 ms) : 0, 49411
Remote Config [baseline] (656.984 µs) : 0, 657
Remote Config [candidate] (662.309 µs) : 0, 662
Telemetry [baseline] (7.529 ms) : 0, 7529
Telemetry [candidate] (7.561 ms) : 0, 7561
section iast
BytebuddyAgent [baseline] (801.094 ms) : 0, 801094
BytebuddyAgent [candidate] (802.344 ms) : 0, 802344
GlobalTracer [baseline] (290.469 ms) : 0, 290469
GlobalTracer [candidate] (292.493 ms) : 0, 292493
AppSec [baseline] (51.549 ms) : 0, 51549
AppSec [candidate] (51.912 ms) : 0, 51912
IAST [baseline] (24.27 ms) : 0, 24270
IAST [candidate] (23.516 ms) : 0, 23516
Remote Config [baseline] (577.942 µs) : 0, 578
Remote Config [candidate] (584.252 µs) : 0, 584
Telemetry [baseline] (6.609 ms) : 0, 6609
Telemetry [candidate] (6.731 ms) : 0, 6731
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.667 ms) : 0, 800667
BytebuddyAgent [candidate] (800.66 ms) : 0, 800660
GlobalTracer [baseline] (289.925 ms) : 0, 289925
GlobalTracer [candidate] (290.681 ms) : 0, 290681
AppSec [baseline] (49.39 ms) : 0, 49390
AppSec [candidate] (53.163 ms) : 0, 53163
IAST [baseline] (24.487 ms) : 0, 24487
IAST [candidate] (21.899 ms) : 0, 21899
Remote Config [baseline] (2.057 ms) : 0, 2057
Remote Config [candidate] (583.268 µs) : 0, 583
Telemetry [baseline] (6.769 ms) : 0, 6769
Telemetry [candidate] (6.72 ms) : 0, 6720
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (798.472 ms) : 0, 798472
BytebuddyAgent [candidate] (793.179 ms) : 0, 793179
GlobalTracer [baseline] (290.055 ms) : 0, 290055
GlobalTracer [candidate] (288.808 ms) : 0, 288808
AppSec [baseline] (50.586 ms) : 0, 50586
AppSec [candidate] (50.372 ms) : 0, 50372
IAST [baseline] (23.496 ms) : 0, 23496
IAST [candidate] (24.431 ms) : 0, 24431
Remote Config [baseline] (590.154 µs) : 0, 590
Remote Config [candidate] (582.219 µs) : 0, 582
Telemetry [baseline] (7.367 ms) : 0, 7367
Telemetry [candidate] (6.643 ms) : 0, 6643

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-05-02T09:21:53 2024-05-02T09:28:41
git_branch master paul.laffon/increase-spark-listener-max-event-size
git_commit_date 1714584487 1714640286
git_commit_sha 5c8d55c dbf06bb
release_version 1.34.0-SNAPSHOT~5c8d55ca87 1.34.0-SNAPSHOT~dbf06bb509
start_time 2024-05-02T09:21:40 2024-05-02T09:28:28
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1714642466 1714642466
ci_job_id 502347397 502347397
ci_pipeline_id 33413260 33413260
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~dbf06bb509, baseline=1.34.0-SNAPSHOT~5c8d55ca87
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.69 µs) : 353, 392
.   : milestone, 373,
iast (476.205 µs) : 455, 497
.   : milestone, 476,
iast_FULL (542.026 µs) : 521, 564
.   : milestone, 542,
iast_GLOBAL (496.996 µs) : 476, 518
.   : milestone, 497,
iast_HARDCODED_SECRET_DISABLED (473.872 µs) : 454, 494
.   : milestone, 474,
iast_INACTIVE (448.071 µs) : 428, 468
.   : milestone, 448,
iast_TELEMETRY_OFF (470.763 µs) : 449, 492
.   : milestone, 471,
tracing (444.969 µs) : 425, 465
.   : milestone, 445,
section candidate
no_agent (374.532 µs) : 354, 395
.   : milestone, 375,
iast (472.185 µs) : 451, 493
.   : milestone, 472,
iast_FULL (547.796 µs) : 526, 570
.   : milestone, 548,
iast_GLOBAL (503.919 µs) : 483, 525
.   : milestone, 504,
iast_HARDCODED_SECRET_DISABLED (477.605 µs) : 456, 499
.   : milestone, 478,
iast_INACTIVE (458.026 µs) : 437, 480
.   : milestone, 458,
iast_TELEMETRY_OFF (475.232 µs) : 454, 497
.   : milestone, 475,
tracing (439.424 µs) : 419, 459
.   : milestone, 439,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.69 µs [352.905 µs, 392.474 µs] -
iast 476.205 µs [455.46 µs, 496.95 µs] 103.515 µs (27.8%)
iast_FULL 542.026 µs [520.534 µs, 563.518 µs] 169.336 µs (45.4%)
iast_GLOBAL 496.996 µs [475.927 µs, 518.065 µs] 124.306 µs (33.4%)
iast_HARDCODED_SECRET_DISABLED 473.872 µs [453.593 µs, 494.151 µs] 101.182 µs (27.1%)
iast_INACTIVE 448.071 µs [427.778 µs, 468.365 µs] 75.382 µs (20.2%)
iast_TELEMETRY_OFF 470.763 µs [449.425 µs, 492.102 µs] 98.074 µs (26.3%)
tracing 444.969 µs [424.551 µs, 465.387 µs] 72.279 µs (19.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 374.532 µs [354.312 µs, 394.752 µs] -
iast 472.185 µs [451.237 µs, 493.133 µs] 97.653 µs (26.1%)
iast_FULL 547.796 µs [526.002 µs, 569.59 µs] 173.264 µs (46.3%)
iast_GLOBAL 503.919 µs [482.678 µs, 525.159 µs] 129.387 µs (34.5%)
iast_HARDCODED_SECRET_DISABLED 477.605 µs [456.25 µs, 498.959 µs] 103.073 µs (27.5%)
iast_INACTIVE 458.026 µs [436.521 µs, 479.53 µs] 83.494 µs (22.3%)
iast_TELEMETRY_OFF 475.232 µs [453.883 µs, 496.581 µs] 100.7 µs (26.9%)
tracing 439.424 µs [419.436 µs, 459.413 µs] 64.893 µs (17.3%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~dbf06bb509, baseline=1.34.0-SNAPSHOT~5c8d55ca87
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.352 ms) : 1332, 1371
.   : milestone, 1352,
appsec (1.749 ms) : 1724, 1773
.   : milestone, 1749,
appsec_no_iast (1.713 ms) : 1688, 1738
.   : milestone, 1713,
iast (1.498 ms) : 1476, 1520
.   : milestone, 1498,
profiling (1.5 ms) : 1475, 1524
.   : milestone, 1500,
tracing (1.474 ms) : 1450, 1499
.   : milestone, 1474,
section candidate
no_agent (1.341 ms) : 1323, 1360
.   : milestone, 1341,
appsec (1.728 ms) : 1702, 1753
.   : milestone, 1728,
appsec_no_iast (1.72 ms) : 1696, 1744
.   : milestone, 1720,
iast (1.488 ms) : 1465, 1510
.   : milestone, 1488,
profiling (1.5 ms) : 1476, 1525
.   : milestone, 1500,
tracing (1.479 ms) : 1454, 1505
.   : milestone, 1479,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.352 ms [1.332 ms, 1.371 ms] -
appsec 1.749 ms [1.724 ms, 1.773 ms] 396.943 µs (29.4%)
appsec_no_iast 1.713 ms [1.688 ms, 1.738 ms] 361.428 µs (26.7%)
iast 1.498 ms [1.476 ms, 1.52 ms] 146.663 µs (10.9%)
profiling 1.5 ms [1.475 ms, 1.524 ms] 147.958 µs (10.9%)
tracing 1.474 ms [1.45 ms, 1.499 ms] 122.742 µs (9.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.341 ms [1.323 ms, 1.36 ms] -
appsec 1.728 ms [1.702 ms, 1.753 ms] 386.486 µs (28.8%)
appsec_no_iast 1.72 ms [1.696 ms, 1.744 ms] 378.832 µs (28.2%)
iast 1.488 ms [1.465 ms, 1.51 ms] 146.296 µs (10.9%)
profiling 1.5 ms [1.476 ms, 1.525 ms] 159.028 µs (11.9%)
tracing 1.479 ms [1.454 ms, 1.505 ms] 138.061 µs (10.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master paul.laffon/increase-spark-listener-max-event-size
git_commit_date 1714584487 1714640286
git_commit_sha 5c8d55c dbf06bb
release_version 1.34.0-SNAPSHOT~5c8d55ca87 1.34.0-SNAPSHOT~dbf06bb509
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1714643085 1714643085
ci_job_id 502347398 502347398
ci_pipeline_id 33413260 33413260
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~dbf06bb509, baseline=1.34.0-SNAPSHOT~5c8d55ca87
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.431 s) : 15431000, 15431000
.   : milestone, 15431000,
appsec (15.249 s) : 15249000, 15249000
.   : milestone, 15249000,
iast (18.836 s) : 18836000, 18836000
.   : milestone, 18836000,
iast_GLOBAL (17.867 s) : 17867000, 17867000
.   : milestone, 17867000,
profiling (15.106 s) : 15106000, 15106000
.   : milestone, 15106000,
tracing (14.856 s) : 14856000, 14856000
.   : milestone, 14856000,
section candidate
no_agent (15.042 s) : 15042000, 15042000
.   : milestone, 15042000,
appsec (14.981 s) : 14981000, 14981000
.   : milestone, 14981000,
iast (18.651 s) : 18651000, 18651000
.   : milestone, 18651000,
iast_GLOBAL (17.918 s) : 17918000, 17918000
.   : milestone, 17918000,
profiling (14.88 s) : 14880000, 14880000
.   : milestone, 14880000,
tracing (14.886 s) : 14886000, 14886000
.   : milestone, 14886000,
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.431 s [15.431 s, 15.431 s] -
appsec 15.249 s [15.249 s, 15.249 s] -182.0 ms (-1.2%)
iast 18.836 s [18.836 s, 18.836 s] 3.405 s (22.1%)
iast_GLOBAL 17.867 s [17.867 s, 17.867 s] 2.436 s (15.8%)
profiling 15.106 s [15.106 s, 15.106 s] -325.0 ms (-2.1%)
tracing 14.856 s [14.856 s, 14.856 s] -575.0 ms (-3.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.042 s [15.042 s, 15.042 s] -
appsec 14.981 s [14.981 s, 14.981 s] -61.0 ms (-0.4%)
iast 18.651 s [18.651 s, 18.651 s] 3.609 s (24.0%)
iast_GLOBAL 17.918 s [17.918 s, 17.918 s] 2.876 s (19.1%)
profiling 14.88 s [14.88 s, 14.88 s] -162.0 ms (-1.1%)
tracing 14.886 s [14.886 s, 14.886 s] -156.0 ms (-1.0%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~dbf06bb509, baseline=1.34.0-SNAPSHOT~5c8d55ca87
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.467 ms) : 1455, 1478
.   : milestone, 1467,
appsec (2.209 ms) : 2175, 2243
.   : milestone, 2209,
iast (1.887 ms) : 1852, 1923
.   : milestone, 1887,
iast_GLOBAL (1.923 ms) : 1888, 1958
.   : milestone, 1923,
profiling (1.848 ms) : 1815, 1882
.   : milestone, 1848,
tracing (1.844 ms) : 1811, 1877
.   : milestone, 1844,
section candidate
no_agent (1.465 ms) : 1454, 1477
.   : milestone, 1465,
appsec (2.207 ms) : 2173, 2241
.   : milestone, 2207,
iast (1.89 ms) : 1854, 1925
.   : milestone, 1890,
iast_GLOBAL (1.93 ms) : 1895, 1966
.   : milestone, 1930,
profiling (1.86 ms) : 1828, 1893
.   : milestone, 1860,
tracing (1.831 ms) : 1799, 1862
.   : milestone, 1831,
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.467 ms [1.455 ms, 1.478 ms] -
appsec 2.209 ms [2.175 ms, 2.243 ms] 742.299 µs (50.6%)
iast 1.887 ms [1.852 ms, 1.923 ms] 420.477 µs (28.7%)
iast_GLOBAL 1.923 ms [1.888 ms, 1.958 ms] 456.239 µs (31.1%)
profiling 1.848 ms [1.815 ms, 1.882 ms] 381.36 µs (26.0%)
tracing 1.844 ms [1.811 ms, 1.877 ms] 377.188 µs (25.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.465 ms [1.454 ms, 1.477 ms] -
appsec 2.207 ms [2.173 ms, 2.241 ms] 741.736 µs (50.6%)
iast 1.89 ms [1.854 ms, 1.925 ms] 424.435 µs (29.0%)
iast_GLOBAL 1.93 ms [1.895 ms, 1.966 ms] 464.952 µs (31.7%)
profiling 1.86 ms [1.828 ms, 1.893 ms] 395.005 µs (27.0%)
tracing 1.831 ms [1.799 ms, 1.862 ms] 365.128 µs (24.9%)

@paul-laffon-dd paul-laffon-dd marked this pull request as ready for review May 2, 2024 11:35
@paul-laffon-dd paul-laffon-dd requested a review from a team as a code owner May 2, 2024 11:35
@paul-laffon-dd paul-laffon-dd merged commit 20e644b into master May 3, 2024
82 checks passed
@paul-laffon-dd paul-laffon-dd deleted the paul.laffon/increase-spark-listener-max-event-size branch May 3, 2024 15:08
@github-actions github-actions bot added this to the 1.34.0 milestone May 3, 2024
cecile75 pushed a commit that referenced this pull request May 6, 2024
@paul-laffon-dd
Increase spark listener max collection size (#6979)
f479a01 
Increase the maximum number of retained events for the injected spark listener. Big spark clusters are reaching this limit
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiliangzhou yiliangzhou yiliangzhou approved these changes

Assignees
No one assigned
Labels
inst: apache spark Apache Spark instrumentation
Projects
None yet
Milestone
1.34.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@paul-laffon-dd @yiliangzhou