Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add logs on datadog spark listener lifecycle #6943

Merged
merged 1 commit into from
Apr 30, 2024
Merged

Add logs on datadog spark listener lifecycle #6943

merged 1 commit into from
Apr 30, 2024

Conversation

paul-laffon-dd
Copy link
Contributor

@paul-laffon-dd paul-laffon-dd commented Apr 25, 2024 •
edited
Loading

What Does This Do

Add logs for creation/completion of datadog spark listener

Motivation

Easier troubleshooting to validate that everything is running as expected

Additional Notes

@paul-laffon-dd paul-laffon-dd force-pushed the paul.laffon/spark-application-end-logging branch from eb1627b to 19d640a Compare April 25, 2024 08:48
@pr-commenter
Copy link

pr-commenter bot commented Apr 25, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master paul.laffon/spark-application-end-logging
git_commit_date 1714466677 1714467895
git_commit_sha d3eae46 7670655
release_version 1.34.0-SNAPSHOT~d3eae46967 1.34.0-SNAPSHOT~7670655c26
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1714470588 1714470588
ci_job_id 500535400 500535400
ci_pipeline_id 33233351 33233351
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.34.0-SNAPSHOT~7670655c26, baseline=1.34.0-SNAPSHOT~d3eae46967

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.076 s) : 0, 1076173
Total [baseline] (8.572 s) : 0, 8571519
Agent [candidate] (1.077 s) : 0, 1077183
Total [candidate] (8.548 s) : 0, 8548116
section iast
Agent [baseline] (1.209 s) : 0, 1209100
Total [baseline] (9.044 s) : 0, 9043954
Agent [candidate] (1.207 s) : 0, 1207479
Total [candidate] (9.011 s) : 0, 9011028
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.199 s) : 0, 1199215
Total [baseline] (8.944 s) : 0, 8944078
Agent [candidate] (1.201 s) : 0, 1201091
Total [candidate] (8.992 s) : 0, 8992233
section iast_TELEMETRY_OFF
Agent [baseline] (1.197 s) : 0, 1197035
Total [baseline] (8.994 s) : 0, 8993824
Agent [candidate] (1.209 s) : 0, 1208943
Total [candidate] (9.016 s) : 0, 9016088
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.076 s -
Agent iast 1.209 s 132.927 ms (12.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.199 s 123.042 ms (11.4%)
Agent iast_TELEMETRY_OFF 1.197 s 120.862 ms (11.2%)
Total tracing 8.572 s -
Total iast 9.044 s 472.436 ms (5.5%)
Total iast_HARDCODED_SECRET_DISABLED 8.944 s 372.56 ms (4.3%)
Total iast_TELEMETRY_OFF 8.994 s 422.306 ms (4.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent iast 1.207 s 130.296 ms (12.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.201 s 123.908 ms (11.5%)
Agent iast_TELEMETRY_OFF 1.209 s 131.76 ms (12.2%)
Total tracing 8.548 s -
Total iast 9.011 s 462.913 ms (5.4%)
Total iast_HARDCODED_SECRET_DISABLED 8.992 s 444.117 ms (5.2%)
Total iast_TELEMETRY_OFF 9.016 s 467.973 ms (5.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.34.0-SNAPSHOT~7670655c26, baseline=1.34.0-SNAPSHOT~d3eae46967

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (674.208 ms) : 0, 674208
BytebuddyAgent [candidate] (674.697 ms) : 0, 674697
GlobalTracer [baseline] (310.052 ms) : 0, 310052
GlobalTracer [candidate] (310.35 ms) : 0, 310350
AppSec [baseline] (49.288 ms) : 0, 49288
AppSec [candidate] (49.454 ms) : 0, 49454
Remote Config [baseline] (657.773 µs) : 0, 658
Remote Config [candidate] (661.572 µs) : 0, 662
Telemetry [baseline] (7.548 ms) : 0, 7548
Telemetry [candidate] (7.62 ms) : 0, 7620
section iast
BytebuddyAgent [baseline] (800.822 ms) : 0, 800822
BytebuddyAgent [candidate] (799.979 ms) : 0, 799979
GlobalTracer [baseline] (290.281 ms) : 0, 290281
GlobalTracer [candidate] (290.534 ms) : 0, 290534
AppSec [baseline] (51.49 ms) : 0, 51490
AppSec [candidate] (50.743 ms) : 0, 50743
Remote Config [baseline] (1.319 ms) : 0, 1319
Remote Config [candidate] (620.13 µs) : 0, 620
Telemetry [baseline] (6.702 ms) : 0, 6702
Telemetry [candidate] (6.734 ms) : 0, 6734
IAST [baseline] (23.873 ms) : 0, 23873
IAST [candidate] (24.32 ms) : 0, 24320
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (794.259 ms) : 0, 794259
BytebuddyAgent [candidate] (794.373 ms) : 0, 794373
GlobalTracer [baseline] (288.262 ms) : 0, 288262
GlobalTracer [candidate] (288.812 ms) : 0, 288812
AppSec [baseline] (50.924 ms) : 0, 50924
AppSec [candidate] (52.451 ms) : 0, 52451
Remote Config [baseline] (575.276 µs) : 0, 575
Remote Config [candidate] (1.532 ms) : 0, 1532
Telemetry [baseline] (6.669 ms) : 0, 6669
Telemetry [candidate] (6.776 ms) : 0, 6776
IAST [baseline] (24.186 ms) : 0, 24186
IAST [candidate] (22.755 ms) : 0, 22755
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (792.297 ms) : 0, 792297
BytebuddyAgent [candidate] (798.972 ms) : 0, 798972
GlobalTracer [baseline] (288.655 ms) : 0, 288655
GlobalTracer [candidate] (291.966 ms) : 0, 291966
AppSec [baseline] (50.922 ms) : 0, 50922
AppSec [candidate] (49.617 ms) : 0, 49617
Remote Config [baseline] (582.917 µs) : 0, 583
Remote Config [candidate] (619.069 µs) : 0, 619
Telemetry [baseline] (7.364 ms) : 0, 7364
Telemetry [candidate] (7.394 ms) : 0, 7394
IAST [baseline] (22.857 ms) : 0, 22857
IAST [candidate] (25.782 ms) : 0, 25782
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.34.0-SNAPSHOT~7670655c26, baseline=1.34.0-SNAPSHOT~d3eae46967

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.075 s) : 0, 1074546
Total [baseline] (10.428 s) : 0, 10427961
Agent [candidate] (1.077 s) : 0, 1076816
Total [candidate] (10.383 s) : 0, 10382734
section appsec
Agent [baseline] (1.197 s) : 0, 1196528
Total [baseline] (10.464 s) : 0, 10464209
Agent [candidate] (1.194 s) : 0, 1193582
Total [candidate] (10.548 s) : 0, 10547723
section iast
Agent [baseline] (1.215 s) : 0, 1215287
Total [baseline] (10.727 s) : 0, 10727470
Agent [candidate] (1.209 s) : 0, 1209122
Total [candidate] (10.72 s) : 0, 10720344
section profiling
Agent [baseline] (1.267 s) : 0, 1266677
Total [baseline] (10.601 s) : 0, 10601376
Agent [candidate] (1.278 s) : 0, 1278466
Total [candidate] (10.589 s) : 0, 10589355
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.075 s -
Agent appsec 1.197 s 121.982 ms (11.4%)
Agent iast 1.215 s 140.741 ms (13.1%)
Agent profiling 1.267 s 192.131 ms (17.9%)
Total tracing 10.428 s -
Total appsec 10.464 s 36.248 ms (0.3%)
Total iast 10.727 s 299.509 ms (2.9%)
Total profiling 10.601 s 173.414 ms (1.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent appsec 1.194 s 116.767 ms (10.8%)
Agent iast 1.209 s 132.306 ms (12.3%)
Agent profiling 1.278 s 201.651 ms (18.7%)
Total tracing 10.383 s -
Total appsec 10.548 s 164.989 ms (1.6%)
Total iast 10.72 s 337.61 ms (3.3%)
Total profiling 10.589 s 206.621 ms (2.0%) 
gantt
    title petclinic - break down per module: candidate=1.34.0-SNAPSHOT~7670655c26, baseline=1.34.0-SNAPSHOT~d3eae46967

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (672.887 ms) : 0, 672887
BytebuddyAgent [candidate] (674.691 ms) : 0, 674691
GlobalTracer [baseline] (309.503 ms) : 0, 309503
GlobalTracer [candidate] (310.008 ms) : 0, 310008
AppSec [baseline] (49.499 ms) : 0, 49499
AppSec [candidate] (49.263 ms) : 0, 49263
Remote Config [baseline] (678.301 µs) : 0, 678
Remote Config [candidate] (658.748 µs) : 0, 659
Telemetry [baseline] (7.656 ms) : 0, 7656
Telemetry [candidate] (7.667 ms) : 0, 7667
section appsec
BytebuddyAgent [baseline] (700.352 ms) : 0, 700352
BytebuddyAgent [candidate] (698.009 ms) : 0, 698009
GlobalTracer [baseline] (293.228 ms) : 0, 293228
GlobalTracer [candidate] (292.712 ms) : 0, 292712
AppSec [baseline] (150.122 ms) : 0, 150122
AppSec [candidate] (149.414 ms) : 0, 149414
Remote Config [baseline] (629.606 µs) : 0, 630
Remote Config [candidate] (619.824 µs) : 0, 620
Telemetry [baseline] (7.85 ms) : 0, 7850
Telemetry [candidate] (8.717 ms) : 0, 8717
IAST [baseline] (19.546 ms) : 0, 19546
IAST [candidate] (19.193 ms) : 0, 19193
section iast
BytebuddyAgent [baseline] (805.326 ms) : 0, 805326
BytebuddyAgent [candidate] (801.018 ms) : 0, 801018
GlobalTracer [baseline] (291.697 ms) : 0, 291697
GlobalTracer [candidate] (290.51 ms) : 0, 290510
AppSec [baseline] (51.179 ms) : 0, 51179
AppSec [candidate] (52.568 ms) : 0, 52568
Remote Config [baseline] (573.061 µs) : 0, 573
Remote Config [candidate] (1.991 ms) : 0, 1991
Telemetry [baseline] (6.591 ms) : 0, 6591
Telemetry [candidate] (6.685 ms) : 0, 6685
IAST [baseline] (25.174 ms) : 0, 25174
IAST [candidate] (21.708 ms) : 0, 21708
section profiling
BytebuddyAgent [baseline] (677.19 ms) : 0, 677190
BytebuddyAgent [candidate] (682.489 ms) : 0, 682489
GlobalTracer [baseline] (380.185 ms) : 0, 380185
GlobalTracer [candidate] (383.348 ms) : 0, 383348
AppSec [baseline] (49.669 ms) : 0, 49669
AppSec [candidate] (50.571 ms) : 0, 50571
Remote Config [baseline] (726.515 µs) : 0, 727
Remote Config [candidate] (731.149 µs) : 0, 731
Telemetry [baseline] (7.526 ms) : 0, 7526
Telemetry [candidate] (7.53 ms) : 0, 7530
ProfilingAgent [baseline] (95.224 ms) : 0, 95224
ProfilingAgent [candidate] (96.85 ms) : 0, 96850
Profiling [baseline] (95.248 ms) : 0, 95248
Profiling [candidate] (96.874 ms) : 0, 96874
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-04-30T09:22:29 2024-04-30T09:29:19
git_branch master paul.laffon/spark-application-end-logging
git_commit_date 1714466677 1714467895
git_commit_sha d3eae46 7670655
release_version 1.34.0-SNAPSHOT~d3eae46967 1.34.0-SNAPSHOT~7670655c26
start_time 2024-04-30T09:22:16 2024-04-30T09:29:06
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1714469704 1714469704
ci_job_id 500535401 500535401
ci_pipeline_id 33233351 33233351
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~7670655c26, baseline=1.34.0-SNAPSHOT~d3eae46967
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.353 ms) : 1334, 1372
.   : milestone, 1353,
appsec (1.729 ms) : 1704, 1754
.   : milestone, 1729,
appsec_no_iast (1.733 ms) : 1708, 1759
.   : milestone, 1733,
iast (1.503 ms) : 1480, 1525
.   : milestone, 1503,
profiling (1.49 ms) : 1465, 1515
.   : milestone, 1490,
tracing (1.496 ms) : 1472, 1521
.   : milestone, 1496,
section candidate
no_agent (1.337 ms) : 1317, 1358
.   : milestone, 1337,
appsec (1.721 ms) : 1697, 1746
.   : milestone, 1721,
appsec_no_iast (1.72 ms) : 1696, 1744
.   : milestone, 1720,
iast (1.485 ms) : 1462, 1507
.   : milestone, 1485,
profiling (1.506 ms) : 1481, 1531
.   : milestone, 1506,
tracing (1.511 ms) : 1486, 1536
.   : milestone, 1511,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.334 ms, 1.372 ms] -
appsec 1.729 ms [1.704 ms, 1.754 ms] 375.897 µs (27.8%)
appsec_no_iast 1.733 ms [1.708 ms, 1.759 ms] 380.334 µs (28.1%)
iast 1.503 ms [1.48 ms, 1.525 ms] 149.685 µs (11.1%)
profiling 1.49 ms [1.465 ms, 1.515 ms] 136.761 µs (10.1%)
tracing 1.496 ms [1.472 ms, 1.521 ms] 143.403 µs (10.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.337 ms [1.317 ms, 1.358 ms] -
appsec 1.721 ms [1.697 ms, 1.746 ms] 384.269 µs (28.7%)
appsec_no_iast 1.72 ms [1.696 ms, 1.744 ms] 382.418 µs (28.6%)
iast 1.485 ms [1.462 ms, 1.507 ms] 147.599 µs (11.0%)
profiling 1.506 ms [1.481 ms, 1.531 ms] 168.891 µs (12.6%)
tracing 1.511 ms [1.486 ms, 1.536 ms] 174.011 µs (13.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~7670655c26, baseline=1.34.0-SNAPSHOT~d3eae46967
    dateFormat X
    axisFormat %s
section baseline
no_agent (368.42 µs) : 349, 388
.   : milestone, 368,
iast (477.78 µs) : 457, 498
.   : milestone, 478,
iast_FULL (549.223 µs) : 528, 571
.   : milestone, 549,
iast_GLOBAL (505.557 µs) : 484, 527
.   : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (482.173 µs) : 461, 503
.   : milestone, 482,
iast_INACTIVE (454.935 µs) : 434, 476
.   : milestone, 455,
iast_TELEMETRY_OFF (468.183 µs) : 447, 489
.   : milestone, 468,
tracing (446.696 µs) : 427, 467
.   : milestone, 447,
section candidate
no_agent (369.413 µs) : 350, 389
.   : milestone, 369,
iast (480.743 µs) : 460, 501
.   : milestone, 481,
iast_FULL (548.806 µs) : 527, 570
.   : milestone, 549,
iast_GLOBAL (505.543 µs) : 484, 527
.   : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (479.128 µs) : 459, 500
.   : milestone, 479,
iast_INACTIVE (454.242 µs) : 433, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (473.8 µs) : 453, 495
.   : milestone, 474,
tracing (450.703 µs) : 430, 472
.   : milestone, 451,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 368.42 µs [349.018 µs, 387.822 µs] -
iast 477.78 µs [457.109 µs, 498.45 µs] 109.36 µs (29.7%)
iast_FULL 549.223 µs [527.821 µs, 570.624 µs] 180.803 µs (49.1%)
iast_GLOBAL 505.557 µs [483.819 µs, 527.294 µs] 137.137 µs (37.2%)
iast_HARDCODED_SECRET_DISABLED 482.173 µs [461.414 µs, 502.931 µs] 113.753 µs (30.9%)
iast_INACTIVE 454.935 µs [433.993 µs, 475.877 µs] 86.515 µs (23.5%)
iast_TELEMETRY_OFF 468.183 µs [447.437 µs, 488.93 µs] 99.763 µs (27.1%)
tracing 446.696 µs [426.866 µs, 466.525 µs] 78.275 µs (21.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.413 µs [349.58 µs, 389.245 µs] -
iast 480.743 µs [460.073 µs, 501.414 µs] 111.331 µs (30.1%)
iast_FULL 548.806 µs [527.305 µs, 570.307 µs] 179.394 µs (48.6%)
iast_GLOBAL 505.543 µs [484.172 µs, 526.914 µs] 136.131 µs (36.9%)
iast_HARDCODED_SECRET_DISABLED 479.128 µs [458.627 µs, 499.629 µs] 109.715 µs (29.7%)
iast_INACTIVE 454.242 µs [433.406 µs, 475.079 µs] 84.83 µs (23.0%)
iast_TELEMETRY_OFF 473.8 µs [452.995 µs, 494.605 µs] 104.388 µs (28.3%)
tracing 450.703 µs [429.579 µs, 471.826 µs] 81.29 µs (22.0%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master paul.laffon/spark-application-end-logging
git_commit_date 1714466677 1714467895
git_commit_sha d3eae46 7670655
release_version 1.34.0-SNAPSHOT~d3eae46967 1.34.0-SNAPSHOT~7670655c26
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1714470246 1714470246
ci_job_id 500535402 500535402
ci_pipeline_id 33233351 33233351
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~7670655c26, baseline=1.34.0-SNAPSHOT~d3eae46967
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.46 ms) : 1449, 1472
.   : milestone, 1460,
appsec (2.213 ms) : 2180, 2246
.   : milestone, 2213,
iast (1.907 ms) : 1871, 1942
.   : milestone, 1907,
iast_GLOBAL (1.936 ms) : 1901, 1972
.   : milestone, 1936,
profiling (1.857 ms) : 1823, 1890
.   : milestone, 1857,
tracing (1.836 ms) : 1804, 1868
.   : milestone, 1836,
section candidate
no_agent (1.462 ms) : 1451, 1474
.   : milestone, 1462,
appsec (2.221 ms) : 2187, 2255
.   : milestone, 2221,
iast (1.9 ms) : 1864, 1936
.   : milestone, 1900,
iast_GLOBAL (1.927 ms) : 1891, 1962
.   : milestone, 1927,
profiling (1.843 ms) : 1809, 1876
.   : milestone, 1843,
tracing (1.833 ms) : 1801, 1864
.   : milestone, 1833,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.46 ms [1.449 ms, 1.472 ms] -
appsec 2.213 ms [2.18 ms, 2.246 ms] 752.82 µs (51.6%)
iast 1.907 ms [1.871 ms, 1.942 ms] 446.693 µs (30.6%)
iast_GLOBAL 1.936 ms [1.901 ms, 1.972 ms] 475.874 µs (32.6%)
profiling 1.857 ms [1.823 ms, 1.89 ms] 396.382 µs (27.1%)
tracing 1.836 ms [1.804 ms, 1.868 ms] 376.248 µs (25.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.462 ms [1.451 ms, 1.474 ms] -
appsec 2.221 ms [2.187 ms, 2.255 ms] 759.038 µs (51.9%)
iast 1.9 ms [1.864 ms, 1.936 ms] 437.68 µs (29.9%)
iast_GLOBAL 1.927 ms [1.891 ms, 1.962 ms] 464.418 µs (31.8%)
profiling 1.843 ms [1.809 ms, 1.876 ms] 380.627 µs (26.0%)
tracing 1.833 ms [1.801 ms, 1.864 ms] 370.455 µs (25.3%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~7670655c26, baseline=1.34.0-SNAPSHOT~d3eae46967
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.385 s) : 15385000, 15385000
.   : milestone, 15385000,
appsec (14.987 s) : 14987000, 14987000
.   : milestone, 14987000,
iast (18.554 s) : 18554000, 18554000
.   : milestone, 18554000,
iast_GLOBAL (17.746 s) : 17746000, 17746000
.   : milestone, 17746000,
profiling (15.252 s) : 15252000, 15252000
.   : milestone, 15252000,
tracing (15.121 s) : 15121000, 15121000
.   : milestone, 15121000,
section candidate
no_agent (15.302 s) : 15302000, 15302000
.   : milestone, 15302000,
appsec (15.176 s) : 15176000, 15176000
.   : milestone, 15176000,
iast (18.661 s) : 18661000, 18661000
.   : milestone, 18661000,
iast_GLOBAL (17.884 s) : 17884000, 17884000
.   : milestone, 17884000,
profiling (15.14 s) : 15140000, 15140000
.   : milestone, 15140000,
tracing (14.917 s) : 14917000, 14917000
.   : milestone, 14917000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.385 s [15.385 s, 15.385 s] -
appsec 14.987 s [14.987 s, 14.987 s] -398.0 ms (-2.6%)
iast 18.554 s [18.554 s, 18.554 s] 3.169 s (20.6%)
iast_GLOBAL 17.746 s [17.746 s, 17.746 s] 2.361 s (15.3%)
profiling 15.252 s [15.252 s, 15.252 s] -133.0 ms (-0.9%)
tracing 15.121 s [15.121 s, 15.121 s] -264.0 ms (-1.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.302 s [15.302 s, 15.302 s] -
appsec 15.176 s [15.176 s, 15.176 s] -126.0 ms (-0.8%)
iast 18.661 s [18.661 s, 18.661 s] 3.359 s (22.0%)
iast_GLOBAL 17.884 s [17.884 s, 17.884 s] 2.582 s (16.9%)
profiling 15.14 s [15.14 s, 15.14 s] -162.0 ms (-1.1%)
tracing 14.917 s [14.917 s, 14.917 s] -385.0 ms (-2.5%)

@paul-laffon-dd paul-laffon-dd changed the title Add logs on spark application end Add debug logs on spark application end Apr 26, 2024
@paul-laffon-dd paul-laffon-dd added the inst: apache spark Apache Spark instrumentation label Apr 26, 2024
@paul-laffon-dd paul-laffon-dd marked this pull request as ready for review April 26, 2024 12:15
@paul-laffon-dd paul-laffon-dd requested a review from a team as a code owner April 26, 2024 12:15
@paul-laffon-dd paul-laffon-dd force-pushed the paul.laffon/spark-application-end-logging branch from b2a1870 to 7670655 Compare April 30, 2024 09:05
@paul-laffon-dd paul-laffon-dd changed the title Add debug logs on spark application end Add logs on datadog spark listener lifecycle Apr 30, 2024
@yiliangzhou
Copy link
Contributor

LGTM

yiliangzhou
yiliangzhou approved these changes Apr 30, 2024
View reviewed changes
Copy link
Contributor

@yiliangzhou yiliangzhou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@paul-laffon-dd paul-laffon-dd merged commit 16c56bf into master Apr 30, 2024
80 checks passed
@paul-laffon-dd paul-laffon-dd deleted the paul.laffon/spark-application-end-logging branch April 30, 2024 17:56
@github-actions github-actions bot added this to the 1.34.0 milestone Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiliangzhou yiliangzhou yiliangzhou approved these changes

Assignees
No one assigned
Labels
inst: apache spark Apache Spark instrumentation
Projects
None yet
Milestone
1.34.0
Development

Successfully merging this pull request may close these issues.
2 participants
@paul-laffon-dd @yiliangzhou