Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Get access to tag values from the top of TraceSegments #6560

Merged
merged 1 commit into from
Jan 30, 2024
Merged

Get access to tag values from the top of TraceSegments #6560

merged 1 commit into from
Jan 30, 2024

Conversation

manuel-alvarez-alvarez
Copy link
Contributor

@manuel-alvarez-alvarez manuel-alvarez-alvarez commented Jan 26, 2024 •
edited
Loading

What Does This Do

Add getters to query for data from the top of the current trace segment.

Motivation

From IAST we want to be able to add vulnerabilities even when there is no active request (e.g. Kafka)

Additional Notes

Jira ticket: APPSEC-10440

This is required to make IAST work in Kafka consumers: #6465

@manuel-alvarez-alvarez manuel-alvarez-alvarez marked this pull request as ready for review January 26, 2024 09:14
@pr-commenter
Copy link

pr-commenter bot commented Jan 26, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master malvarez/trace-segment-getters
git_commit_date 1706562398 1706603639
git_commit_sha 281e492 5930032
release_version 1.29.0-SNAPSHOT~281e492170 1.29.0-SNAPSHOT~59300325de
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1706606728 1706606728
ci_job_id 421372849 421372849
ci_pipeline_id 27382836 27382836
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 8 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.29.0-SNAPSHOT~59300325de, baseline=1.29.0-SNAPSHOT~281e492170

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1056724
Total [baseline] (9.351 s) : 0, 9351485
Agent [candidate] (1.061 s) : 0, 1060902
Total [candidate] (9.4 s) : 0, 9400176
section appsec
Agent [baseline] (1.16 s) : 0, 1160337
Total [baseline] (9.481 s) : 0, 9480766
Agent [candidate] (1.158 s) : 0, 1157957
Total [candidate] (9.466 s) : 0, 9466054
section iast
Agent [baseline] (1.19 s) : 0, 1189582
Total [baseline] (9.715 s) : 0, 9715088
Agent [candidate] (1.181 s) : 0, 1181149
Total [candidate] (9.685 s) : 0, 9684773
section profiling
Agent [baseline] (1.285 s) : 0, 1285240
Total [baseline] (9.548 s) : 0, 9548236
Agent [candidate] (1.288 s) : 0, 1288188
Total [candidate] (9.623 s) : 0, 9622796
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.16 s 103.614 ms (9.8%)
Agent iast 1.19 s 132.858 ms (12.6%)
Agent profiling 1.285 s 228.516 ms (21.6%)
Total tracing 9.351 s -
Total appsec 9.481 s 129.281 ms (1.4%)
Total iast 9.715 s 363.603 ms (3.9%)
Total profiling 9.548 s 196.751 ms (2.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent appsec 1.158 s 97.056 ms (9.1%)
Agent iast 1.181 s 120.247 ms (11.3%)
Agent profiling 1.288 s 227.287 ms (21.4%)
Total tracing 9.4 s -
Total appsec 9.466 s 65.878 ms (0.7%)
Total iast 9.685 s 284.597 ms (3.0%)
Total profiling 9.623 s 222.62 ms (2.4%) 
gantt
    title petclinic - break down per module: candidate=1.29.0-SNAPSHOT~59300325de, baseline=1.29.0-SNAPSHOT~281e492170

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (665.329 ms) : 0, 665329
BytebuddyAgent [candidate] (667.78 ms) : 0, 667780
GlobalTracer [baseline] (296.783 ms) : 0, 296783
GlobalTracer [candidate] (297.895 ms) : 0, 297895
AppSec [baseline] (52.231 ms) : 0, 52231
AppSec [candidate] (52.6 ms) : 0, 52600
Remote Config [baseline] (675.423 µs) : 0, 675
Remote Config [candidate] (685.424 µs) : 0, 685
Telemetry [baseline] (7.461 ms) : 0, 7461
Telemetry [candidate] (7.532 ms) : 0, 7532
section appsec
BytebuddyAgent [baseline] (670.15 ms) : 0, 670150
BytebuddyAgent [candidate] (668.139 ms) : 0, 668139
GlobalTracer [baseline] (297.989 ms) : 0, 297989
GlobalTracer [candidate] (297.776 ms) : 0, 297776
AppSec [baseline] (150.28 ms) : 0, 150280
AppSec [candidate] (150.303 ms) : 0, 150303
Remote Config [baseline] (651.27 µs) : 0, 651
Remote Config [candidate] (652.422 µs) : 0, 652
Telemetry [baseline] (6.774 ms) : 0, 6774
Telemetry [candidate] (6.739 ms) : 0, 6739
section iast
BytebuddyAgent [baseline] (782.09 ms) : 0, 782090
BytebuddyAgent [candidate] (776.279 ms) : 0, 776279
GlobalTracer [baseline] (289.663 ms) : 0, 289663
GlobalTracer [candidate] (287.766 ms) : 0, 287766
AppSec [baseline] (53.986 ms) : 0, 53986
AppSec [candidate] (54.243 ms) : 0, 54243
Remote Config [baseline] (616.003 µs) : 0, 616
Remote Config [candidate] (619.779 µs) : 0, 620
Telemetry [baseline] (7.432 ms) : 0, 7432
Telemetry [candidate] (6.677 ms) : 0, 6677
IAST [baseline] (21.154 ms) : 0, 21154
IAST [candidate] (21.276 ms) : 0, 21276
section profiling
BytebuddyAgent [baseline] (664.938 ms) : 0, 664938
BytebuddyAgent [candidate] (666.256 ms) : 0, 666256
GlobalTracer [baseline] (380.231 ms) : 0, 380231
GlobalTracer [candidate] (381.288 ms) : 0, 381288
AppSec [baseline] (52.168 ms) : 0, 52168
AppSec [candidate] (52.501 ms) : 0, 52501
Remote Config [baseline] (662.043 µs) : 0, 662
Remote Config [candidate] (668.949 µs) : 0, 669
Telemetry [baseline] (7.44 ms) : 0, 7440
Telemetry [candidate] (8.076 ms) : 0, 8076
ProfilingAgent [baseline] (125.296 ms) : 0, 125296
ProfilingAgent [candidate] (124.794 ms) : 0, 124794
Profiling [baseline] (125.32 ms) : 0, 125320
Profiling [candidate] (124.818 ms) : 0, 124818
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-01-30T09:01:30 2024-01-30T09:20:24
git_branch master malvarez/trace-segment-getters
git_commit_date 1706562398 1706603639
git_commit_sha 281e492 5930032
release_version 1.29.0-SNAPSHOT~281e492170 1.29.0-SNAPSHOT~59300325de
start_time 2024-01-30T09:01:17 2024-01-30T09:20:11
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1706606728 1706606728
ci_job_id 421372849 421372849
ci_pipeline_id 27382836 27382836
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 16 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.29.0-SNAPSHOT~59300325de, baseline=1.29.0-SNAPSHOT~281e492170
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.346 ms) : 1327, 1365
.   : milestone, 1346,
appsec (1.757 ms) : 1731, 1783
.   : milestone, 1757,
iast (1.504 ms) : 1479, 1529
.   : milestone, 1504,
profiling (1.5 ms) : 1476, 1525
.   : milestone, 1500,
tracing (1.484 ms) : 1459, 1510
.   : milestone, 1484,
section candidate
no_agent (1.345 ms) : 1326, 1364
.   : milestone, 1345,
appsec (1.762 ms) : 1738, 1787
.   : milestone, 1762,
iast (1.523 ms) : 1498, 1547
.   : milestone, 1523,
profiling (1.511 ms) : 1486, 1536
.   : milestone, 1511,
tracing (1.487 ms) : 1462, 1513
.   : milestone, 1487,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.346 ms [1.327 ms, 1.365 ms] -
appsec 1.757 ms [1.731 ms, 1.783 ms] 411.534 µs (30.6%)
iast 1.504 ms [1.479 ms, 1.529 ms] 158.531 µs (11.8%)
profiling 1.5 ms [1.476 ms, 1.525 ms] 154.796 µs (11.5%)
tracing 1.484 ms [1.459 ms, 1.51 ms] 138.616 µs (10.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.345 ms [1.326 ms, 1.364 ms] -
appsec 1.762 ms [1.738 ms, 1.787 ms] 417.16 µs (31.0%)
iast 1.523 ms [1.498 ms, 1.547 ms] 177.372 µs (13.2%)
profiling 1.511 ms [1.486 ms, 1.536 ms] 166.2 µs (12.4%)
tracing 1.487 ms [1.462 ms, 1.513 ms] 142.117 µs (10.6%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.29.0-SNAPSHOT~59300325de, baseline=1.29.0-SNAPSHOT~281e492170
    dateFormat X
    axisFormat %s
section baseline
no_agent (363.333 µs) : 344, 383
.   : milestone, 363,
iast (467.843 µs) : 447, 489
.   : milestone, 468,
iast_FULL (527.82 µs) : 507, 548
.   : milestone, 528,
iast_GLOBAL (514.984 µs) : 492, 538
.   : milestone, 515,
iast_HARDCODED_SECRET_DISABLED (470.859 µs) : 450, 492
.   : milestone, 471,
iast_INACTIVE (439.556 µs) : 419, 460
.   : milestone, 440,
iast_TELEMETRY_OFF (466.535 µs) : 446, 487
.   : milestone, 467,
tracing (439.213 µs) : 419, 460
.   : milestone, 439,
section candidate
no_agent (365.764 µs) : 346, 386
.   : milestone, 366,
iast (468.048 µs) : 447, 489
.   : milestone, 468,
iast_FULL (532.513 µs) : 512, 553
.   : milestone, 533,
iast_GLOBAL (490.584 µs) : 469, 512
.   : milestone, 491,
iast_HARDCODED_SECRET_DISABLED (465.759 µs) : 445, 487
.   : milestone, 466,
iast_INACTIVE (440.278 µs) : 420, 461
.   : milestone, 440,
iast_TELEMETRY_OFF (457.565 µs) : 437, 478
.   : milestone, 458,
tracing (434.023 µs) : 414, 454
.   : milestone, 434,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 363.333 µs [343.618 µs, 383.049 µs] -
iast 467.843 µs [446.841 µs, 488.845 µs] 104.51 µs (28.8%)
iast_FULL 527.82 µs [507.182 µs, 548.457 µs] 164.486 µs (45.3%)
iast_GLOBAL 514.984 µs [491.962 µs, 538.006 µs] 151.65 µs (41.7%)
iast_HARDCODED_SECRET_DISABLED 470.859 µs [449.798 µs, 491.921 µs] 107.526 µs (29.6%)
iast_INACTIVE 439.556 µs [418.888 µs, 460.224 µs] 76.223 µs (21.0%)
iast_TELEMETRY_OFF 466.535 µs [445.991 µs, 487.078 µs] 103.201 µs (28.4%)
tracing 439.213 µs [418.521 µs, 459.904 µs] 75.879 µs (20.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.764 µs [345.564 µs, 385.965 µs] -
iast 468.048 µs [446.852 µs, 489.244 µs] 102.284 µs (28.0%)
iast_FULL 532.513 µs [511.886 µs, 553.14 µs] 166.749 µs (45.6%)
iast_GLOBAL 490.584 µs [469.303 µs, 511.865 µs] 124.82 µs (34.1%)
iast_HARDCODED_SECRET_DISABLED 465.759 µs [445.012 µs, 486.506 µs] 99.995 µs (27.3%)
iast_INACTIVE 440.278 µs [419.6 µs, 460.956 µs] 74.514 µs (20.4%)
iast_TELEMETRY_OFF 457.565 µs [437.265 µs, 477.864 µs] 91.8 µs (25.1%)
tracing 434.023 µs [413.734 µs, 454.313 µs] 68.259 µs (18.7%)

@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/trace-segment-getters branch 3 times, most recently from a397fb2 to 04cb62f Compare January 26, 2024 12:47
smola
smola reviewed Jan 26, 2024
View reviewed changes
Copy link
Member

@smola smola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me other than the log level I mentioned.

It'd be good for APM core to review TracerSegment/DDSpanContext changes.

dd-java-agent/agent-iast/src/main/java/com/datadog/iast/Reporter.java Outdated Show resolved Hide resolved
@smola smola added comp: asm iast Application Security Management (IAST) type: refactoring labels Jan 26, 2024
@manuel-alvarez-alvarez manuel-alvarez-alvarez merged commit fe7485c into master Jan 30, 2024
80 checks passed
@manuel-alvarez-alvarez manuel-alvarez-alvarez deleted the malvarez/trace-segment-getters branch January 30, 2024 10:01
@github-actions github-actions bot added this to the 1.29.0 milestone Jan 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jandro996 jandro996 jandro996 approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

@am312 am312 Awaiting requested review from am312 am312 is a code owner automatically assigned from DataDog/apm-java

@smola smola Awaiting requested review from smola

Assignees
No one assigned
Labels
comp: asm iast Application Security Management (IAST) type: refactoring
Projects
None yet
Milestone
1.29.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@manuel-alvarez-alvarez @smola @jandro996