Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support SSRF in Apache HttpClient V4 #6112

Merged
merged 27 commits into from
Dec 19, 2023

Conversation

jandro996
Copy link
Member

@jandro996 jandro996 commented Oct 30, 2023

What Does This Do

Add new IAST instrumentation for org.apache.http.client.HttpClient

Motivation

Add SSRF support for Apache HttpClient V4

Additional Notes

ApacheHttpClientInstrumentation

As org.apache.http.client.HttpClient is currently instrumented for tracing implementing Instrumenter.CanShortcutTypeMatching we will keep the same approach

Jira ticket: APPSEC-11925

@jandro996 jandro996 force-pushed the alejandro.gonzalez/Support_SSRF_in_Apache_HttpClient branch 2 times, most recently from ae66378 to 0cc36ee Compare October 30, 2023 09:31
@pr-commenter
Copy link

pr-commenter bot commented Oct 30, 2023

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alejandro.gonzalez/Support_SSRF_in_Apache_HttpClient
git_commit_date 1703003620 1703003907
git_commit_sha 030073d ff2b409
release_version 1.27.0-SNAPSHOT~030073d069 1.27.0-SNAPSHOT~ff2b409800
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1703006615 1703006615
ci_job_id 393223934 393223934
ci_pipeline_id 25352225 25352225
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 5 unstable metrics.

Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.27.0-SNAPSHOT~ff2b409800, baseline=1.27.0-SNAPSHOT~030073d069

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1047872
Total [baseline] (9.431 s) : 0, 9431134
Agent [candidate] (1.057 s) : 0, 1057390
Total [candidate] (9.392 s) : 0, 9391987
section appsec
Agent [baseline] (1.156 s) : 0, 1155830
Total [baseline] (9.477 s) : 0, 9477303
Agent [candidate] (1.149 s) : 0, 1149294
Total [candidate] (9.443 s) : 0, 9443099
section iast
Agent [baseline] (1.171 s) : 0, 1171190
Total [baseline] (9.671 s) : 0, 9670768
Agent [candidate] (1.177 s) : 0, 1177366
Total [candidate] (9.67 s) : 0, 9669554
section profiling
Agent [baseline] (1.245 s) : 0, 1245136
Total [baseline] (9.651 s) : 0, 9650532
Agent [candidate] (1.245 s) : 0, 1245181
Total [candidate] (9.62 s) : 0, 9620030
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.048 s -
Agent appsec 1.156 s 107.958 ms (10.3%)
Agent iast 1.171 s 123.318 ms (11.8%)
Agent profiling 1.245 s 197.263 ms (18.8%)
Total tracing 9.431 s -
Total appsec 9.477 s 46.17 ms (0.5%)
Total iast 9.671 s 239.634 ms (2.5%)
Total profiling 9.651 s 219.398 ms (2.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.149 s 91.905 ms (8.7%)
Agent iast 1.177 s 119.976 ms (11.3%)
Agent profiling 1.245 s 187.791 ms (17.8%)
Total tracing 9.392 s -
Total appsec 9.443 s 51.112 ms (0.5%)
Total iast 9.67 s 277.567 ms (3.0%)
Total profiling 9.62 s 228.043 ms (2.4%)
gantt
    title petclinic - break down per module: candidate=1.27.0-SNAPSHOT~ff2b409800, baseline=1.27.0-SNAPSHOT~030073d069

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (647.893 ms) : 0, 647893
BytebuddyAgent [candidate] (653.983 ms) : 0, 653983
GlobalTracer [baseline] (306.905 ms) : 0, 306905
GlobalTracer [candidate] (309.819 ms) : 0, 309819
AppSec [baseline] (50.821 ms) : 0, 50821
AppSec [candidate] (51.071 ms) : 0, 51071
Remote Config [baseline] (674.732 µs) : 0, 675
Remote Config [candidate] (678.38 µs) : 0, 678
Telemetry [baseline] (7.174 ms) : 0, 7174
Telemetry [candidate] (7.326 ms) : 0, 7326
section appsec
BytebuddyAgent [baseline] (653.818 ms) : 0, 653818
BytebuddyAgent [candidate] (650.456 ms) : 0, 650456
GlobalTracer [baseline] (309.815 ms) : 0, 309815
GlobalTracer [candidate] (307.984 ms) : 0, 307984
AppSec [baseline] (149.951 ms) : 0, 149951
AppSec [candidate] (148.99 ms) : 0, 148990
Remote Config [baseline] (655.185 µs) : 0, 655
Remote Config [candidate] (641.483 µs) : 0, 641
Telemetry [baseline] (6.948 ms) : 0, 6948
Telemetry [candidate] (6.852 ms) : 0, 6852
section iast
BytebuddyAgent [baseline] (770.949 ms) : 0, 770949
BytebuddyAgent [candidate] (776.743 ms) : 0, 776743
GlobalTracer [baseline] (287.25 ms) : 0, 287250
GlobalTracer [candidate] (288.43 ms) : 0, 288430
AppSec [baseline] (49.153 ms) : 0, 49153
AppSec [candidate] (49.479 ms) : 0, 49479
IAST [baseline] (20.905 ms) : 0, 20905
IAST [candidate] (20.108 ms) : 0, 20108
Remote Config [baseline] (617.083 µs) : 0, 617
Remote Config [candidate] (597.445 µs) : 0, 597
Telemetry [baseline] (7.916 ms) : 0, 7916
Telemetry [candidate] (7.445 ms) : 0, 7445
section profiling
BytebuddyAgent [baseline] (658.677 ms) : 0, 658677
BytebuddyAgent [candidate] (659.024 ms) : 0, 659024
GlobalTracer [baseline] (377.851 ms) : 0, 377851
GlobalTracer [candidate] (377.929 ms) : 0, 377929
AppSec [baseline] (51.47 ms) : 0, 51470
AppSec [candidate] (51.268 ms) : 0, 51268
Remote Config [baseline] (1.008 ms) : 0, 1008
Remote Config [candidate] (986.018 µs) : 0, 986
Telemetry [baseline] (7.25 ms) : 0, 7250
Telemetry [candidate] (7.163 ms) : 0, 7163
ProfilingAgent [baseline] (94.632 ms) : 0, 94632
ProfilingAgent [candidate] (94.571 ms) : 0, 94571
Profiling [baseline] (94.657 ms) : 0, 94657
Profiling [candidate] (94.596 ms) : 0, 94596
Loading
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.27.0-SNAPSHOT~ff2b409800, baseline=1.27.0-SNAPSHOT~030073d069

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.056 s) : 0, 1056200
Total [baseline] (8.726 s) : 0, 8725844
Agent [candidate] (1.057 s) : 0, 1056551
Total [candidate] (8.77 s) : 0, 8769755
section iast
Agent [baseline] (1.166 s) : 0, 1166182
Total [baseline] (9.251 s) : 0, 9250929
Agent [candidate] (1.166 s) : 0, 1165776
Total [candidate] (9.262 s) : 0, 9262221
section iast_TELEMETRY_OFF
Agent [baseline] (1.159 s) : 0, 1158689
Total [baseline] (9.263 s) : 0, 9262562
Agent [candidate] (1.161 s) : 0, 1161361
Total [candidate] (9.278 s) : 0, 9277582
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.056 s -
Agent iast 1.166 s 109.982 ms (10.4%)
Agent iast_TELEMETRY_OFF 1.159 s 102.489 ms (9.7%)
Total tracing 8.726 s -
Total iast 9.251 s 525.085 ms (6.0%)
Total iast_TELEMETRY_OFF 9.263 s 536.718 ms (6.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent iast 1.166 s 109.226 ms (10.3%)
Agent iast_TELEMETRY_OFF 1.161 s 104.811 ms (9.9%)
Total tracing 8.77 s -
Total iast 9.262 s 492.466 ms (5.6%)
Total iast_TELEMETRY_OFF 9.278 s 507.828 ms (5.8%)
gantt
    title insecure-bank - break down per module: candidate=1.27.0-SNAPSHOT~ff2b409800, baseline=1.27.0-SNAPSHOT~030073d069

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (653.071 ms) : 0, 653071
BytebuddyAgent [candidate] (653.584 ms) : 0, 653584
GlobalTracer [baseline] (309.625 ms) : 0, 309625
GlobalTracer [candidate] (309.265 ms) : 0, 309265
AppSec [baseline] (51.06 ms) : 0, 51060
AppSec [candidate] (51.273 ms) : 0, 51273
Remote Config [baseline] (673.045 µs) : 0, 673
Remote Config [candidate] (670.099 µs) : 0, 670
Telemetry [baseline] (7.268 ms) : 0, 7268
Telemetry [candidate] (7.263 ms) : 0, 7263
section iast
BytebuddyAgent [baseline] (768.742 ms) : 0, 768742
BytebuddyAgent [candidate] (768.527 ms) : 0, 768527
GlobalTracer [baseline] (285.369 ms) : 0, 285369
GlobalTracer [candidate] (284.948 ms) : 0, 284948
AppSec [baseline] (48.738 ms) : 0, 48738
AppSec [candidate] (49.022 ms) : 0, 49022
IAST [baseline] (22.001 ms) : 0, 22001
IAST [candidate] (21.909 ms) : 0, 21909
Remote Config [baseline] (596.384 µs) : 0, 596
Remote Config [candidate] (673.311 µs) : 0, 673
Telemetry [baseline] (6.42 ms) : 0, 6420
Telemetry [candidate] (6.494 ms) : 0, 6494
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (761.743 ms) : 0, 761743
BytebuddyAgent [candidate] (763.501 ms) : 0, 763501
GlobalTracer [baseline] (286.254 ms) : 0, 286254
GlobalTracer [candidate] (286.472 ms) : 0, 286472
AppSec [baseline] (49.164 ms) : 0, 49164
AppSec [candidate] (49.003 ms) : 0, 49003
IAST [baseline] (19.121 ms) : 0, 19121
IAST [candidate] (20.021 ms) : 0, 20021
Remote Config [baseline] (618.957 µs) : 0, 619
Remote Config [candidate] (615.558 µs) : 0, 616
Telemetry [baseline] (7.408 ms) : 0, 7408
Telemetry [candidate] (7.281 ms) : 0, 7281
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2023-12-19T17:02:45 2023-12-19T17:19:21
git_branch master alejandro.gonzalez/Support_SSRF_in_Apache_HttpClient
git_commit_date 1703003620 1703003907
git_commit_sha 030073d ff2b409
release_version 1.27.0-SNAPSHOT~030073d069 1.27.0-SNAPSHOT~ff2b409800
start_time 2023-12-19T17:02:32 2023-12-19T17:19:08
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1703006615 1703006615
ci_job_id 393223934 393223934
ci_pipeline_id 25352225 25352225
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 13 unstable metrics.

Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.27.0-SNAPSHOT~ff2b409800, baseline=1.27.0-SNAPSHOT~030073d069
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.359 ms) : 1340, 1378
.   : milestone, 1359,
appsec (1.78 ms) : 1755, 1805
.   : milestone, 1780,
iast (1.507 ms) : 1482, 1531
.   : milestone, 1507,
profiling (1.567 ms) : 1540, 1593
.   : milestone, 1567,
tracing (1.507 ms) : 1482, 1533
.   : milestone, 1507,
section candidate
no_agent (1.352 ms) : 1333, 1371
.   : milestone, 1352,
appsec (1.757 ms) : 1732, 1782
.   : milestone, 1757,
iast (1.513 ms) : 1488, 1538
.   : milestone, 1513,
profiling (1.534 ms) : 1509, 1559
.   : milestone, 1534,
tracing (1.516 ms) : 1492, 1541
.   : milestone, 1516,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.359 ms [1.34 ms, 1.378 ms] -
appsec 1.78 ms [1.755 ms, 1.805 ms] 420.734 µs (31.0%)
iast 1.507 ms [1.482 ms, 1.531 ms] 147.475 µs (10.8%)
profiling 1.567 ms [1.54 ms, 1.593 ms] 207.248 µs (15.2%)
tracing 1.507 ms [1.482 ms, 1.533 ms] 147.963 µs (10.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.352 ms [1.333 ms, 1.371 ms] -
appsec 1.757 ms [1.732 ms, 1.782 ms] 404.842 µs (29.9%)
iast 1.513 ms [1.488 ms, 1.538 ms] 161.046 µs (11.9%)
profiling 1.534 ms [1.509 ms, 1.559 ms] 181.859 µs (13.5%)
tracing 1.516 ms [1.492 ms, 1.541 ms] 164.408 µs (12.2%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.27.0-SNAPSHOT~ff2b409800, baseline=1.27.0-SNAPSHOT~030073d069
    dateFormat X
    axisFormat %s
section baseline
no_agent (364.123 µs) : 345, 384
.   : milestone, 364,
iast (475.43 µs) : 455, 496
.   : milestone, 475,
iast_FULL (538.244 µs) : 518, 559
.   : milestone, 538,
iast_INACTIVE (446.526 µs) : 426, 467
.   : milestone, 447,
iast_TELEMETRY_OFF (469.32 µs) : 449, 490
.   : milestone, 469,
tracing (438.848 µs) : 418, 460
.   : milestone, 439,
section candidate
no_agent (375.096 µs) : 355, 395
.   : milestone, 375,
iast (472.485 µs) : 452, 493
.   : milestone, 472,
iast_FULL (531.447 µs) : 511, 552
.   : milestone, 531,
iast_INACTIVE (457.889 µs) : 437, 479
.   : milestone, 458,
iast_TELEMETRY_OFF (468.475 µs) : 448, 489
.   : milestone, 468,
tracing (444.256 µs) : 423, 465
.   : milestone, 444,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.123 µs [344.59 µs, 383.657 µs] -
iast 475.43 µs [454.856 µs, 496.005 µs] 111.307 µs (30.6%)
iast_FULL 538.244 µs [517.826 µs, 558.662 µs] 174.121 µs (47.8%)
iast_INACTIVE 446.526 µs [426.036 µs, 467.017 µs] 82.403 µs (22.6%)
iast_TELEMETRY_OFF 469.32 µs [448.81 µs, 489.829 µs] 105.197 µs (28.9%)
tracing 438.848 µs [418.037 µs, 459.658 µs] 74.724 µs (20.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 375.096 µs [354.814 µs, 395.379 µs] -
iast 472.485 µs [451.765 µs, 493.206 µs] 97.389 µs (26.0%)
iast_FULL 531.447 µs [511.067 µs, 551.827 µs] 156.351 µs (41.7%)
iast_INACTIVE 457.889 µs [436.935 µs, 478.844 µs] 82.793 µs (22.1%)
iast_TELEMETRY_OFF 468.475 µs [448.055 µs, 488.895 µs] 93.379 µs (24.9%)
tracing 444.256 µs [423.091 µs, 465.422 µs] 69.16 µs (18.4%)

@smola smola added the comp: asm iast Application Security Management (IAST) label Oct 30, 2023
@jandro996 jandro996 force-pushed the alejandro.gonzalez/Support_SSRF_in_Apache_HttpClient branch from 9f59593 to 7808112 Compare November 7, 2023 13:14
@jandro996 jandro996 force-pushed the alejandro.gonzalez/Support_SSRF_in_Apache_HttpClient branch from 4f54b87 to c5a4757 Compare November 14, 2023 08:00
@jandro996 jandro996 force-pushed the alejandro.gonzalez/Support_SSRF_in_Apache_HttpClient branch from c8bc4dd to 1a62cf3 Compare November 14, 2023 08:43
@jandro996 jandro996 marked this pull request as ready for review December 12, 2023 12:10
@jandro996 jandro996 requested review from a team as code owners December 12, 2023 12:10
@@ -189,6 +189,8 @@
1 org.aopalliance.*
1 org.antlr.*
1 org.apache.*
#apache httpClient needs URI propagation
0 org.apache.http.client.methods.*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not instrumenting all of the classes on that package. My understanding is that the policy is to explicit the classes that are instrumented:
0 org.apache.http.client.methods.HttpUriRequest

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change in the exclusion list is not necessary for HttpUriRequest, it aims to instrument the call to URI#create in all the http verb classes like:

HttpTrace
HttpPut
HttpPost
HttpOptions
HttpHead
HttpGet
HttpDelete

This list change between version, so I decided to exclude all the package

Let me know if this is a valid approach or we need to invest in look for each class in all versions

@jandro996 jandro996 merged commit 7574c1f into master Dec 19, 2023
73 checks passed
@jandro996 jandro996 deleted the alejandro.gonzalez/Support_SSRF_in_Apache_HttpClient branch December 19, 2023 17:55
@github-actions github-actions bot added this to the 1.27.0 milestone Dec 19, 2023
@bantonsson bantonsson modified the milestones: 1.26.1, previous 1.26.1 Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp: asm iast Application Security Management (IAST)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants