Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce object iterations in NoSQL vulnerabilities #4186

Merged
merged 1 commit into from
Mar 26, 2024

Conversation

uurien
Copy link
Collaborator

@uurien uurien commented Mar 25, 2024

What does this PR do?

When we format a mongodb injection vulnerability:

  • Reduce the depth from 50 to 10: Vulnerability is always in the first levels, we don't need to go too deep to send the vulnerability to backend.
  • Prevent circular references: Ignore circular references when we are creating the vulnerability to save time and memory.

Motivation

Improve memory footprint detected in some customers in this method.

Plugin Checklist

  • Unit tests.

Additional Notes

Security

Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
  • This PR doesn't touch any of that.

Unsure? Have a question? Request a review!

Copy link

Overall package size

Self size: 6.24 MB
Deduped: 61.19 MB
No deduping: 61.94 MB

Dependency sizes

name version self size total size
@datadog/native-iast-taint-tracking 1.7.0 16.71 MB 16.72 MB
@datadog/native-appsec 7.1.0 14.37 MB 14.38 MB
@datadog/pprof 5.1.0 8.83 MB 9.68 MB
protobufjs 7.2.5 2.77 MB 6.56 MB
@datadog/native-iast-rewriter 2.3.0 2.15 MB 2.24 MB
@opentelemetry/core 1.14.0 872.87 kB 1.47 MB
@datadog/native-metrics 2.0.0 898.77 kB 1.3 MB
@opentelemetry/api 1.4.1 780.32 kB 780.32 kB
import-in-the-middle 1.7.3 67.62 kB 731.01 kB
pprof-format 2.0.7 588.12 kB 588.12 kB
msgpack-lite 0.1.26 201.16 kB 281.59 kB
opentracing 0.14.7 194.81 kB 194.81 kB
semver 7.5.4 93.4 kB 123.8 kB
@datadog/sketches-js 2.1.0 109.9 kB 109.9 kB
lodash.sortby 4.7.0 75.76 kB 75.76 kB
lru-cache 7.14.0 74.95 kB 74.95 kB
ipaddr.js 2.1.0 60.23 kB 60.23 kB
ignore 5.2.4 51.22 kB 51.22 kB
int64-buffer 0.1.10 49.18 kB 49.18 kB
shell-quote 1.8.1 44.96 kB 44.96 kB
istanbul-lib-coverage 3.2.0 29.34 kB 29.34 kB
tlhunter-sorted-set 0.1.0 24.94 kB 24.94 kB
limiter 1.1.5 23.17 kB 23.17 kB
dc-polyfill 0.1.4 23.1 kB 23.1 kB
retry 0.13.1 18.85 kB 18.85 kB
node-abort-controller 3.1.1 16.89 kB 16.89 kB
jest-docblock 29.7.0 8.99 kB 12.76 kB
crypto-randomuuid 1.0.0 11.18 kB 11.18 kB
path-to-regexp 0.1.7 6.78 kB 6.78 kB
koalas 1.0.2 6.47 kB 6.47 kB
methods 1.1.2 5.29 kB 5.29 kB
module-details-from-path 1.0.3 4.47 kB 4.47 kB

🤖 This report was automatically generated by heaviest-objects-in-the-universe

Copy link

codecov bot commented Mar 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.22%. Comparing base (bc78d8b) to head (e7fd46a).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #4186   +/-   ##
=======================================
  Coverage   85.22%   85.22%           
=======================================
  Files         247      247           
  Lines       10948    10950    +2     
  Branches       33       33           
=======================================
+ Hits         9330     9332    +2     
  Misses       1618     1618           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -13,15 +13,18 @@ const KEYS_REGEX_WITHOUT_SENSITIVE_RANGES = new RegExp(`"(${STRINGIFY_RANGE_KEY}

const sensitiveValueRegex = new RegExp(DEFAULT_IAST_REDACTION_VALUE_PATTERN, 'gmi')

function iterateObject (target, fn, levelKeys = [], depth = 50) {
function iterateObject (target, fn, levelKeys = [], depth = 10, visited = new Set()) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reduced it to 10 instead to a 20, because this function is used only when we need to format a nosql vulnerability. The vulnerability is found only in the first levels, so 10 is more than enough to add context for the vulnerability.

@pr-commenter
Copy link

pr-commenter bot commented Mar 25, 2024

Benchmarks

Benchmark execution time: 2024-03-25 17:50:03

Comparing candidate commit e7fd46a in PR branch ugaitz/improve-vulnerability-formater-with-json with baseline commit bc78d8b in branch master.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 260 metrics, 5 unstable metrics.

scenario:plugin-graphql-with-depth-and-collapse-on-18

  • 🟩 max_rss_usage [-122.857MB; -106.763MB] or [-12.969%; -11.270%]

@uurien uurien marked this pull request as ready for review March 26, 2024 08:20
@uurien uurien requested a review from a team as a code owner March 26, 2024 08:20
Copy link
Member

@simon-id simon-id left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any test for circular reference ?

@uurien
Copy link
Collaborator Author

uurien commented Mar 26, 2024

We already had one, I modified it to work with the new approach (remove circular reference) instead of previous "check depth"
https://github.com/DataDog/dd-trace-js/pull/4186/files#diff-a7214cb9a3e3470f04bda05fd7e21321d3bdf279b6c70f9d570e94f284db3a9dL154

@uurien uurien merged commit f8a0d4b into master Mar 26, 2024
114 checks passed
@uurien uurien deleted the ugaitz/improve-vulnerability-formater-with-json branch March 26, 2024 09:53
This was referenced Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants