fix(parsers): use unsaved_tags instead of tags= in Finding constructor#14626
Open
valentijnscholten wants to merge 3 commits intoDefectDojo:bugfixfrom
Open
fix(parsers): use unsaved_tags instead of tags= in Finding constructor#14626valentijnscholten wants to merge 3 commits intoDefectDojo:bugfixfrom
valentijnscholten wants to merge 3 commits intoDefectDojo:bugfixfrom
Conversation
…r for performance Passing tags= directly to the Finding() constructor triggers expensive tagulous processing for every finding. Using finding.unsaved_tags instead bypasses this overhead and lets the import pipeline handle tags efficiently. Affected parsers: jfrog_xray_unified, dependency_check, cargo_audit, anchore_grype, threat_composer. Benchmark on 14,219 findings: 99s -> 7.97s (12x faster).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
While working performance improvements I noticed the JFrog xray parser looked quite slow compared to other parsers.
Turns out that setting the
tagsfield while constructingFindingmodels (in-memory) made it very slow.The 14k findings report took anywhere between 60 and 90 second just to parse. By switching to
unsaved_tagsit's now ~9seconds. Some other parsers needed the same fix and are part of this PR.tags=directly to theFinding()constructor triggers expensive tagulous tag processing for every finding object created during parsingfinding.unsaved_tags = tagsafter construction instead, which lets the import pipeline handle tags efficientlyBenchmark on JFrog Xray Unified with 14,219 findings: 99s → 7.97s (12x faster)