Skip to content

Limit SARIF code snippet size#1827

Merged
koesie10 merged 3 commits intomainfrom
koesie10/filter-sarif-snippets
Dec 2, 2022
Merged

Limit SARIF code snippet size#1827
koesie10 merged 3 commits intomainfrom
koesie10/filter-sarif-snippets

Conversation

@koesie10
Copy link
Copy Markdown
Member

@koesie10 koesie10 commented Dec 2, 2022

This adds a new filtering on SARIF code snippets for very large code snippets (defined as 8KB or more). If less than 1% of such a snippet is highlighted, it will not include the code snippet in the analysed results, and it will thus not be shown in the UI.

This is to avoid very large SARIF files that can cause the extension host to crash when the analysis results are send to the UI. I don't think any of these snippets would ever be useful to show, so it should be fine to just not include them.

Checklist

  • CHANGELOG.md has been updated to incorporate all user visible changes made by this pull request.
  • Issues have been created for any UI or other user-facing changes made by this pull request.
  • [Maintainers only] If this pull request makes user-facing changes that require documentation changes, open a corresponding docs pull request in the github/codeql repo and add the ready-for-doc-review label there.

This adds a new filtering on SARIF code snippets for very large code
snippets (defined as 8MB or more). If less than 1% of such a snippet
is highlighted, it will not include the code snippet in the analysed
results, and it will thus not be shown in the UI.

This is to avoid very large SARIF files that can cause the extension
host to crash when the analysis results are send to the UI. I don't
think any of these snippets would ever be useful to show, so it should
be fine to just not include them.
@koesie10 koesie10 added the secexp label Dec 2, 2022
@koesie10
Copy link
Copy Markdown
Member Author

koesie10 commented Dec 2, 2022

I'm not sure if this is the best approach to make the UI work in more cases, but this makes the results view work for vercel/next.js (it still takes more than 10 seconds to load, but that can be fixed in a follow-up PR). If you think it's useful, I'd be happy to add tests for this behaviour as well.

@koesie10 koesie10 marked this pull request as ready for review December 2, 2022 14:38
@koesie10 koesie10 requested review from a team as code owners December 2, 2022 14:38
Copy link
Copy Markdown
Contributor

@robertbrignull robertbrignull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for implementing this. I agree with the logic about region sizes and this should hopefully catch cases of generated or minimized code. I think we can certainly go with this as a first attempt and adjust the logic if we find other cases we want to filter out, so I'm not too worried about trying to make the logic absolutely perfect in this PR.

I have a few comments about variable naming, but the actual logic in the code looks correct to me 👍

Comment thread extensions/ql-vscode/src/remote-queries/sarif-processing.ts Outdated
Comment thread extensions/ql-vscode/src/remote-queries/sarif-processing.ts
Comment thread extensions/ql-vscode/src/remote-queries/sarif-processing.ts Outdated
Comment thread extensions/ql-vscode/src/remote-queries/sarif-processing.ts Outdated
@robertbrignull
Copy link
Copy Markdown
Contributor

Sorry just saw your other comment. Tests for this would be great too!

Copy link
Copy Markdown
Contributor

@robertbrignull robertbrignull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates and tests

@koesie10 koesie10 enabled auto-merge December 2, 2022 16:02
@koesie10 koesie10 merged commit 994ebaa into main Dec 2, 2022
@koesie10 koesie10 deleted the koesie10/filter-sarif-snippets branch December 2, 2022 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants