Gracefully handle unique constraint violations #3648
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nscuro
force-pushed
the
gracefully-handle-unique-constraint-violations
branch
from
cfed094
to
a8d0ba7
Compare
Coverage summary from CodacySee diff coverage on Codacy
Coverage variation details
Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: Diff coverage details
Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: See your quality gate settings Change summary preferencesCodacy will stop sending the deprecated coverage status from June 5th, 2024. Learn more |
When uploading many BOMs in bulk, it can happen that vulnerability and repository metadata analysis is performed multiple times for the same component identity in parallel. It is more likely to happen when the BOMs being uploaded have big overlap in terms of components they contain. This can lead to unique constraint violations when DT updates the `COMPONENTANALYSISCACHE` table. Occurrences of this are non-critical, and kind of expected. Instead of letting the task crash, simply "accept" the exception and consider the job done. Note that DataNucleus will log a warning even when we handle the exception. Signed-off-by: nscuro <nscuro@protonmail.com>
nscuro
force-pushed
the
gracefully-handle-unique-constraint-violations
branch
from
a8d0ba7
to
9ae124a
Compare
|
Purposefully not adding tests since the conditions require race conditions in order to be triggered. Such tests have proven to be incredibly flaky. The changes made are low risk. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
When uploading many BOMs in bulk, it can happen that vulnerability and repository metadata analysis is performed multiple times for the same component identity in parallel. It is more likely to happen when the BOMs being uploaded have big overlap in terms of components they contain.
This can lead to unique constraint violations when DT updates the
COMPONENTANALYSISCACHEtable. Occurrences of this are non-critical, and kind of expected. Instead of letting the task crash, simply "accept" the exception and consider the job done.Note that DataNucleus will log a warning even when we handle the exception.
Addressed Issue
N/A
Additional Details
N/A
Checklist
This PR fixes a defect, and I have provided tests to verify that the fix is effectiveThis PR introduces changes to the database model, and I have added corresponding update logicThis PR introduces new or alters existing behavior, and I have updated the documentation accordingly