Prevent massive database errors on screenshot insertion #2597
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The previous approach was to catch conflicts on screenshot insertion in perl context yielding the right functional behaviour but also causing a log entry for every image object amounting to multiple gigabytes of postgres database error log files which can cause obvious problems. By using a similar approach as is already done in lib/OpenQA/Schema/Result/Jobs.pm with raw SQL we can ask the database for silent conflict handling and more efficiently not involving perl exception handling. Related progress issue: https://progress.opensuse.org/issues/60926
okurz
force-pushed
the
feature/clone_custom_list
branch
from
0725a56
to
f8daf5a
Compare
Codecov Report
@@ Coverage Diff @@
## master #2597 +/- ##
==========================================
+ Coverage 91.57% 91.59% +0.01%
==========================================
Files 181 181
Lines 11233 11226 -7
==========================================
- Hits 10287 10282 -5
+ Misses 946 944 -2
Continue to review full report at Codecov.
|
Martchus
approved these changes
Dec 12, 2019
of course! |
kraih
reviewed
Dec 12, 2019
kraih
reviewed
Dec 12, 2019
kraih
reviewed
Dec 12, 2019
okurz
added a commit
to okurz/openQA
that referenced
this pull request
Dec 18, 2019
Same as previously for screenshots where we fixed the same error in both locations where a DB insertion is done already in os-autoinst#1792 and os-autoinst#2597 we should also prevent expected conflicts to cause an error entry in postgres logs about job modules. This is a followup to ecb7d1d where the unique constraint for job modules was introduced. Related progress issue: https://progress.opensuse.org/issues/60923
okurz
added a commit
to okurz/openQA
that referenced
this pull request
Dec 18, 2019
Same as previously for screenshots where we fixed the same error in both locations where a DB insertion is done already in os-autoinst#1792 and os-autoinst#2597 we should also prevent expected conflicts to cause an error entry in postgres logs about job modules. This is a followup to ecb7d1d where the unique constraint for job modules was introduced. Related progress issue: https://progress.opensuse.org/issues/60923
okurz
added a commit
to okurz/openQA
that referenced
this pull request
Dec 18, 2019
Same as previously for screenshots where we fixed the same error in both locations where a DB insertion is done already in os-autoinst#1792 and os-autoinst#2597 we should also prevent expected conflicts to cause an error entry in postgres logs about job modules. This is a followup to ecb7d1d where the unique constraint for job modules was introduced. Related progress issue: https://progress.opensuse.org/issues/60923
okurz
added a commit
to okurz/openQA
that referenced
this pull request
Jan 24, 2020
Same as previously for screenshots where we fixed the same error in both locations where a DB insertion is done already in os-autoinst#1792 and os-autoinst#2597 we should also prevent expected conflicts to cause an error entry in postgres logs about job modules. This is a followup to ecb7d1d where the unique constraint for job modules was introduced. Related progress issue: https://progress.opensuse.org/issues/60923
okurz
added a commit
to okurz/openQA
that referenced
this pull request
Mar 2, 2020
Same as previously for screenshots where we fixed the same error in both locations where a DB insertion is done already in os-autoinst#1792 and os-autoinst#2597 we should also prevent expected conflicts to cause an error entry in postgres logs about job modules. This is a followup to ecb7d1d where the unique constraint for job modules was introduced. Related progress issue: https://progress.opensuse.org/issues/60923
okurz
added a commit
to okurz/openQA
that referenced
this pull request
Mar 2, 2020
Same as previously for screenshots where we fixed the same error in both locations where a DB insertion is done already in os-autoinst#1792 and os-autoinst#2597 we should also prevent expected conflicts to cause an error entry in postgres logs about job modules. This is a followup to ecb7d1d where the unique constraint for job modules was introduced. Related progress issue: https://progress.opensuse.org/issues/60923
okurz
added a commit
to okurz/openQA
that referenced
this pull request
Mar 4, 2020
Same as previously for screenshots where we fixed the same error in both locations where a DB insertion is done already in os-autoinst#1792 and os-autoinst#2597 we should also prevent expected conflicts to cause an error entry in postgres logs about job modules. This is a followup to ecb7d1d where the unique constraint for job modules was introduced. Related progress issue: https://progress.opensuse.org/issues/60923
okurz
added a commit
to okurz/openQA
that referenced
this pull request
Mar 4, 2020
Same as previously for screenshots where we fixed the same error in both locations where a DB insertion is done already in os-autoinst#1792 and os-autoinst#2597 we should also prevent expected conflicts to cause an error entry in postgres logs about job modules. This is a followup to ecb7d1d where the unique constraint for job modules was introduced. Related progress issue: https://progress.opensuse.org/issues/60923
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The previous approach was to catch conflicts on screenshot insertion in
perl context yielding the right functional behaviour but also causing a
log entry for every image object amounting to multiple gigabytes of
postgres database error log files which can cause obvious problems. By
using a similar approach as is already done in
lib/OpenQA/Schema/Result/Jobs.pm with raw SQL we can ask the database
for silent conflict handling and more efficiently not involving perl
exception handling.
Related progress issue: https://progress.opensuse.org/issues/60926