rptest: Ignore error logs generated by anomaly injection #16496
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
/ci-repeat 10 |
|
/ci-repeat 10 |
abhijat
force-pushed
the
issue/16474/ignore-test-generated-error-logs
branch
from
a24233b
to
4f2acee
Compare
|
/ci-repeat 10 |
abhijat
force-pushed
the
issue/16474/ignore-test-generated-error-logs
branch
from
302e0dc
to
4f2acee
Compare
|
/ci-repeat 10 |
|
/ci-repeat |
abhijat
force-pushed
the
issue/16474/ignore-test-generated-error-logs
branch
from
4f2acee
to
c9982b5
Compare
The log allow list decorator conventionally matches logs on string or regex. Sometimes finer grained checks are required, where extra context is needed to see if a log message is acceptable or signals an error. The AllowLogsOnPredicate class allows a test to register one of its methods to be called for each error log, that method can then return a not-None style result to signal that an error is acceptable, and False or None to signal that an error should cause the test to fail.
The base offset value is useful when validating error logs.
This test removes a segment from the manifest to generate anomalies. A side effect of this is that the validation in manifest done during spillover which checks that the start of the manifest and the start of spillover must match can fail, because the first segment of the manifest has been force reset away. Effectively, spillover manifest is creating by iterating over segment list in main manifest. This main manifest is missing its first segment because of the anomaly injection. When applying the spillover command, the manifest compares the start offsets of the spillover meta and the main manifest. The start offset has not changed during the anomaly injection. As a result the apply-spillover command fails. This is acceptable because the anomaly injection should cause this failure, and this change adds the error to allow list.
abhijat
force-pushed
the
issue/16474/ignore-test-generated-error-logs
branch
from
c9982b5
to
b4ee8fb
Compare
dotnwat
approved these changes
Feb 8, 2024
| the predicate to return falsy values to reject a log from being allowed. | ||
| """ | ||
| assert self.predicate is not None, f"Log predicate method {self.method} is not initialized" | ||
| return self.predicate(log_line) or None |
|
/backport v23.3.x |
|
/backport v23.2.x |
|
Failed to create a backport PR to v23.3.x branch. I tried: |
|
Failed to create a backport PR to v23.2.x branch. I tried: |
|
Backporting as it is looks tricky. It might be easier to just add the string to log allow list in the two branches. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
FIXES #16474
This test removes a segment from the manifest to generate anomalies. A side effect of this is that the validation in manifest done during spillover which checks that the start of the manifest and the start of spillover must match can fail, because the first segment of the manifest has been force reset away.
Effectively, spillover manifest is creating by iterating over segment list in main manifest. This main manifest is missing its first segment because of the anomaly injection.
When applying the spillover command, the manifest compares the start offsets of the spillover meta and the main manifest. The start offset has not changed during the anomaly injection. As a result the apply-spillover command fails.
This is acceptable because the anomaly injection should cause this failure, and this change adds the error to allow list.
Additionally, a utility is added which can allow logs using information from a test run.
Backports Required
Release Notes