Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: stabilize compat tests #7227

Merged
merged 2 commits into from Mar 25, 2024
Merged

tests: stabilize compat tests #7227

merged 2 commits into from Mar 25, 2024

Conversation

jcsp
Copy link
Contributor

@jcsp jcsp commented Mar 25, 2024

This test had two flaky failure modes:

  • pageserver log error for timeline not found: this resulted from changes for DR when timeline destroy/create was added, but endpoint was left running during that operation.
  • storage controller log error because the test was running for long enough that a background reconcile happened at almost the exact moment of test teardown, and our test fixtures tear down the pageservers before the controller.

Closes: #7224

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@jcsp jcsp added c/storage/pageserver Component: storage: pageserver a/test Area: related to testing labels Mar 25, 2024
@jcsp jcsp requested a review from arpad-m March 25, 2024 13:32
Copy link

2718 tests run: 2581 passed, 0 failed, 137 skipped (full report)


Code coverage* (full report)

  • functions: 28.1% (6287 of 22345 functions)
  • lines: 47.0% (44221 of 94108 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
8027012 at 2024-03-25T14:17:53.839Z :recycle:

@jcsp jcsp requested a review from bayandin March 25, 2024 14:19
@jcsp jcsp marked this pull request as ready for review March 25, 2024 14:19
Copy link
Member

@bayandin bayandin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me, thanks!

@jcsp jcsp merged commit 2713142 into main Mar 25, 2024
56 of 57 checks passed
@jcsp jcsp deleted the jcsp/issue-7224 branch March 25, 2024 14:35
jcsp added a commit that referenced this pull request Mar 26, 2024
## Problem

#7227 destabilized various
tests in the performance suite, with log errors during shutdown. It's
because we switched shutdown order to stop the storage controller before
the pageservers.

## Summary of changes

- Tolerate "connection failed" errors from pageservers trying to
validation their deletion queue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a/test Area: related to testing c/storage/pageserver Component: storage: pageserver
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test_backward_compatibility is flaky
2 participants