Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(test): only test num of L1 layers in compaction smoke test #9186

Merged
merged 6 commits into from
Oct 2, 2024

Conversation

skyzh
Copy link
Member

@skyzh skyzh commented Sep 27, 2024

Problem

close #9160

For whatever reason, pg17's WAL pattern seems different from others, which triggers some flaky behavior within the compaction smoke test.

Summary of changes

  • Run L0 compaction before proceeding with the read benchmark.
  • So that we can ensure the num of L0 layers is 0 and test the compaction behavior only with L1 layers.

We have a threshold for triggering L0 compaction. In some cases, the test case did not produce enough L0 layers to do a L0 compaction, therefore leaving the layer map with 3+ L0 layers above the L1 layers. This increases the average read depth for the timeline.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@skyzh skyzh marked this pull request as ready for review September 27, 2024 18:28
@skyzh skyzh requested a review from a team as a code owner September 27, 2024 18:28
@skyzh skyzh changed the title fix(test): only test num of L1 layers in compaction smoke test fix(test): try to test num of L1 layers in compaction smoke test Sep 27, 2024
@skyzh skyzh changed the title fix(test): try to test num of L1 layers in compaction smoke test fix(test): only test num of L1 layers in compaction smoke test Sep 27, 2024
Copy link

github-actions bot commented Sep 27, 2024

5022 tests run: 4864 passed, 0 failed, 158 skipped (full report)


Flaky tests (2)

Postgres 17

Postgres 16

Code coverage* (full report)

  • functions: 31.4% (7489 of 23887 functions)
  • lines: 49.6% (60127 of 121270 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
2a24091 at 2024-10-02T14:35:45.312Z :recycle:

Copy link
Contributor

@VladLazar VladLazar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments, but looks good to me.

test_runner/regress/test_compaction.py Show resolved Hide resolved
pageserver/src/tenant/timeline/compaction.rs Outdated Show resolved Hide resolved
test_runner/fixtures/pageserver/utils.py Outdated Show resolved Hide resolved
Signed-off-by: Alex Chi Z <chi@neon.tech>
Signed-off-by: Alex Chi Z <chi@neon.tech>
Signed-off-by: Alex Chi Z <chi@neon.tech>
Signed-off-by: Alex Chi Z <chi@neon.tech>
@skyzh skyzh force-pushed the skyzh/fix-compaction-smoke branch from 791f9a6 to e5023ec Compare October 1, 2024 21:17
@skyzh skyzh enabled auto-merge (squash) October 1, 2024 23:27
@skyzh skyzh disabled auto-merge October 2, 2024 13:38
Signed-off-by: Alex Chi Z <chi@neon.tech>
@skyzh skyzh force-pushed the skyzh/fix-compaction-smoke branch from 4a2216b to 2a24091 Compare October 2, 2024 13:42
@skyzh skyzh enabled auto-merge (squash) October 2, 2024 16:31
@skyzh skyzh merged commit 7008854 into main Oct 2, 2024
83 checks passed
@skyzh skyzh deleted the skyzh/fix-compaction-smoke branch October 2, 2024 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failures in test_pageserver_compaction_smoke
2 participants