Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: add more info-level logging in shard splits #8137

Merged
merged 1 commit into from
Jun 24, 2024

Conversation

jcsp
Copy link
Contributor

@jcsp jcsp commented Jun 23, 2024

Problem

test_sharding_autosplit is occasionally failing on warnings about shard splits taking longer than expected (Exclusive lock by ShardSplit was held for...)

It's not obvious which part is taking the time (I suspect remote storage uploads).

Example: https://neon-github-public-dev.s3.amazonaws.com/reports/main/9618788427/index.html#testresult/b395294d5bdeb783/

Summary of changes

  • Since shard splits are infrequent events, we can afford to be very chatty: add a bunch of info-level logging throughout the process.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@jcsp jcsp added c/storage/pageserver Component: storage: pageserver a/tech_debt Area: related to tech debt labels Jun 23, 2024
Copy link

2910 tests run: 2793 passed, 0 failed, 117 skipped (full report)


Flaky tests (1)

Postgres 16

  • test_tenant_delete_smoke: release

Code coverage* (full report)

  • functions: 32.4% (6869 of 21175 functions)
  • lines: 49.9% (53431 of 107090 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
7f7f33f at 2024-06-23T20:44:32.953Z :recycle:

@jcsp jcsp marked this pull request as ready for review June 24, 2024 08:38
@jcsp jcsp requested a review from a team as a code owner June 24, 2024 08:38
@jcsp jcsp requested a review from arssher June 24, 2024 08:38
@jcsp jcsp merged commit de05f90 into main Jun 24, 2024
68 checks passed
@jcsp jcsp deleted the jcsp/shard-split-logging branch June 24, 2024 10:53
conradludgate pushed a commit that referenced this pull request Jun 27, 2024
## Problem

`test_sharding_autosplit` is occasionally failing on warnings about
shard splits taking longer than expected (`Exclusive lock by ShardSplit
was held for`...)

It's not obvious which part is taking the time (I suspect remote storage
uploads).

Example:
https://neon-github-public-dev.s3.amazonaws.com/reports/main/9618788427/index.html#testresult/b395294d5bdeb783/

## Summary of changes

- Since shard splits are infrequent events, we can afford to be very
chatty: add a bunch of info-level logging throughout the process.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a/tech_debt Area: related to tech debt c/storage/pageserver Component: storage: pageserver
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants