Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(test): storage scrubber should only log to stdout with info #9067

Merged
merged 4 commits into from
Sep 24, 2024

Conversation

skyzh
Copy link
Member

@skyzh skyzh commented Sep 19, 2024

Problem

As @koivunej mentioned in the storage channel, for regress test, we don't need to create a log file for the scrubber, and we should reduce noisy logs.

Summary of changes

  • Disable log file creation for storage scrubber
  • Only log at info level

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@skyzh skyzh requested review from koivunej and a team September 19, 2024 19:03
Copy link

github-actions bot commented Sep 19, 2024

5038 tests run: 4874 passed, 0 failed, 164 skipped (full report)


Flaky tests (6)

Postgres 17

Postgres 15

Postgres 14

Code coverage* (full report)

  • functions: 32.2% (7477 of 23255 functions)
  • lines: 50.0% (60231 of 120574 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
1e0f817 at 2024-09-24T22:45:26.199Z :recycle:

@koivunej
Copy link
Member

koivunej commented Sep 20, 2024

Example test report. It still has the same captured (note captured stderr and stdout files) AND forwarded logging (see test log).

Overall it's more healthy to output two (stderr, "pipe" to test log) rather than three times (stderr, "pipe" to test log, write to special file which must've been ignored by allure report generation?). I do not understand why we need to pipe it to the test log in any case. We do not do this for pageserver, for example, or safekeepers, or postgres, or storage controller.

@skyzh
Copy link
Member Author

skyzh commented Sep 20, 2024

Well, probably it's because scrubber is a foreground process that gets explicitly called in the code path. I'll see how to fully remove it...

@skyzh
Copy link
Member Author

skyzh commented Sep 20, 2024

added a commit 0d1e1d3

@skyzh skyzh force-pushed the skyzh/disable-scrubber-logging-test branch from 0d1e1d3 to 002df5b Compare September 24, 2024 14:57
@skyzh
Copy link
Member Author

skyzh commented Sep 24, 2024

should finally work now (tried different combinations of these parameters 🤣)

@skyzh skyzh requested a review from arpad-m September 24, 2024 17:44
@skyzh skyzh enabled auto-merge (squash) September 24, 2024 20:31
Signed-off-by: Alex Chi Z <chi@neon.tech>
Signed-off-by: Alex Chi Z <chi@neon.tech>
Signed-off-by: Alex Chi Z <chi@neon.tech>
Signed-off-by: Alex Chi Z <chi@neon.tech>
@skyzh skyzh force-pushed the skyzh/disable-scrubber-logging-test branch from 4d6db72 to 1e0f817 Compare September 24, 2024 20:31
@skyzh skyzh merged commit 5f2f31e into main Sep 24, 2024
79 checks passed
@skyzh skyzh deleted the skyzh/disable-scrubber-logging-test branch September 24, 2024 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants