-
Notifications
You must be signed in to change notification settings - Fork 269
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ticdc/log: suppress duplicated log to avoid flooding #3641
base: master
Are you sure you want to change the base?
Conversation
[REVIEW NOTIFICATION] This pull request has not been approved. To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
Can this sampling be scoped? I.e., can it only filter logs from some packages, but not others? Using sampling on some mission critical modules might cause difficulty debugging. |
@liuzix No, at present, zap only supports global sampling configuration, it cannot filter logs based on package name. |
+1, btw
|
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #3641 +/- ##
================================================
+ Coverage 56.5356% 58.0874% +1.5517%
================================================
Files 211 241 +30
Lines 22798 24501 +1703
================================================
+ Hits 12889 14232 +1343
- Misses 8598 8845 +247
- Partials 1311 1424 +113 |
@zhaoxinyu: PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zhaoxinyu Do you have time to advance it?
This PR is preempted by other high-priority things. I will keep advancing it after completing things in this sprint. |
/hold |
What problem does this PR solve?
#3524 and #4006
What is changed and how it works?
Add log sampling config items into cdc server config. We use zap.SamplingConfig to efficiently determine whether to drop or write a log.
For example, we set the configs as follows:
And if the server wants to write 200 same messages into log, only 101 messages (indexed by [0~99, 199]) could be written to log, the remaining 99 messages are supposed to be dropped.
Check List
Tests
Release note