Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mysql (ticdc): Improve the performance of the mysql sink by refining the transaction event batching logic (#10466) #11248

Merged

Conversation

ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #10466

What problem does this PR solve?

Issue Number: close #11241

What is changed and how it works?

  1. When there are a lot of txns in worker's txnCh, we try to do flush until save batches to maxTxnRows, instead of doing flush with tiny txns because of ticker.C.
  2. Adjust the calculation of worker-busy-ratio to make it more precise and more clear.
    The original calculation of worker-busy-ratio may have certain deviation when the flush time is long (such as larger than hundreds of milliseconds). And the longer the flush time, the larger the deviation. The reason for the deviation is that when the flsuh time is long, the interval between each time we increase the worker busy ratio value clearly exceeds 1s, resulting in the growth rate of the worker-busy-ratio per second being less than actual rate.

Performance Test Result

We take a large workload on upstream to keep worker-busy-ratio as 100%, and compare the improvement of sink performance before and after optimization.

  1. Simulate network latency between the upstream and downstream is 2ms.
    After Optimization -- 33339 rows/s (+14%). Before Optimization -- 29075 rows/s
  2. Simulate network latency between the upstream and downstream is 5ms.
    After Optimization -- 20528 rows/s(+13%). Before Optimization -- 18129 rows/s
  3. Simulate network latency between the upstream and downstream is 20ms.
    After Optimization -- 7273 rows/s(+20%). Before Optimization -- 6048 rows/s
    image

New panel of worker busy ratio

img_v3_0274_02f665a8-9887-4a35-a4f7-9dceaa8d497g

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note

None

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot added lgtm release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. type/cherry-pick-for-release-7.5 This PR is cherry-picked to release-7.5 from a source PR. labels Jun 4, 2024
@ti-chi-bot ti-chi-bot bot added the do-not-merge/cherry-pick-not-approved The current cherry-pick pull request has not been approved and cannot be merged. label Jun 4, 2024
@hongyunyan
Copy link
Collaborator

/retest

Copy link
Contributor

ti-chi-bot bot commented Jun 7, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: asddongmen

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label Jun 7, 2024
@ti-chi-bot ti-chi-bot added the cherry-pick-approved Cherry pick PR approved by release team. label Jun 13, 2024
@ti-chi-bot ti-chi-bot bot removed the do-not-merge/cherry-pick-not-approved The current cherry-pick pull request has not been approved and cannot be merged. label Jun 13, 2024
@ti-chi-bot ti-chi-bot bot merged commit 8588082 into pingcap:release-7.5 Jun 13, 2024
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved cherry-pick-approved Cherry pick PR approved by release team. lgtm release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. type/cherry-pick-for-release-7.5 This PR is cherry-picked to release-7.5 from a source PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants