statistics: add upper bound of retry for sync load #52658

hawkingrei · 2024-04-17T06:03:59Z

What problem does this PR solve?

Issue Number: close #52657

Problem Summary:

What changed and how does it work?

We find if the sync load task fails to run, it will retry forever.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

codecov · 2024-04-17T07:11:35Z

Codecov Report

Merging #52658 (30fa279) into master (a9e3398) will increase coverage by 1.9236%.
Report is 10 commits behind head on master.
The diff coverage is 88.2352%.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #52658        +/-   ##
================================================
+ Coverage   72.3821%   74.3057%   +1.9236%     
================================================
  Files          1471       1473         +2     
  Lines        427187     428516      +1329     
================================================
+ Hits         309207     318412      +9205     
+ Misses        98761      90238      -8523     
- Partials      19219      19866       +647

Flag	Coverage Δ
integration	`48.7894% <47.0588%> (?)`
unit	`71.1762% <88.2352%> (-0.0983%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`53.9957% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`49.4332% <ø> (+8.3204%)`	⬆️

hawkingrei · 2024-04-17T07:52:04Z

/retest

hawkingrei · 2024-04-17T08:39:02Z

/retest

pkg/statistics/handle/syncload/stats_syncload_test.go

pkg/statistics/handle/syncload/stats_syncload.go

hawkingrei · 2024-04-17T15:43:04Z

/retest

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

hawkingrei · 2024-04-18T07:23:29Z

pkg/statistics/handle/syncload/stats_syncload.go

+	return task.Retry <= 3
+}
+
+func (s *statsSyncLoad) handleOneItemTask(sctx sessionctx.Context, task *statstypes.NeededItemTask) (err error) {


Here is the clean code. the origin's result *stmtctx.StatsLoadResult is unnecessary.

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

hawkingrei · 2024-04-22T09:23:47Z

@hi-rustin PTAL

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

ti-chi-bot · 2024-04-22T10:44:09Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hi-rustin, winoros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/statistics/OWNERS~~ [hi-rustin,winoros]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-04-22T10:44:12Z

[LGTM Timeline notifier]

Timeline:

2024-04-17 09:00:29.237240412 +0000 UTC m=+415048.348286861: ☑️ agreed by winoros.
2024-04-22 10:44:11.21615385 +0000 UTC m=+853270.327200300: ☑️ agreed by hi-rustin.

hawkingrei · 2024-04-22T15:04:34Z

/cherrypick release-8.1
/cherrypick release-7.5
/cherrypick release-7.1
/cherrypick release-6.5

ti-chi-bot · 2024-04-22T15:05:20Z

@hawkingrei: new pull request created to branch release-6.5: #52816.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5
/cherrypick release-7.1
/cherrypick release-6.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2024-04-22T15:06:04Z

@hawkingrei: new pull request created to branch release-7.1: #52817.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5
/cherrypick release-7.1
/cherrypick release-6.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2024-04-22T15:06:48Z

@hawkingrei: new pull request created to branch release-7.5: #52818.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5
/cherrypick release-7.1
/cherrypick release-6.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2024-04-22T15:07:29Z

@hawkingrei: new pull request created to branch release-8.1: #52819.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5
/cherrypick release-7.1
/cherrypick release-6.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

close #52657

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

close #52657

…ingcap#52816) close pingcap#52657

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

close #52657

ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none sig/planner SIG: Planner size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked labels Apr 17, 2024

hawkingrei requested review from winoros, elsa0520 and Rustin170506 April 17, 2024 07:26

winoros approved these changes Apr 17, 2024

View reviewed changes

ti-chi-bot bot added approved needs-1-more-lgtm labels Apr 17, 2024

Rustin170506 reviewed Apr 17, 2024

View reviewed changes

pkg/statistics/handle/syncload/stats_syncload_test.go Outdated Show resolved Hide resolved

Rustin170506 reviewed Apr 17, 2024

View reviewed changes

pkg/statistics/handle/syncload/stats_syncload.go Outdated Show resolved Hide resolved

ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 17, 2024

hawkingrei added 4 commits April 18, 2024 14:27

statistics: add upper bound of retry for sync load

ba41bc9

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

statistics: add upper bound of retry for sync load

6d906cd

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

update

551c02c

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

update

72fa39d

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

hawkingrei force-pushed the avoid_cannot_stop_retry branch from 23374db to 3b57e4c Compare April 18, 2024 06:28

update

2f3c90c

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

hawkingrei force-pushed the avoid_cannot_stop_retry branch from 51f350d to 2f3c90c Compare April 18, 2024 06:55

hawkingrei commented Apr 18, 2024

View reviewed changes

update

f2d5def

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

hawkingrei requested a review from AilinKid April 22, 2024 06:40

hawkingrei added 2 commits April 22, 2024 18:41

update

fee47d8

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

update

e9d8b6f

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

Rustin170506 approved these changes Apr 22, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm labels Apr 22, 2024

ti-chi-bot bot merged commit cfcc770 into pingcap:master Apr 22, 2024
12 checks passed

hawkingrei deleted the avoid_cannot_stop_retry branch April 22, 2024 15:04

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Apr 22, 2024

This is an automated cherry-pick of pingcap#52658

27d7681

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Apr 22, 2024

statistics: add upper bound of retry for sync load (#52658) #52816

Merged

13 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Apr 22, 2024

This is an automated cherry-pick of pingcap#52658

15b2ad8

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Apr 22, 2024

statistics: add upper bound of retry for sync load (#52658) #52817

Merged

13 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Apr 22, 2024

This is an automated cherry-pick of pingcap#52658

d3e5ae3

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Apr 22, 2024

statistics: add upper bound of retry for sync load (#52658) #52818

Merged

13 tasks

ti-chi-bot mentioned this pull request Apr 22, 2024

statistics: add upper bound of retry for sync load (#52658) #52819

Merged

13 tasks

ti-chi-bot bot pushed a commit that referenced this pull request Apr 23, 2024

statistics: add upper bound of retry for sync load (#52658) (#52819)

cfeca9b

close #52657

ti-chi-bot bot pushed a commit that referenced this pull request May 16, 2024

statistics: add upper bound of retry for sync load (#52658) (#52818)

d9788b9

close #52657

hawkingrei added a commit to ti-chi-bot/tidb that referenced this pull request Jun 5, 2024

This is an automated cherry-pick of pingcap#52658

b9ef08c

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot bot pushed a commit that referenced this pull request Jun 5, 2024

statistics: add upper bound of retry for sync load (#52658) (#52816)

55bccae

close #52657

hawkingrei pushed a commit to hawkingrei/tidb that referenced this pull request Jul 1, 2024

statistics: add upper bound of retry for sync load (pingcap#52658) (p…

30911a2

…ingcap#52816) close pingcap#52657

hawkingrei added a commit to ti-chi-bot/tidb that referenced this pull request Aug 28, 2024

This is an automated cherry-pick of pingcap#52658

551e603

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot bot pushed a commit that referenced this pull request Aug 28, 2024

statistics: add upper bound of retry for sync load (#52658) (#52817)

3764806

close #52657

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

statistics: add upper bound of retry for sync load #52658

statistics: add upper bound of retry for sync load #52658

hawkingrei commented Apr 17, 2024 •

edited

Loading

codecov bot commented Apr 17, 2024 •

edited

Loading

hawkingrei commented Apr 17, 2024

hawkingrei commented Apr 17, 2024

hawkingrei commented Apr 17, 2024

hawkingrei Apr 18, 2024

hawkingrei commented Apr 22, 2024

ti-chi-bot bot commented Apr 22, 2024

ti-chi-bot bot commented Apr 22, 2024

hawkingrei commented Apr 22, 2024

ti-chi-bot commented Apr 22, 2024

ti-chi-bot commented Apr 22, 2024

ti-chi-bot commented Apr 22, 2024

ti-chi-bot commented Apr 22, 2024

statistics: add upper bound of retry for sync load #52658

statistics: add upper bound of retry for sync load #52658

Conversation

hawkingrei commented Apr 17, 2024 • edited Loading

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

codecov bot commented Apr 17, 2024 • edited Loading

Codecov Report

hawkingrei commented Apr 17, 2024

hawkingrei commented Apr 17, 2024

hawkingrei commented Apr 17, 2024

hawkingrei Apr 18, 2024

Choose a reason for hiding this comment

hawkingrei commented Apr 22, 2024

ti-chi-bot bot commented Apr 22, 2024

ti-chi-bot bot commented Apr 22, 2024

[LGTM Timeline notifier]

hawkingrei commented Apr 22, 2024

ti-chi-bot commented Apr 22, 2024

ti-chi-bot commented Apr 22, 2024

ti-chi-bot commented Apr 22, 2024

ti-chi-bot commented Apr 22, 2024

hawkingrei commented Apr 17, 2024 •

edited

Loading

codecov bot commented Apr 17, 2024 •

edited

Loading