Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rule-checker does not consider the engine label constraint #6662

Closed
JmPotato opened this issue Jun 24, 2023 · 1 comment · Fixed by #6660
Closed

rule-checker does not consider the engine label constraint #6662

JmPotato opened this issue Jun 24, 2023 · 1 comment · Fixed by #6660
Labels
affects-7.1 severity/critical The issue's severity is critical. type/bug The issue is confirmed as a bug.
Projects
Questions and Bug Reports

Comments

@JmPotato
Copy link
Member

JmPotato commented Jun 24, 2023 •
edited

Bug Report

#6633 broke the constrain that we can only move peer between the stores that satisfy the rule, as a result, rule-checker will move the learner peer away to a TiKV store while keep adding rule peer in the TiFlash stores. This will cause a TiFlash replica of a region never to be replicated well while bringing many redundant replicas to other TiKV nodes.

b4e2acdf-68ad-42b4-86e8-540e759e45b9

In monitoring, its phenomenon may be as shown in the figure above, with a large number of add-rule-peer and move-to-better-location at the same time. And in the log, you can see that the two act in the same region successively.
@JmPotato JmPotato added the type/bug The issue is confirmed as a bug. label Jun 24, 2023
@github-actions github-actions bot added this to Need Triage in Questions and Bug Reports Jun 24, 2023
@JmPotato JmPotato moved this from Need Triage to Fix: High Priority in Questions and Bug Reports Jun 24, 2023
@JmPotato JmPotato added the severity/critical The issue's severity is critical. label Jun 24, 2023
This was referenced Jun 24, 2023
Merged
Open
Questions and Bug Reports automation moved this from Fix: High Priority to Closed Jun 25, 2023
ti-chi-bot bot pushed a commit that referenced this issue Jun 25, 2023
@rleungx
checker: fix the conflict between tiflash learner and location labels (…
f4d774a 
…#6660)

close #6662

Signed-off-by: Ryan Leung <rleungx@gmail.com>
@JmPotato JmPotato mentioned this issue Jun 25, 2023
Merged
@rleungx rleungx reopened this Jul 10, 2023
Questions and Bug Reports automation moved this from Closed to Need Triage Jul 10, 2023
rleungx added a commit to rleungx/pd that referenced this issue Jul 10, 2023
@rleungx
checker: fix the conflict between tiflash learner and location labels (…
1f0f401 
…tikv#6660)

close tikv#6662

Signed-off-by: Ryan Leung <rleungx@gmail.com>
@rleungx rleungx mentioned this issue Jul 10, 2023
Merged
ti-chi-bot bot pushed a commit that referenced this issue Jul 10, 2023
@rleungx
checker: fix the conflict between tiflash learner and location labels (…
ffe7b9b 
…#6660) (#6778)

close #6662

Signed-off-by: Ryan Leung <rleungx@gmail.com>
@nolouch nolouch closed this as completed Jul 10, 2023
Questions and Bug Reports automation moved this from Need Triage to Closed Jul 10, 2023
@ti-chi-bot ti-chi-bot mentioned this issue Aug 14, 2023
Merged
ti-chi-bot pushed a commit to ti-chi-bot/pd that referenced this issue Aug 14, 2023
@rleungx @ti-chi-bot
This is an automated cherry-pick of tikv#6660
85765d0 
close tikv#6662

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@rleungx
Copy link
Member

rleungx commented Aug 14, 2023

We don't need to pick this fix to release-6.5 since it doesn't support survival preference in the placement rule in SQL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects-7.1 severity/critical The issue's severity is critical. type/bug The issue is confirmed as a bug.
Projects
Questions and Bug Reports
  
Closed
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

checker: fix the conflict between tiflash learner and location labels rleungx/pd
4 participants
@JmPotato @nolouch @rleungx @ti-chi-bot