*: fix member race problem #6070

rleungx · 2023-03-02T06:09:23Z

What problem does this PR solve?

Issue Number: Ref #6069.

What is changed and how does it work?

#5310 did some changes to the header but for GetMembers, we may encounter a kind of situation: The server hasn't served but the request is coming. Because the cluster id could not be initialized, there might be a data race when reading/writing the cluster id.

Check List

Tests

Unit test

Release note

None.

ti-chi-bot · 2023-03-02T06:09:25Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

JmPotato

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

rleungx · 2023-03-02T06:09:41Z

@HuSharp PTAL

JmPotato · 2023-03-02T06:34:27Z

server/grpc_service.go

-		return &pdpb.GetMembersResponse{
-			Header: s.wrapErrorToHeader(pdpb.ErrorType_UNKNOWN, err.Error()),
-		}, nil
+		return nil, status.Errorf(codes.Unknown, err.Error())


I think TiKV may relays on this error to retry, changing it to a gRPC error seems will change the behavior.

Yes, updated.

JmPotato

Basically LGTM.

JmPotato · 2023-03-02T06:51:16Z

server/grpc_service.go

@@ -102,7 +102,12 @@ func (s *GrpcServer) GetMembers(context.Context, *pdpb.GetMembersRequest) (*pdpb
 	members, err := s.Server.GetMembers()
 	if err != nil {
 		return &pdpb.GetMembersResponse{
-			Header: s.wrapErrorToHeader(pdpb.ErrorType_UNKNOWN, err.Error()),
+			Header: &pdpb.ResponseHeader{
+				Error: &pdpb.Error{


Will the missing cluster ID field be a problem?

I distinguish the two situations in the original s.Server.GetMembers().

codecov · 2023-03-02T06:53:21Z

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.05 ⚠️

Comparison is base (c40e319) 74.00% compared to head (63f7b4c) 73.96%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6070      +/-   ##
==========================================
- Coverage   74.00%   73.96%   -0.05%     
==========================================
  Files         383      383              
  Lines       37874    37882       +8     
==========================================
- Hits        28030    28018      -12     
- Misses       7386     7400      +14     
- Partials     2458     2464       +6

Flag	Coverage Δ
unittests	`73.96% <100.00%> (-0.05%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
server/grpc_service.go	`49.12% <100.00%> (+0.36%)`	⬆️
server/server.go	`75.00% <100.00%> (+0.09%)`	⬆️
pkg/mcs/discovery/register.go	`53.48% <0.00%> (-18.61%)`	⬇️
pkg/utils/metricutil/metricutil.go	`82.75% <0.00%> (-10.35%)`	⬇️
...erver/config/service_middleware_persist_options.go	`91.66% <0.00%> (-8.34%)`	⬇️
pkg/storage/endpoint/meta.go	`63.63% <0.00%> (-4.55%)`	⬇️
pkg/tso/local_allocator.go	`71.62% <0.00%> (-4.06%)`	⬇️
server/schedule/labeler/labeler.go	`76.77% <0.00%> (-3.88%)`	⬇️
server/schedule/schedulers/shuffle_hot_region.go	`56.66% <0.00%> (-3.34%)`	⬇️
pkg/utils/etcdutil/etcdutil.go	`78.04% <0.00%> (-2.44%)`	⬇️
... and 14 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Signed-off-by: Ryan Leung <rleungx@gmail.com>

server/grpc_service.go

Signed-off-by: Ryan Leung <rleungx@gmail.com>

server/server.go

Signed-off-by: Ryan Leung <rleungx@gmail.com>

ti-chi-bot · 2023-03-03T07:38:54Z

@HuSharp: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-03-03T19:35:39Z

@binshi-bing: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

rleungx · 2023-03-06T02:45:29Z

/run-check-issue-triage-complete

rleungx · 2023-03-06T02:45:41Z

/merge

ti-chi-bot · 2023-03-06T02:45:42Z

@rleungx: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-03-06T02:45:43Z

@rleungx: /merge in this pull request requires 2 approval(s).

In response to this:

/merge

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

rleungx · 2023-03-06T02:47:24Z

/merge

ti-chi-bot · 2023-03-06T02:47:25Z

@rleungx: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-03-06T02:47:26Z

This pull request has been accepted and is ready to merge.

Commit hash: f696bce

ti-chi-bot · 2023-03-06T02:47:39Z

@rleungx: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-03-06T02:59:48Z

In response to a cherrypick label: new pull request created to branch release-6.5: #6091.

ref #5310, ref #6069, ref #6070 Signed-off-by: Ryan Leung <rleungx@gmail.com> Co-authored-by: Ryan Leung <rleungx@gmail.com>

ti-chi-bot added do-not-merge/needs-triage-completed release-note-none labels Mar 2, 2023

ti-chi-bot requested review from JmPotato and nolouch March 2, 2023 06:09

JmPotato reviewed Mar 2, 2023

View reviewed changes

rleungx force-pushed the fix-race9 branch from 537d446 to dc02825 Compare March 2, 2023 06:44

rleungx requested a review from JmPotato March 2, 2023 06:46

JmPotato approved these changes Mar 2, 2023

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 2, 2023

fix member race problem

dec0bdc

Signed-off-by: Ryan Leung <rleungx@gmail.com>

rleungx force-pushed the fix-race9 branch from dc02825 to dec0bdc Compare March 2, 2023 07:11

HuSharp reviewed Mar 2, 2023

View reviewed changes

server/grpc_service.go Show resolved Hide resolved

address the comment

9047907

Signed-off-by: Ryan Leung <rleungx@gmail.com>

HuSharp reviewed Mar 2, 2023

View reviewed changes

server/server.go Show resolved Hide resolved

address the comment

f696bce

Signed-off-by: Ryan Leung <rleungx@gmail.com>

rleungx requested a review from HuSharp March 3, 2023 07:02

HuSharp approved these changes Mar 3, 2023

View reviewed changes

binshi-bing approved these changes Mar 3, 2023

View reviewed changes

ti-chi-bot added needs-cherry-pick-release-6.5 and removed do-not-merge/needs-triage-completed labels Mar 6, 2023

rleungx added the require-LGT1 Indicates that the PR requires an LGTM. label Mar 6, 2023

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 6, 2023

Merge branch 'master' into fix-race9

63f7b4c

ti-chi-bot merged commit 312e5b4 into tikv:master Mar 6, 2023
18 checks passed

ti-chi-bot mentioned this pull request Mar 6, 2023

*: fix member race problem (#6070) #6091

Merged

ti-chi-bot added a commit that referenced this pull request Mar 6, 2023

*: fix member race problem (#6070) (#6091)

51f382c

ref #5310, ref #6069, ref #6070 Signed-off-by: Ryan Leung <rleungx@gmail.com> Co-authored-by: Ryan Leung <rleungx@gmail.com>

rleungx deleted the fix-race9 branch May 17, 2023 03:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: fix member race problem #6070

*: fix member race problem #6070

rleungx commented Mar 2, 2023

ti-chi-bot commented Mar 2, 2023 •

edited

rleungx commented Mar 2, 2023

JmPotato Mar 2, 2023

rleungx Mar 2, 2023

JmPotato left a comment

JmPotato Mar 2, 2023

rleungx Mar 2, 2023

codecov bot commented Mar 2, 2023 •

edited

ti-chi-bot commented Mar 3, 2023

ti-chi-bot commented Mar 3, 2023

rleungx commented Mar 6, 2023

rleungx commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

rleungx commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

*: fix member race problem #6070

*: fix member race problem #6070

Conversation

rleungx commented Mar 2, 2023

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

ti-chi-bot commented Mar 2, 2023 • edited

rleungx commented Mar 2, 2023

JmPotato Mar 2, 2023

Choose a reason for hiding this comment

rleungx Mar 2, 2023

Choose a reason for hiding this comment

JmPotato left a comment

Choose a reason for hiding this comment

JmPotato Mar 2, 2023

Choose a reason for hiding this comment

rleungx Mar 2, 2023

Choose a reason for hiding this comment

codecov bot commented Mar 2, 2023 • edited

Codecov Report

ti-chi-bot commented Mar 3, 2023

ti-chi-bot commented Mar 3, 2023

rleungx commented Mar 6, 2023

rleungx commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

rleungx commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

ti-chi-bot commented Mar 6, 2023

ti-chi-bot commented Mar 2, 2023 •

edited

codecov bot commented Mar 2, 2023 •

edited