Fix: reserved connection retry logic when vttablet or mysql is down #10005
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
deepthi
reviewed
Mar 29, 2022
go/vt/vtgate/scatter_conn.go
Outdated
| if err != nil { | ||
| return nil, err | ||
| } | ||
| qs = rs.Gateway |
There was a problem hiding this comment.
Is the idea here that we route it through the gateway again and it will pick a tablet for us?
That's clever 💯
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
…ablet is down then it should go through gateway Signed-off-by: Harshit Gangal <harshit@planetscale.com>
… if needed and not executeLock Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
… healthy tablet for the given tablet_type Signed-off-by: Harshit Gangal <harshit@planetscale.com>
harshit-gangal
force-pushed
the
fix-rconn-retry
branch
from
37eac82
to
2b89ecd
Compare
harshit-gangal
requested review from
systay,
shlomi-noach and
frouioui
as code owners
systay
reviewed
Mar 31, 2022
systay
reviewed
Mar 31, 2022
| } | ||
|
|
||
| func requireNewQS(err error, target *querypb.Target) bool { | ||
| code := vterrors.Code(err) | ||
| msg := err.Error() | ||
| return (code == vtrpcpb.Code_FAILED_PRECONDITION && vterrors.RxWrongTablet.MatchString(msg)) || | ||
| (code == vtrpcpb.Code_CLUSTER_EVENT && ((target != nil && target.TabletType == topodatapb.TabletType_PRIMARY) || vterrors.RxOp.MatchString(msg))) | ||
| if (code == vtrpcpb.Code_FAILED_PRECONDITION && vterrors.RxWrongTablet.MatchString(msg)) || |
There was a problem hiding this comment.
nit: maybe this could benefit from the same treatment as ☝️
systay
approved these changes
Mar 31, 2022
harshit-gangal
added a commit
to planetscale/vitess
that referenced
this pull request
Apr 7, 2022
…itessio#10005) * fix: when mysql is down and vttablet is up Signed-off-by: Harshit Gangal <harshit@planetscale.com> * fix: in reserved conn with no transaction if the queryservice i.e vttablet is down then it should go through gateway Signed-off-by: Harshit Gangal <harshit@planetscale.com> * refactor: execute and streamExecute should be able to reset the shard if needed and not executeLock Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: e2e test for bug fix Signed-off-by: Harshit Gangal <harshit@planetscale.com> * fix: when mysql is down, query should go through gateway to elect new healthy tablet for the given tablet_type Signed-off-by: Harshit Gangal <harshit@planetscale.com>
2 tasks
frouioui
pushed a commit
that referenced
this pull request
Apr 7, 2022
…10005) (#10052) * fix: when mysql is down and vttablet is up Signed-off-by: Harshit Gangal <harshit@planetscale.com> * fix: in reserved conn with no transaction if the queryservice i.e vttablet is down then it should go through gateway Signed-off-by: Harshit Gangal <harshit@planetscale.com> * refactor: execute and streamExecute should be able to reset the shard if needed and not executeLock Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: e2e test for bug fix Signed-off-by: Harshit Gangal <harshit@planetscale.com> * fix: when mysql is down, query should go through gateway to elect new healthy tablet for the given tablet_type Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes issues with reserved connection when the vttablet or the underlying MySQL process is down.
The VTGate needs to understand the error codes/messages to redirect it to the new available tablet.
In all the above scenarios the current reserved connection held should be dropped and a new one should be created by calling the ReserveExecute API to the tablet.
Related Issue(s)
Checklist
Deployment Notes