Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner, executor: support Change and ChangeExec #9789

Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
e94d660
support update pump or drainer status.
aliiohs Mar 19, 2019
249958f
fix
aliiohs Mar 19, 2019
fabd472
fix
aliiohs Mar 19, 2019
32517d1
fix
aliiohs Mar 19, 2019
b984d2d
fix
aliiohs Mar 19, 2019
d6f6000
rm go.sum
aliiohs Mar 19, 2019
e62f63a
fix
aliiohs Mar 19, 2019
dc09862
fix
aliiohs Mar 19, 2019
e2947b1
unless a blank line.
aliiohs Mar 19, 2019
b503dfe
add go.sum
aliiohs Mar 20, 2019
b723f42
Merge branch 'master' into Support_update_pump_or_drainer_status_lastest
WangXiangUSTC Mar 20, 2019
d045a56
fix
aliiohs Mar 20, 2019
180271d
Merge remote-tracking branch 'origin/Support_update_pump_or_drainer_s…
aliiohs Mar 20, 2019
2c5848a
1. use the context from Next.
aliiohs Mar 20, 2019
ef7f7f9
fix
aliiohs Mar 20, 2019
4349d6b
fix
aliiohs Mar 20, 2019
dafee9d
fix
aliiohs Mar 20, 2019
d37a807
fix
aliiohs Mar 20, 2019
b068758
Merge branch 'master' into Support_update_pump_or_drainer_status_lastest
WangXiangUSTC Mar 20, 2019
6511ac2
add metrics counter for 'change'
aliiohs Mar 20, 2019
5bd6683
Merge remote-tracking branch 'origin/Support_update_pump_or_drainer_s…
aliiohs Mar 20, 2019
b857671
Merge branch 'master' into Support_update_pump_or_drainer_status_lastest
WangXiangUSTC Mar 21, 2019
5640a97
merge updateNodeState method into Next
aliiohs Mar 21, 2019
4dfcdf3
Merge remote-tracking branch 'origin/Support_update_pump_or_drainer_s…
aliiohs Mar 21, 2019
e9d2a96
support update pump or drainer status.
aliiohs Mar 19, 2019
2decbc6
fix
aliiohs Mar 19, 2019
7f08429
fix
aliiohs Mar 19, 2019
4fde154
fix
aliiohs Mar 19, 2019
0f43b59
fix
aliiohs Mar 19, 2019
5525a92
fix
aliiohs Mar 19, 2019
46dca32
unless a blank line.
aliiohs Mar 19, 2019
85d1b6d
add go.sum
aliiohs Mar 20, 2019
81fe589
fix
aliiohs Mar 20, 2019
c0821c4
1. use the context from Next.
aliiohs Mar 20, 2019
43a3c53
fix
aliiohs Mar 20, 2019
12cca72
fix
aliiohs Mar 20, 2019
b0b5d93
fix
aliiohs Mar 20, 2019
8c1b00f
fix
aliiohs Mar 20, 2019
458ecce
add metrics counter for 'change'
aliiohs Mar 20, 2019
d61af1a
merge updateNodeState method into Next
aliiohs Mar 21, 2019
5d83bb8
fix
aliiohs Mar 21, 2019
ae18f5c
fix
aliiohs Mar 21, 2019
7fe3ad1
fix
aliiohs Mar 21, 2019
e964c41
format code
WangXiangUSTC Mar 21, 2019
fd18583
Merge branch 'master' into Support_update_pump_or_drainer_status_lastest
WangXiangUSTC Mar 21, 2019
2a92897
Merge branch 'master' into Support_update_pump_or_drainer_status_lastest
WangXiangUSTC Mar 21, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 8 additions & 0 deletions executor/builder.go
Expand Up @@ -78,6 +78,8 @@ func (b *executorBuilder) build(p plannercore.Plan) Executor {
switch v := p.(type) {
case nil:
return nil
case *plannercore.Change:
return b.buildChange(v)
case *plannercore.CheckTable:
return b.buildCheckTable(v)
case *plannercore.CheckIndex:
Expand Down Expand Up @@ -207,6 +209,12 @@ func (b *executorBuilder) buildCancelDDLJobs(v *plannercore.CancelDDLJobs) Execu
return e
}

func (b *executorBuilder) buildChange(v *plannercore.Change) Executor {
WangXiangUSTC marked this conversation as resolved.
Show resolved Hide resolved
return &ChangeExec{
Statement: v.Statement,
}
}

func (b *executorBuilder) buildShowNextRowID(v *plannercore.ShowNextRowID) Executor {
e := &ShowNextRowIDExec{
baseExecutor: newBaseExecutor(b.ctx, v.Schema(), v.ExplainID()),
Expand Down
62 changes: 62 additions & 0 deletions executor/change.go
@@ -0,0 +1,62 @@
// Copyright 2019 PingCAP, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
WangXiangUSTC marked this conversation as resolved.
Show resolved Hide resolved
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// See the License for the specific language governing permissions and
// limitations under the License.

package executor

import (
"context"
WangXiangUSTC marked this conversation as resolved.
Show resolved Hide resolved
"strings"

"github.com/pingcap/errors"
"github.com/pingcap/parser/ast"
"github.com/pingcap/tidb-tools/tidb-binlog/node"
"github.com/pingcap/tidb/config"
"github.com/pingcap/tidb/util/chunk"
)

// ChangeExec represents a change executor.
type ChangeExec struct {
baseExecutor

Statement *ast.ChangeStmt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can remove this, and only store kind, state, nodeID.

}

// Next implements the Executor Next interface.
func (e *ChangeExec) Next(ctx context.Context, req *chunk.RecordBatch) error {
stmt := e.Statement
kind := strings.ToLower(stmt.NodeType)
urls := config.GetGlobalConfig().Path
registry, err := createRegistry(urls)
if err != nil {
return err
}
nodes, _, err := registry.Nodes(ctx, node.NodePrefix[kind])
if err != nil {
return err
}
state := stmt.State
nodeID := stmt.NodeID
for _, n := range nodes {
if n.NodeID != nodeID {
continue
}
switch state {
case node.Online, node.Pausing, node.Paused, node.Closing, node.Offline:
n.State = state
return registry.UpdateNode(ctx, node.NodePrefix[kind], n)
WangXiangUSTC marked this conversation as resolved.
Show resolved Hide resolved
default:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be checked eagerly. We check it with parser may be better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, we will update parser, and then create another pr to remove these judgement.

return errors.Errorf("state %s is illegal", state)
}
}
return errors.NotFoundf("node %s, id %s from etcd %s", kind, nodeID, urls)
}
2 changes: 2 additions & 0 deletions executor/compiler.go
Expand Up @@ -242,6 +242,8 @@ func GetStmtLabel(stmtNode ast.StmtNode) string {
return "AnalyzeTable"
case *ast.BeginStmt:
return "Begin"
case *ast.ChangeStmt:
return "Change"
case *ast.CommitStmt:
return "Commit"
case *ast.CreateDatabaseStmt:
Expand Down
4 changes: 2 additions & 2 deletions go.mod
Expand Up @@ -51,7 +51,7 @@ require (
github.com/pingcap/goleveldb v0.0.0-20171020122428-b9ff6c35079e
github.com/pingcap/kvproto v0.0.0-20190215154024-7f2fc73ef562
github.com/pingcap/log v0.0.0-20190307075452-bd41d9273596
github.com/pingcap/parser v0.0.0-20190312024907-3f6280b08c8b
github.com/pingcap/parser v0.0.0-20190318135537-2575f2d0ba4b
github.com/pingcap/pd v2.1.0-rc.4+incompatible
github.com/pingcap/tidb-tools v2.1.3-0.20190116051332-34c808eef588+incompatible
github.com/pingcap/tipb v0.0.0-20190107072121-abbec73437b7
Expand Down Expand Up @@ -82,7 +82,7 @@ require (
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e
golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb // indirect
golang.org/x/text v0.3.0
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c // indirect
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4 // indirect
golang.org/x/tools v0.0.0-20190130214255-bb1329dc71a0
google.golang.org/genproto v0.0.0-20190108161440-ae2f86662275 // indirect
google.golang.org/grpc v1.17.0
Expand Down
9 changes: 4 additions & 5 deletions go.sum
Expand Up @@ -107,7 +107,6 @@ github.com/philhofer/fwd v1.0.0 h1:UbZqGr5Y38ApvM/V/jEljVxwocdweyH+vmYvRPBnbqQ=
github.com/philhofer/fwd v1.0.0/go.mod h1:gk3iGcWd9+svBvR0sR+KPcfE+RNWozjowpeBVG3ZVNU=
github.com/pingcap/check v0.0.0-20190102082844-67f458068fc8 h1:USx2/E1bX46VG32FIw034Au6seQ2fY9NEILmNh/UlQg=
github.com/pingcap/check v0.0.0-20190102082844-67f458068fc8/go.mod h1:B1+S9LNcuMyLH/4HMTViQOJevkGiik3wW2AN9zb2fNQ=
github.com/pingcap/errors v0.11.0 h1:DCJQB8jrHbQ1VVlMFIrbj2ApScNNotVmkSNplu2yUt4=
github.com/pingcap/errors v0.11.0/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8=
github.com/pingcap/errors v0.11.1 h1:BXFZ6MdDd2U1uJUa2sRAWTmm+nieEzuyYM0R4aUTcC8=
github.com/pingcap/errors v0.11.1/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8=
Expand All @@ -119,8 +118,8 @@ github.com/pingcap/kvproto v0.0.0-20190215154024-7f2fc73ef562 h1:32oF1/8lVnBR2JV
github.com/pingcap/kvproto v0.0.0-20190215154024-7f2fc73ef562/go.mod h1:QMdbTAXCHzzygQzqcG9uVUgU2fKeSN1GmfMiykdSzzY=
github.com/pingcap/log v0.0.0-20190307075452-bd41d9273596 h1:t2OQTpPJnrPDGlvA+3FwJptMTt6MEPdzK1Wt99oaefQ=
github.com/pingcap/log v0.0.0-20190307075452-bd41d9273596/go.mod h1:WpHUKhNZ18v116SvGrmjkA9CBhYmuUTKL+p8JC9ANEw=
github.com/pingcap/parser v0.0.0-20190312024907-3f6280b08c8b h1:NlvTrxqezIJh6CD5Leky12IZ8E/GtpEEmzgNNb34wbw=
github.com/pingcap/parser v0.0.0-20190312024907-3f6280b08c8b/go.mod h1:1FNvfp9+J0wvc4kl8eGNh7Rqrxveg15jJoWo/a0uHwA=
github.com/pingcap/parser v0.0.0-20190318135537-2575f2d0ba4b h1:cfmy3QgCQkjjsHzw3HgWLZYg62MBLXSRXHN6TcsKhSA=
github.com/pingcap/parser v0.0.0-20190318135537-2575f2d0ba4b/go.mod h1:1FNvfp9+J0wvc4kl8eGNh7Rqrxveg15jJoWo/a0uHwA=
github.com/pingcap/pd v2.1.0-rc.4+incompatible h1:/buwGk04aHO5odk/+O8ZOXGs4qkUjYTJ2UpCJXna8NE=
github.com/pingcap/pd v2.1.0-rc.4+incompatible/go.mod h1:nD3+EoYes4+aNNODO99ES59V83MZSI+dFbhyr667a0E=
github.com/pingcap/tidb-tools v2.1.3-0.20190116051332-34c808eef588+incompatible h1:e9Gi/LP9181HT3gBfSOeSBA+5JfemuE4aEAhqNgoE4k=
Expand Down Expand Up @@ -209,8 +208,8 @@ golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb h1:1w588/yEchbPNpa9sEvOcMZYb
golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c h1:fqgJT0MGcGpPgpWU7VRdRjuArfcOvC4AoJmILihzhDg=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4 h1:SvFZT6jyqRaOeXpc5h/JSfZenJ2O330aBsf7JfSUXmQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190130214255-bb1329dc71a0 h1:iRpjPej1fPzmfoBhMFkp3HdqzF+ytPmAwiQhJGV0zGw=
golang.org/x/tools v0.0.0-20190130214255-bb1329dc71a0/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
Expand Down
7 changes: 7 additions & 0 deletions planner/core/common_plans.go
Expand Up @@ -133,6 +133,13 @@ type CancelDDLJobs struct {
JobIDs []int64
}

// Change represents a change plan.
type Change struct {
baseSchemaProducer
zz-jason marked this conversation as resolved.
Show resolved Hide resolved

Statement *ast.ChangeStmt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will make the code more concise

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can define it as :
type Change struct {
baseSchemaProducer
*ast.ChangeStmt
}

Thus we can use e.xxx to without stmt := e.Statement

}

// Prepare represents prepare plan.
type Prepare struct {
baseSchemaProducer
Expand Down
7 changes: 7 additions & 0 deletions planner/core/planbuilder.go
Expand Up @@ -217,10 +217,17 @@ func (b *PlanBuilder) Build(node ast.Node) (Plan, error) {
return b.buildSimple(node.(ast.StmtNode))
case ast.DDLNode:
return b.buildDDL(x)
case *ast.ChangeStmt:
return b.buildChange(x)
}
return nil, ErrUnsupportedType.GenWithStack("Unsupported type %T", node)
}

func (b *PlanBuilder) buildChange(v *ast.ChangeStmt) (Plan, error) {
WangXiangUSTC marked this conversation as resolved.
Show resolved Hide resolved
exe := &Change{Statement: v}
return exe, nil
WangXiangUSTC marked this conversation as resolved.
Show resolved Hide resolved
}

func (b *PlanBuilder) buildExecute(v *ast.ExecuteStmt) (Plan, error) {
vars := make([]expression.Expression, 0, len(v.UsingVars))
for _, expr := range v.UsingVars {
Expand Down