Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plan: refactor the code of building `Insert`. #7068

Merged
merged 9 commits into from Jul 25, 2018

Conversation

@winoros
Copy link
Member

commented Jul 17, 2018

What have you changed? (mandatory)

Refactor the code when building Insert plan.
To fix some bugs make code more maintainable.
Fix #7064
Fix #7061
And another bug:
the table have 3 columns.
insert into t values(default, default, default, default) will make tidb panicked.

Main change:
We used to check the length of value_list and column_list when executing. Now we move this change to planner.

What is the type of the changes? (mandatory)

  • Refactor
  • Improvement
  • Bug fix

How has this PR been tested? (mandatory)

existing tests and newly added tests.

Does this PR need to be added to the release notes? (mandatory)

Fix the behavior of INSERT statement in some scenarios.

@jackysp

This comment has been minimized.

Copy link
Member

commented Jul 17, 2018

Please fix CI @winoros

@winoros winoros force-pushed the winoros:insert-plan-refactor branch from 2e963da to 8d99c66 Jul 17, 2018

@winoros

This comment has been minimized.

Copy link
Member Author

commented Jul 17, 2018

/run-all-tests

@winoros

This comment has been minimized.

Copy link
Member Author

commented Jul 17, 2018

/run-integration-common-test tidb-test=pr/589

@winoros

This comment has been minimized.

Copy link
Member Author

commented Jul 17, 2018

/run-integration-ddl-test tidb-test=pr/589

@winoros

This comment has been minimized.

Copy link
Member Author

commented Jul 17, 2018

Integration-common-test is failed because the bug in tikv.
Now it's ready to review.

@@ -1332,7 +1332,7 @@ func (s *testPlanSuite) TestVisitInfo(c *C) {
ans []visitInfo
}{
{
sql: "insert into t values (1)",

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 19, 2018

Member

Does the old test result in an error after this change?

This comment has been minimized.

Copy link
@winoros

winoros Jul 24, 2018

Author Member

Yes, this the length of value_list is not same with the number of table's columns.

return onDupColSet, dupCols, nil
}

func (p *Insert) getAffectCols(insert *ast.InsertStmt) (affectedValuesCols []*table.Column, err error) {

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 19, 2018

Member

I think it's better to make the receiver of this function be the planBuilder.

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 19, 2018

Member

how about s/getAffectCols/getInsertColumns/?

}

} else if len(insert.Setlist) == 0 {
// Branch for `INSERT INTO tbl_name {VALUES | VALUE} (value_list) [, (value_list)] ...`,

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 19, 2018

Member

How about:

// This is for the following scenario:
// 1. `INSERT INTO tbl_name {VALUES | VALUE} ...`
// 2. `INSERT INTO tbl_name SELECT ...`

func (p *Insert) getAffectCols(insert *ast.InsertStmt) (affectedValuesCols []*table.Column, err error) {
if len(insert.Columns) > 0 {
// Branch for `INSERT INTO tbl_name (col_name [, col_name] ...) {VALUES | VALUE} (value_list) [, (value_list)] ...`,

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 19, 2018

Member

How about:

// This is for the following scenario:
// 1. `INSERT INTO tbl_name (col_name [, col_name] ...) {VALUES | VALUE} ...`
// 2. `INSERT INTO tbl_name (col_name [, col_name] ...) SELECT ...`
}
}
// If the value_list and col_list is empty and we have generated column, we can still write to this table.

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 19, 2018

Member

This comment should reside to if len(insert.Columns) == 0 && len(insert.Lists[0]) == 0 branch? Maybe we can remove it?

This comment has been minimized.

Copy link
@winoros

winoros Jul 24, 2018

Author Member

Yes, it's for the else branch, explaining that else branch doesn't need to do anything.
I think it's useful. But maybe we can move it other place.

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 25, 2018

Member

I think we can put this comment to the if len(insert.Columns) > 0 || len(insert.Lists[0]) > 0 { to explain why we don't check whether there is generated column in the table when the value_list and col_list is empty.

// "insert into t values (), (1)" is not valid.
// "insert into t values (1), ()" is not valid.
// "insert into t values (1,2), (1)" is not valid.
if i > 0 && len(insert.Lists[i-1]) != len(valuesItem) {

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 19, 2018

Member

Maybe this form can help reading the code:

if i > 0 && len(insert.Lists[i]) != len(insert.Lists[i-1])
for j, valueItem := range valuesItem {
var expr expression.Expression
var err error
if dft, ok := valueItem.(*ast.DefaultExpr); ok {

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 19, 2018

Member

Usually the abbreviation for default is def, see https://www.abbreviations.com/abbreviation/Default

for j, valueItem := range valuesItem {
var expr expression.Expression
var err error
if dft, ok := valueItem.(*ast.DefaultExpr); ok {

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 19, 2018

Member

I think the "type switch" is more suitable.

return nil
totalTableCols := insertPlan.Table.Cols()
for i, valuesItem := range insert.Lists {
// The length of the value_list should keep unchanged.

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 19, 2018

Member

How about:

// The lengths of all the value lists should be the same, e.g:
if err != nil {
b.err = errors.Trace(err)
return nil
// Check that there's no generated column.

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 20, 2018

Member

@CaitinChen @lilin90 PTAL this comment, thanks

This comment has been minimized.

Copy link
@CaitinChen

CaitinChen Jul 20, 2018

Member

Check to guarantee that no generated column exists.

@jackysp
Copy link
Member

left a comment

LGTM

@winoros

This comment has been minimized.

Copy link
Member Author

commented Jul 24, 2018

PTAL @zz-jason

// i.e. table t have two column a and b where b is generated column.
// `insert into t (b) select * from t` will raise a error which tells you that the column count is not matched.
// `insert into t select * from t` will raise a error which tells you that there's generated column in column list.
// If we do this check before the above one, the first example will raise the error same with the second example.

This comment has been minimized.

Copy link
@zz-jason

zz-jason Jul 24, 2018

Member

@CaitinChen Could help us to refine this comment block 😂

This comment has been minimized.

Copy link
@CaitinChen

CaitinChen Jul 24, 2018

Member

If we do this check before the above one, "insert into t (b) select * from t" will raise an error that there's a generated column in the column list.
Note: I don't use "the first example" and "the second example" because there is only one example. "insert into t (b) select * from t" and "insert into t select * from t" are not named as two examples. Both of them are in this example.

return nil
// Check to guarantee that there's no generated column.
// This check is done after the above one is to make compatible with mysql.
// i.e. table t have two column a and b where b is generated column.

This comment has been minimized.

Copy link
@CaitinChen

CaitinChen Jul 24, 2018

Member

For example, table t has two columns, namely a and b, and b is a generated column.

This comment has been minimized.

Copy link
@CaitinChen

CaitinChen Jul 24, 2018

Member

Note: "i.e." means "that is to say" or "namely".

// This check is done after the above one is to make compatible with mysql.
// i.e. table t have two column a and b where b is generated column.
// `insert into t (b) select * from t` will raise a error which tells you that the column count is not matched.
// `insert into t select * from t` will raise a error which tells you that there's generated column in column list.

This comment has been minimized.

Copy link
@CaitinChen

CaitinChen Jul 24, 2018

Member

"insert into t select * from t" will raise an error that there's a generated column in the column list.

// Check to guarantee that there's no generated column.
// This check is done after the above one is to make compatible with mysql.
// i.e. table t have two column a and b where b is generated column.
// `insert into t (b) select * from t` will raise a error which tells you that the column count is not matched.

This comment has been minimized.

Copy link
@CaitinChen

CaitinChen Jul 24, 2018

Member

"insert into t (b) select * from t" will raise an error that the column count is not matched.

Note: I use "insert into t (b) select * from t" instead of insert into t (b) select * from t because the inline code format will not be displayed in the code comments as expected.

// i.e. table t have two column a and b where b is generated column.
// `insert into t (b) select * from t` will raise a error which tells you that the column count is not matched.
// `insert into t select * from t` will raise a error which tells you that there's generated column in column list.
// If we do this check before the above one, the first example will raise the error same with the second example.

This comment has been minimized.

Copy link
@CaitinChen

CaitinChen Jul 24, 2018

Member

If we do this check before the above one, "insert into t (b) select * from t" will raise an error that there's a generated column in the column list.
Note: I don't use "the first example" and "the second example" because there is only one example. "insert into t (b) select * from t" and "insert into t select * from t" are not named as two examples. Both of them are in this example.

b.err = errors.Trace(err)
return nil
// Check to guarantee that there's no generated column.
// This check is done after the above one is to make compatible with mysql.

This comment has been minimized.

Copy link
@CaitinChen

CaitinChen Jul 24, 2018

Member

This check should be done after the above one to make its behavior compatible with MySQL.

@zz-jason
Copy link
Member

left a comment

LGTM

@winoros

This comment has been minimized.

Copy link
Member Author

commented Jul 25, 2018

@jackysp PTAL

@jackysp
Copy link
Member

left a comment

LGTM

@jackysp

This comment has been minimized.

Copy link
Member

commented Jul 25, 2018

/run-all-tests

@jackysp jackysp added the status/LGT2 label Jul 25, 2018

@winoros

This comment has been minimized.

Copy link
Member Author

commented Jul 25, 2018

/run-integration-ddl-test tidb-test=pr/589

@winoros winoros merged commit b5f9b35 into pingcap:master Jul 25, 2018

11 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci-tidb/build Jenkins job succeeded.
Details
jenkins-ci-tidb/common-test Jenkins job succeeded.
Details
jenkins-ci-tidb/integration-common-test Jenkins job succeeded.
Details
jenkins-ci-tidb/integration-compatibility-test Jenkins job succeeded.
Details
jenkins-ci-tidb/integration-ddl-test Jenkins job succeeded.
Details
jenkins-ci-tidb/mybatis-test Jenkins job succeeded.
Details
jenkins-ci-tidb/sqllogic-test Jenkins job succeeded.
Details
jenkins-ci-tidb/unit-test Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details

@winoros winoros deleted the winoros:insert-plan-refactor branch Jul 25, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.