[SPARK-20845][SQL] Support specification of column names in INSERT INTO command.

[misutoth]

What changes were proposed in this pull request?

One can specify a list of columns for an INSERT INTO command. The columns shall be listed in parenthesis just following the table name. Query columns are then matched to this very same order.

scala> sql("CREATE TABLE t (s string, i int)")
scala> sql("INSERT INTO t values ('first', 1)")
scala> sql("INSERT INTO t (i, s) values (2, 'second')")
scala> sql("SELECT * FROM t").show
+------+---+
|     s|  i|
+------+---+
| first|  1|
|second|  2|
+------+---+


scala>

In the above example the second insertion utilizes the new functionality. The number and its associated string is given in reverse order (2, 'second') according to the column list specified for the table (i, s). The result can be seen at the end of the command list. Intermediate output of the commands are omitted for the sake of brevity.

How was this patch tested?

InsertSuite (both in source and in hive sub-packages) were extended with tests exercising specification of column names listing in INSERT INTO commands.

Also ran the above sample, and ran tests in sql.

[misutoth]

@janewangfb, @gatorsmile could you please possibly review this change?

[vanzin]

ok to test

[SparkQA]

Test build #97006 has finished for PR 22532 at commit 1dda672.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

Thanks for submitting the PR! I quickly scan the code changes. It sounds like the general direction is right but the quality is not ready.

I would suggest to write the test plan before doing the code review. Could you try your best to write down what we should test for supporting this feature? Both negative and positive cases.

[misutoth]

Many thanks for the feedback. I will list the test scenarios that I had in mind and collected while I implemented this item.

And sorry about the failure, seems like I did not rerun all the tests in my last step... For example when the same field is queried multiple times it is not handled properly. I will fix them also ...

[weixiuli]

Is there any progress? @misutoth

[weixiuli]

Why does it need to be overrided ?

[AmplabJenkins]

Can one of the admins verify this patch?

[HyukjinKwon]

Closing this due to author's inactivity.

[igreenfield]

@misutoth Can we push that forward? hence hive support that so thrift is not compatible?

[chrisknoll]

Our project will also not be able to use Apache Spark unless the standard insert with column names syntax is supported, so I would also be interested in this change being applied.

[igreenfield]

@weixiuli @misutoth I found bug in that PR and also have the fix. can we continue with that one?