[CARBONDATA-3628] Support alter hive table add complex column type #3529
Conversation
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1279/ |
|
Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1268/ |
|
Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1258/ |
...ter-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/AlterTableTestCase.scala
Show resolved
Hide resolved
|
Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1261/ |
|
Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1271/ |
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1282/ |
| sql("create table alter_hive(name string) stored as rcfile partitioned by (dt string)") | ||
| if (SparkUtil.isSparkVersionXandAbove("2.2")) { | ||
| sql("alter table alter_hive add columns(add string)") | ||
| sql("alter table alter_hive add columns (var map<string, string>)") |
There was a problem hiding this comment.
Can you please add a case with STRUCT Datatype also
There was a problem hiding this comment.
It does not work with default STRUCT Datatype (struct<>)
There was a problem hiding this comment.
why it doesn't work with Struct type?
There was a problem hiding this comment.
DataTypeUtil.valueOf just return default struct type (struct<>), here should work more for this.
There was a problem hiding this comment.
If this PR is just to support map, please modify the PR title and description.
To support struct type, need another PR?
There was a problem hiding this comment.
Yes, this PR only support map<string, string>, array<string>, struct<> currently. I can do some work to fully support this.
There was a problem hiding this comment.
Ok, I will merge this PR. Please raise another JIRA ticket for the struct support. Thanks
integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonMergerRDD.scala
Show resolved
Hide resolved
|
Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1273/ |
|
Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1283/ |
|
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1294/ |
|
LGTM |
Support adding array and map data type column by ALTER TABLE This closes apache#3529
Support adding array and map data type column by ALTER TABLE This closes apache#3529
Support adding array and map data type column by ALTER TABLE This closes apache#3529
Tips:
see DataTypeUtil#valueOfBe sure to do all of the following checklists to help us incorporate
your contribution quickly and easily:
Any interfaces changed?
Any backward compatibility impacted?
Document update required?
Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach the test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.