[WIP][SPARK-28476][SQL] Support ALTER DATABASE SET LOCATION by WeichenXu123 · Pull Request #25294 · apache/spark

WeichenXu123 · 2019-07-30T01:59:23Z

What changes were proposed in this pull request?

Support the syntax of ALTER (DATABASE|SCHEMA) database_name SET LOCATION path

Ref: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL

How was this patch tested?

UT.

Please review https://spark.apache.org/contributing.html before opening a pull request.

SparkQA · 2019-07-30T05:58:00Z

Test build #108360 has finished for PR 25294 at commit 361d70f.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class AlterDatabaseLocationCommand(databaseName: String, location: String)

sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala

sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala

gatorsmile · 2019-07-30T16:30:25Z

sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLParserSuite.scala


+  test("alter database set location") {
+    // ALTER (DATABASE|SCHEMA) database_name SET LOCATION
+    val sql1 = "ALTER DATABASE database_name SET LOCATION '/home/user/db'"


This is the parser test case.

Add both negative and positive end to end test cases?

Add it to DDLSuite?

I add a positive e2e test case.
What's the negative test case you want to add ?

A database that does not exist before issuing this command?

Setting an illegal path that could trigger an exception inside the function stringToURI

sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala

gatorsmile · 2019-07-30T16:33:20Z

cc @rednaxelafx

SparkQA · 2019-07-30T18:27:28Z

Test build #108400 has finished for PR 25294 at commit 71033f8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-07-30T20:12:39Z

Test build #108408 has finished for PR 25294 at commit f3eb6e9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-07-31T03:09:22Z

Test build #108431 has finished for PR 25294 at commit bd4fb39.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class AlterDatabaseSetLocationCommand(databaseName: String, location: String)

SparkQA · 2019-07-31T11:03:25Z

Test build #108456 has finished for PR 25294 at commit 7140624.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-07-31T17:40:28Z

Test build #108473 has finished for PR 25294 at commit d209d85.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-08-01T03:10:07Z

Test build #108498 has finished for PR 25294 at commit 93d76d6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-08-02T09:07:21Z

Test build #108563 has finished for PR 25294 at commit 11255a5.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-08-02T12:22:36Z

Test build #108564 has finished for PR 25294 at commit 0631959.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala

SparkQA · 2019-08-02T17:48:18Z

Test build #108572 has finished for PR 25294 at commit c51f31c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-08-02T18:30:44Z

Test build #108575 has finished for PR 25294 at commit 734c302.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

WeichenXu123 · 2019-08-03T05:18:58Z

Jenkins, retest this please.

WeichenXu123 · 2019-08-03T10:26:59Z

Jenkins, retest this please.

SparkQA · 2019-08-04T00:55:05Z

Test build #108596 has finished for PR 25294 at commit efed7d0.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-08-04T00:59:17Z

Test build #108599 has finished for PR 25294 at commit 2a95580.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-08-04T01:19:48Z

Test build #108597 has finished for PR 25294 at commit efed7d0.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

WeichenXu123 · 2019-08-13T02:43:14Z

@gatorsmile Any idea to address the hive metadata api "set db location" not working issue ?

wangyum · 2019-09-21T04:58:27Z

retest this please

SparkQA · 2019-09-21T06:56:07Z

Test build #111110 has finished for PR 25294 at commit 2a95580.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun added the SQL label Jul 30, 2019

WeichenXu123 added 2 commits July 30, 2019 22:01

init pr

d2a68b3

add test

71033f8

WeichenXu123 force-pushed the alter_db_loc branch from 361d70f to 71033f8 Compare July 30, 2019 14:13

WeichenXu123 changed the title ~~[SPARK-28476][SQL][WIP] Support ALTER DATABASE SET LOCATION~~ [SPARK-28476][SQL] Support ALTER DATABASE SET LOCATION Jul 30, 2019

gatorsmile reviewed Jul 30, 2019

View reviewed changes

sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala Outdated Show resolved Hide resolved

gatorsmile reviewed Jul 30, 2019

View reviewed changes

sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala Outdated Show resolved Hide resolved

fix style

f3eb6e9

gatorsmile reviewed Jul 30, 2019

View reviewed changes

sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala Outdated Show resolved Hide resolved

gatorsmile reviewed Jul 30, 2019

View reviewed changes

sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala Outdated Show resolved Hide resolved

address comments

bd4fb39

fix test

7140624

fix test

d209d85

fix test

93d76d6

add test in hive VersionsSuite

11255a5

fix style

0631959

update

c51f31c

WeichenXu123 commented Aug 2, 2019

View reviewed changes

sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala Outdated Show resolved Hide resolved

update

734c302

add log for debug

efed7d0

WeichenXu123 changed the title ~~[SPARK-28476][SQL] Support ALTER DATABASE SET LOCATION~~ [WIP][SPARK-28476][SQL] Support ALTER DATABASE SET LOCATION Aug 3, 2019

use shim msc

2a95580

cloud-fan mentioned this pull request Sep 2, 2019

[SPARK-28628][SQL] Implement SupportsNamespaces in V2SessionCatalog #25363

Closed

WeichenXu123 closed this Sep 25, 2019

Conversation

WeichenXu123 commented Jul 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

SparkQA commented Jul 30, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gatorsmile Jul 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gatorsmile Jul 30, 2019

Choose a reason for hiding this comment

Uh oh!

WeichenXu123 Jul 31, 2019

Choose a reason for hiding this comment

Uh oh!

gatorsmile Aug 1, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gatorsmile commented Jul 30, 2019

Uh oh!

SparkQA commented Jul 30, 2019

Uh oh!

SparkQA commented Jul 30, 2019

Uh oh!

SparkQA commented Jul 31, 2019

Uh oh!

SparkQA commented Jul 31, 2019

Uh oh!

SparkQA commented Jul 31, 2019

Uh oh!

SparkQA commented Aug 1, 2019

Uh oh!

SparkQA commented Aug 2, 2019

Uh oh!

SparkQA commented Aug 2, 2019

Uh oh!

Uh oh!

SparkQA commented Aug 2, 2019

Uh oh!

SparkQA commented Aug 2, 2019

Uh oh!

WeichenXu123 commented Aug 3, 2019

Uh oh!

WeichenXu123 commented Aug 3, 2019

Uh oh!

SparkQA commented Aug 4, 2019

Uh oh!

SparkQA commented Aug 4, 2019

Uh oh!

SparkQA commented Aug 4, 2019

Uh oh!

WeichenXu123 commented Aug 13, 2019

Uh oh!

wangyum commented Sep 21, 2019

Uh oh!

SparkQA commented Sep 21, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

WeichenXu123 commented Jul 30, 2019 •

edited

Loading

gatorsmile Jul 30, 2019 •

edited

Loading