[SPARK-22686][SQL] DROP TABLE IF EXISTS should not show AnalysisException #19888
Conversation
| @@ -206,7 +206,8 @@ case class DropTableCommand( | |||
| try { | |||
| sparkSession.sharedState.cacheManager.uncacheQuery(sparkSession.table(tableName)) | |||
| } catch { | |||
| case _: NoSuchTableException if ifExists => | |||
| case ae: AnalysisException | |||
| if ifExists && ae.cause.nonEmpty && ae.getCause.isInstanceOf[NoSuchTableException] => | |||
There was a problem hiding this comment.
There was a problem hiding this comment.
scala> spark.version
res2: String = 2.3.0-SNAPSHOT
scala> sql("DROP TABLE IF EXISTS t")
res3: org.apache.spark.sql.DataFrame = []
scala> sql("DROP TABLE IF EXISTS t")
res4: org.apache.spark.sql.DataFrame = []
Unable to reproduce it.
There was a problem hiding this comment.
it's in the log, which means it's not a serious bug.
There was a problem hiding this comment.
I can see it in 2.2.1. It is a warning message.
There was a problem hiding this comment.
hmmm, it's fixed in master?
There was a problem hiding this comment.
@dongjoon-hyun can you verify it? thanks!
There was a problem hiding this comment.
I can reproduce it in master now.
There was a problem hiding this comment.
Yep. It exists in master, too.
I'll update like the example by @cloud-fan .
|
Test build #84454 has finished for PR 19888 at commit
|
|
Test build #84458 has finished for PR 19888 at commit
|
|
Retest this please. |
| case _: NoSuchTableException if ifExists => | ||
| case NonFatal(e) => log.warn(e.toString, e) | ||
|
|
||
| if (!ifExists || catalog.tableExists(tableName)) { |
There was a problem hiding this comment.
I feel it's more readable to write
if (catalog.tableExists(tableName)) {
... do the drop
} else if (ifExists) {
// do nothing
} else {
log.warn("The table to drop does not exist: " +tableName)
}
There was a problem hiding this comment.
We can also update the UNCACHE TABLE, cc @gatorsmile
|
Test build #84463 has finished for PR 19888 at commit
|
|
I added a VIEW case and we need to throw |
|
LGTM pending Jenkins |
|
Thank you, @gatorsmile . |
|
Test build #84495 has finished for PR 19888 at commit
|
dongjoon-hyun
changed the title
|
thanks, merging to master/2.2! |
…tion ## What changes were proposed in this pull request? During [SPARK-22488](#19713) to fix view resolution issue, there occurs a regression at `2.2.1` and `master` branch like the following. This PR fixes that. ```scala scala> spark.version res2: String = 2.2.1 scala> sql("DROP TABLE IF EXISTS t").show 17/12/04 21:01:06 WARN DropTableCommand: org.apache.spark.sql.AnalysisException: Table or view not found: t; org.apache.spark.sql.AnalysisException: Table or view not found: t; ``` ## How was this patch tested? Manual. Author: Dongjoon Hyun <dongjoon@apache.org> Closes #19888 from dongjoon-hyun/SPARK-22686. (cherry picked from commit 82183f7) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
|
Thank you, @cloud-fan and @gatorsmile ! |
…tion ## What changes were proposed in this pull request? During [SPARK-22488](apache#19713) to fix view resolution issue, there occurs a regression at `2.2.1` and `master` branch like the following. This PR fixes that. ```scala scala> spark.version res2: String = 2.2.1 scala> sql("DROP TABLE IF EXISTS t").show 17/12/04 21:01:06 WARN DropTableCommand: org.apache.spark.sql.AnalysisException: Table or view not found: t; org.apache.spark.sql.AnalysisException: Table or view not found: t; ``` ## How was this patch tested? Manual. Author: Dongjoon Hyun <dongjoon@apache.org> Closes apache#19888 from dongjoon-hyun/SPARK-22686. (cherry picked from commit 82183f7) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
What changes were proposed in this pull request?
During SPARK-22488 to fix view resolution issue, there occurs a regression at
2.2.1andmasterbranch like the following. This PR fixes that.How was this patch tested?
Manual.