New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-14346][SQL] Lists unsupported Hive features in SHOW CREATE TABLE output #13173
Conversation
Test build #58781 has finished for PR 13173 at commit
|
Test build #58782 has finished for PR 13173 at commit
|
"because it contains table structure(s) (e.g. skewed columns) that Spark SQL doesn't " + | ||
"support yet." | ||
"which is created by Hive and uses the following feature(s) that are not yet supported " + | ||
"by Spark SQL:\n" + features.map(" - " + _).mkString("\n") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It sounds like that we are missing functionalities. Can we update the error message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated
Test build #58802 has finished for PR 13173 at commit
|
retest this please |
Test build #58845 has finished for PR 13173 at commit
|
ca22d71
to
925e50a
Compare
Test build #58865 has finished for PR 13173 at commit
|
Looks good. Merging into master 2.0. |
…LE output ## What changes were proposed in this pull request? This PR is a follow-up of #13079. It replaces `hasUnsupportedFeatures: Boolean` in `CatalogTable` with `unsupportedFeatures: Seq[String]`, which contains unsupported Hive features of the underlying Hive table. In this way, we can accurately report all unsupported Hive features in the exception message. ## How was this patch tested? Updated existing test case to check exception message. Author: Cheng Lian <lian@databricks.com> Closes #13173 from liancheng/spark-14346-follow-up. (cherry picked from commit 6ac1c3a) Signed-off-by: Andrew Or <andrew@databricks.com>
What changes were proposed in this pull request?
This PR is a follow-up of #13079. It replaces
hasUnsupportedFeatures: Boolean
inCatalogTable
withunsupportedFeatures: Seq[String]
, which contains unsupported Hive features of the underlying Hive table. In this way, we can accurately report all unsupported Hive features in the exception message.How was this patch tested?
Updated existing test case to check exception message.