New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-42600][SQL] CatalogImpl.currentDatabase shall use NamespaceHelper instead of MultipartIdentifierHelper #40192
Conversation
…of MultipartIdentifierHelper ```java Exception in thread "main" org.apache.spark.sql.AnalysisException: Multi-part identifier cannot be empty. at org.apache.spark.sql.errors.QueryCompilationErrors$.emptyMultipartIdentifierError(QueryCompilationErrors.scala:1887) at org.apache.spark.sql.connector.catalog.CatalogV2Implicits$MultipartIdentifierHelper.<init>(CatalogV2Implicits.scala:152) at org.apache.spark.sql.connector.catalog.CatalogV2Implicits$.MultipartIdentifierHelper(CatalogV2Implicits.scala:150) at org.apache.spark.sql.internal.CatalogImpl.currentDatabase(CatalogImpl.scala:65) ```
@@ -62,7 +62,7 @@ class CatalogImpl(sparkSession: SparkSession) extends Catalog { | |||
* Returns the current default database in this session. | |||
*/ | |||
override def currentDatabase: String = | |||
sparkSession.sessionState.catalogManager.currentNamespace.toSeq.quoted | |||
sparkSession.sessionState.catalogManager.currentNamespace.quoted |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm does this make a difference?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh I see. It's so tricky to have implicit class for both Seq[String]
and Array[String]
. We should clean it up in the future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's so tricky to have implicit class for both Seq[String] and Array[String]
Yea 👀
thanks, merging to master/3.4! |
…per instead of MultipartIdentifierHelper ### What changes were proposed in this pull request? v2 catalog default namespace may be empty ```java Exception in thread "main" org.apache.spark.sql.AnalysisException: Multi-part identifier cannot be empty. at org.apache.spark.sql.errors.QueryCompilationErrors$.emptyMultipartIdentifierError(QueryCompilationErrors.scala:1887) at org.apache.spark.sql.connector.catalog.CatalogV2Implicits$MultipartIdentifierHelper.<init>(CatalogV2Implicits.scala:152) at org.apache.spark.sql.connector.catalog.CatalogV2Implicits$.MultipartIdentifierHelper(CatalogV2Implicits.scala:150) at org.apache.spark.sql.internal.CatalogImpl.currentDatabase(CatalogImpl.scala:65) ``` ### Why are the changes needed? bugfix ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? locally verified Closes #40192 from yaooqinn/SPARK-42600. Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit 53c4158) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
…per instead of MultipartIdentifierHelper ### What changes were proposed in this pull request? v2 catalog default namespace may be empty ```java Exception in thread "main" org.apache.spark.sql.AnalysisException: Multi-part identifier cannot be empty. at org.apache.spark.sql.errors.QueryCompilationErrors$.emptyMultipartIdentifierError(QueryCompilationErrors.scala:1887) at org.apache.spark.sql.connector.catalog.CatalogV2Implicits$MultipartIdentifierHelper.<init>(CatalogV2Implicits.scala:152) at org.apache.spark.sql.connector.catalog.CatalogV2Implicits$.MultipartIdentifierHelper(CatalogV2Implicits.scala:150) at org.apache.spark.sql.internal.CatalogImpl.currentDatabase(CatalogImpl.scala:65) ``` ### Why are the changes needed? bugfix ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? locally verified Closes apache#40192 from yaooqinn/SPARK-42600. Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit 53c4158) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
What changes were proposed in this pull request?
v2 catalog default namespace may be empty
Why are the changes needed?
bugfix
Does this PR introduce any user-facing change?
no
How was this patch tested?
locally verified