[SPARK-33788][SQL] Throw NoSuchPartitionsException from HiveExternalCatalog.dropPartitions() #30778
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
HyukjinKwon
approved these changes
Dec 15, 2020
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
Test build #132820 has finished for PR 30778 at commit
|
|
The test failure "org.apache.spark.sql.hive.thriftserver.ThriftServerQueryTestSuite.subquery/scalar-subquery/scalar-subquery-select.sql" is not related to the changes. |
|
jenkins, retest this, please |
|
Kubernetes integration test starting |
|
Test build #132824 has finished for PR 30778 at commit
|
|
Kubernetes integration test status success |
|
Merged to master. |
|
@MaxGekk, it has a conflict in other branches. do you want to backport? |
MaxGekk
added a commit
to MaxGekk/spark
that referenced
this pull request
Dec 16, 2020
…atalog.dropPartitions() Throw `NoSuchPartitionsException` from `ALTER TABLE .. DROP TABLE` for not existing partitions of a table in V1 Hive external catalog. The behaviour of Hive external catalog deviates from V1/V2 in-memory catalogs that throw `NoSuchPartitionsException`. To improve user experience with Spark SQL, it would be better to throw the same exception. Yes, the command throws `NoSuchPartitionsException` instead of the general exception `AnalysisException`. By running tests for `ALTER TABLE .. DROP PARTITION`: ``` $ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly *AlterTableDropPartitionSuite" ``` Closes apache#30778 from MaxGekk/hive-drop-partition-exception. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org> (cherry picked from commit 3dfdcf4) Signed-off-by: Max Gekk <max.gekk@gmail.com>
|
@HyukjinKwon Here is the backports to 3.1/3.0/2.4: #30802 . I ran the test suite |
This was referenced Dec 21, 2020
Closed
Closed
Closed
HyukjinKwon
pushed a commit
that referenced
this pull request
Dec 27, 2020
… in `HiveClientImpl` ### What changes were proposed in this pull request? Update the SQL migration guide about the changes made by: - #30778 - #30711 - #30866 ### Why are the changes needed? To inform users about the recent changes in the upcoming releases. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? N/A Closes #30925 from MaxGekk/sql-migr-guide-hiveclientimpl. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
HyukjinKwon
pushed a commit
that referenced
this pull request
Dec 27, 2020
…anges in `HiveClientImpl` ### What changes were proposed in this pull request? Update the SQL migration guide about the changes made by: - #30778 - #30711 ### Why are the changes needed? To inform users about the recent changes in the upcoming releases. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? N/A Closes #30931 from MaxGekk/sql-migr-guide-hiveclientimpl-3.1. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
HyukjinKwon
pushed a commit
that referenced
this pull request
Dec 27, 2020
…anges in `HiveClientImpl` ### What changes were proposed in this pull request? Update the SQL migration guide about the changes made by: - #30778 - #30711 ### Why are the changes needed? To inform users about the recent changes in the upcoming releases. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? N/A Closes #30932 from MaxGekk/sql-migr-guide-hiveclientimpl-3.0. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
HyukjinKwon
pushed a commit
that referenced
this pull request
Dec 27, 2020
…anges in `HiveClientImpl` ### What changes were proposed in this pull request? Update the SQL migration guide about the changes made by: - #30778 - #30711 ### Why are the changes needed? To inform users about the recent changes in the upcoming releases. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? N/A Closes #30933 from MaxGekk/sql-migr-guide-hiveclientimpl-2.4. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Throw
NoSuchPartitionsExceptionfromALTER TABLE .. DROP TABLEfor not existing partitions of a table in V1 Hive external catalog.Why are the changes needed?
The behaviour of Hive external catalog deviates from V1/V2 in-memory catalogs that throw
NoSuchPartitionsException. To improve user experience with Spark SQL, it would be better to throw the same exception.Does this PR introduce any user-facing change?
Yes, the command throws
NoSuchPartitionsExceptioninstead of the general exceptionAnalysisException.How was this patch tested?
By running tests for
ALTER TABLE .. DROP PARTITION: