[SPARK-30904][SQL] Thrift RowBasedSet serialization throws NullPointerException on NULL BigDecimal #27654
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CJStuart
changed the title
|
ok to test |
|
Test build #118742 has finished for PR 27654 at commit
|
CJStuart
changed the title
|
Test build #118781 has finished for PR 27654 at commit
|
juliuszsompolski
approved these changes
Feb 21, 2020
wangyum
changed the title
|
retest this please. |
|
Test build #118809 has finished for PR 27654 at commit
|
wangyum
changed the title
wangyum
approved these changes
Feb 22, 2020
wangyum
pushed a commit
that referenced
this pull request
Feb 22, 2020
…rException on NULL BigDecimal ### What changes were proposed in this pull request? This PR fixes SPARK-30904 by adding a null check. ### Why are the changes needed? For HIVE_CLI_SERVICE_PROTOCOL_V5 and below, serialization fails on NULL-containing decimal columns, caused by a call to `value.toPlainString()`, where `value` might be null. This null check fixes it. ### Does this PR introduce any user-facing change? No ### How was this patch tested? A test was added for serialization of NULL decimals for all HIVE_CLI_SERVICE_PROTOCOL versions. Closes #27654 from CJStuart/SPARK-30904. Authored-by: Christian Stuart <christian.stuart@databricks.com> Signed-off-by: Yuming Wang <wgyumg@gmail.com> (cherry picked from commit bcce1b1) Signed-off-by: Yuming Wang <wgyumg@gmail.com>
|
Merged to master and branch-3.0. |
sjincho
pushed a commit
to sjincho/spark
that referenced
this pull request
Apr 15, 2020
…rException on NULL BigDecimal ### What changes were proposed in this pull request? This PR fixes SPARK-30904 by adding a null check. ### Why are the changes needed? For HIVE_CLI_SERVICE_PROTOCOL_V5 and below, serialization fails on NULL-containing decimal columns, caused by a call to `value.toPlainString()`, where `value` might be null. This null check fixes it. ### Does this PR introduce any user-facing change? No ### How was this patch tested? A test was added for serialization of NULL decimals for all HIVE_CLI_SERVICE_PROTOCOL versions. Closes apache#27654 from CJStuart/SPARK-30904. Authored-by: Christian Stuart <christian.stuart@databricks.com> Signed-off-by: Yuming Wang <wgyumg@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR fixes SPARK-30904 by adding a null check.
Why are the changes needed?
For HIVE_CLI_SERVICE_PROTOCOL_V5 and below, serialization fails on NULL-containing decimal columns, caused by a call to
value.toPlainString(), wherevaluemight be null. This null check fixes it.Does this PR introduce any user-facing change?
No
How was this patch tested?
A test was added for serialization of NULL decimals for all HIVE_CLI_SERVICE_PROTOCOL versions.