Skip to content

[SPARK-3004][SQL] Added null checking when retrieving row set #1920

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
liancheng wants to merge 2 commits into from
Closed

[SPARK-3004][SQL] Added null checking when retrieving row set #1920

liancheng wants to merge 2 commits into from

Conversation

@liancheng
Copy link
Contributor

@liancheng liancheng commented Aug 13, 2014

JIRA issue: SPARK-3004

HiveThriftServer2 throws exception when the result set contains NULL. Should check isNullAt in SparkSQLOperationManager.getNextRowSet.

Note that simply using row.addColumnValue(null) doesn't work, since Hive set the column type of a null ColumnValue to String by default.

@SparkQA
Copy link

SparkQA commented Aug 13, 2014

QA tests have started for PR 1920. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18436/consoleFull

@SparkQA
Copy link

SparkQA commented Aug 13, 2014

QA results for PR 1920:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18436/consoleFull

@liancheng
Copy link
Contributor Author

liancheng commented Aug 13, 2014

I believe the build failure is caused by SPARK-3013. Should retest this after the issue is fixed.

@marmbrus
Copy link
Contributor

marmbrus commented Aug 13, 2014

test this please

@SparkQA
Copy link

SparkQA commented Aug 13, 2014

QA tests have started for PR 1920. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18466/consoleFull

@SparkQA
Copy link

SparkQA commented Aug 13, 2014

QA results for PR 1920:
- This patch FAILED unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18466/consoleFull

@marmbrus
Copy link
Contributor

marmbrus commented Aug 13, 2014

Test failures are only in pyspark. I'm going to merge this into master and 1.1.

Thanks @liancheng !

@asfgit asfgit closed this in bdc7a1a Aug 13, 2014
asfgit pushed a commit that referenced this pull request Aug 13, 2014
@liancheng @marmbrus
[SPARK-3004][SQL] Added null checking when retrieving row set
8732375 
JIRA issue: [SPARK-3004](https://issues.apache.org/jira/browse/SPARK-3004)

HiveThriftServer2 throws exception when the result set contains `NULL`. Should check `isNullAt` in `SparkSQLOperationManager.getNextRowSet`.

Note that simply using `row.addColumnValue(null)` doesn't work, since Hive set the column type of a null `ColumnValue` to String by default.

Author: Cheng Lian <lian.cs.zju@gmail.com>

Closes #1920 from liancheng/spark-3004 and squashes the following commits:

1b1db1c [Cheng Lian] Adding NULL column values in the Hive way
2217722 [Cheng Lian] Fixed SPARK-3004: added null checking when retrieving row set

(cherry picked from commit bdc7a1a)
Signed-off-by: Michael Armbrust <michael@databricks.com>
xiliu82 pushed a commit to xiliu82/spark that referenced this pull request Sep 4, 2014
@liancheng @conviva-zz
[SPARK-3004][SQL] Added null checking when retrieving row set
3d82b21 
JIRA issue: [SPARK-3004](https://issues.apache.org/jira/browse/SPARK-3004)

HiveThriftServer2 throws exception when the result set contains `NULL`. Should check `isNullAt` in `SparkSQLOperationManager.getNextRowSet`.

Note that simply using `row.addColumnValue(null)` doesn't work, since Hive set the column type of a null `ColumnValue` to String by default.

Author: Cheng Lian <lian.cs.zju@gmail.com>

Closes apache#1920 from liancheng/spark-3004 and squashes the following commits:

1b1db1c [Cheng Lian] Adding NULL column values in the Hive way
2217722 [Cheng Lian] Fixed SPARK-3004: added null checking when retrieving row set
@liancheng liancheng deleted the spark-3004 branch September 24, 2014 00:07
snmvaughan pushed a commit to snmvaughan/spark that referenced this pull request Mar 26, 2024
rdar://124368074 Bump Boson version to 0.3.30 (apache#1920)
3fd14d2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@liancheng @SparkQA @marmbrus