Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-21446] [SQL] Fix setAutoCommit never executed #18665

Closed
wants to merge 1 commit into from
Closed

[SPARK-21446] [SQL] Fix setAutoCommit never executed #18665

wants to merge 1 commit into from

Conversation

DFFuture
Copy link
Contributor

What changes were proposed in this pull request?

JIRA Issue: https://issues.apache.org/jira/browse/SPARK-21446
options.asConnectionProperties can not have fetchsize，because fetchsize belongs to Spark-only options, and Spark-only options have been excluded in connection properities.
So change properties of beforeFetch from options.asConnectionProperties.asScala.toMap to options.asProperties.asScala.toMap

How was this patch tested?

srowen
srowen approved these changes Jul 18, 2017
View reviewed changes
Copy link
Member

@srowen srowen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're correct, because this is an instance where properties are passed to a Spark class, not a JDBC class, so it's sensible to retain Spark properties.

@SparkQA
Copy link

SparkQA commented Jul 18, 2017

Test build #3844 has finished for PR 18665 at commit 9ba431a.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

gatorsmile
gatorsmile approved these changes Jul 19, 2017
View reviewed changes
Copy link
Member

@gatorsmile gatorsmile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

asfgit pushed a commit that referenced this pull request Jul 19, 2017
@DFFuture @gatorsmile
[SPARK-21446][SQL] Fix setAutoCommit never executed
308bce0 
## What changes were proposed in this pull request?
JIRA Issue: https://issues.apache.org/jira/browse/SPARK-21446
options.asConnectionProperties can not have fetchsize，because fetchsize belongs to Spark-only options, and Spark-only options have been excluded in connection properities.
So change properties of beforeFetch from  options.asConnectionProperties.asScala.toMap to options.asProperties.asScala.toMap

## How was this patch tested?

Author: DFFuture <albert.zhang23@gmail.com>

Closes #18665 from DFFuture/sparksql_pg.

(cherry picked from commit c972918)
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
asfgit pushed a commit that referenced this pull request Jul 19, 2017
@DFFuture @gatorsmile
[SPARK-21446][SQL] Fix setAutoCommit never executed
9498798 
## What changes were proposed in this pull request?
JIRA Issue: https://issues.apache.org/jira/browse/SPARK-21446
options.asConnectionProperties can not have fetchsize，because fetchsize belongs to Spark-only options, and Spark-only options have been excluded in connection properities.
So change properties of beforeFetch from  options.asConnectionProperties.asScala.toMap to options.asProperties.asScala.toMap

## How was this patch tested?

Author: DFFuture <albert.zhang23@gmail.com>

Closes #18665 from DFFuture/sparksql_pg.

(cherry picked from commit c972918)
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
@gatorsmile
Copy link
Member

Thanks! Merging to master/2.2/2.1

@asfgit asfgit closed this in c972918 Jul 19, 2017
@DFFuture DFFuture deleted the sparksql_pg branch July 20, 2017 03:04
MatthewRBruce pushed a commit to Shopify/spark that referenced this pull request Jul 31, 2018
@DFFuture @MatthewRBruce
[SPARK-21446][SQL] Fix setAutoCommit never executed
fff38be 
## What changes were proposed in this pull request?
JIRA Issue: https://issues.apache.org/jira/browse/SPARK-21446
options.asConnectionProperties can not have fetchsize，because fetchsize belongs to Spark-only options, and Spark-only options have been excluded in connection properities.
So change properties of beforeFetch from  options.asConnectionProperties.asScala.toMap to options.asProperties.asScala.toMap

## How was this patch tested?

Author: DFFuture <albert.zhang23@gmail.com>

Closes apache#18665 from DFFuture/sparksql_pg.

(cherry picked from commit c972918)
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
jzhuge pushed a commit to jzhuge/spark that referenced this pull request Aug 20, 2018
@DFFuture
[SPARK-21446][SQL] Fix setAutoCommit never executed
3f3e71b 
JIRA Issue: https://issues.apache.org/jira/browse/SPARK-21446
options.asConnectionProperties can not have fetchsize，because fetchsize belongs to Spark-only options, and Spark-only options have been excluded in connection properities.
So change properties of beforeFetch from  options.asConnectionProperties.asScala.toMap to options.asProperties.asScala.toMap

Author: DFFuture <albert.zhang23@gmail.com>

Closes apache#18665 from DFFuture/sparksql_pg.

(cherry picked from commit c972918)
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@srowen srowen srowen approved these changes

@gatorsmile gatorsmile gatorsmile approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@DFFuture @SparkQA @gatorsmile @srowen