Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-10170] [SQL] Add DB2 JDBC dialect support. #8393

Closed

Conversation

sureshthalamati
Copy link
Contributor

Data frame write to DB2 database is failing because by default JDBC data source implementation is generating a table schema with DB2 unsupported data types TEXT for String, and BIT1(1) for Boolean.

This patch registers DB2 JDBC Dialect that maps String, Boolean to valid DB2 data types.

…o map string, boolean to valid db2 types on write.
@rxin
Copy link
Contributor

rxin commented Aug 25, 2015

Jenkins, test this please.

@rxin
Copy link
Contributor

rxin commented Aug 25, 2015

Can you update the pull request title to say "Add DB2 JDBC dialect support"?

@sureshthalamati sureshthalamati changed the title [SPARK-10170] [SQL] Writing from data frame into db2 database using jdbc data source api fails with error for string, and boolean column types. [SPARK-10170] [SQL] Add DB2 JDBC dialect support. Aug 25, 2015
@sureshthalamati
Copy link
Contributor Author

Thanks for reviewing the change. Changed the pull request title.

@SparkQA
Copy link

SparkQA commented Aug 25, 2015

Test build #1692 has finished for PR 8393 at commit 0c999ce.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@lresende
Copy link
Member

LGTM

@rxin
Copy link
Contributor

rxin commented Aug 27, 2015

BTW - are there any way we can create a lightweight docker container with some version of DB2 for testing the JDBC connectivity to DB2? (both from a licensing pov and technical packaging pov)

@lresende
Copy link
Member

Looks like there is a DB2 Express version that is free, let me see if I can make a docker image with that.

@rxin
Copy link
Contributor

rxin commented Aug 31, 2015

Merging this in. Please look into docker test for this. Thanks.

@asfgit asfgit closed this in a2d5c72 Aug 31, 2015
@sureshthalamati
Copy link
Contributor Author

Thank you for merging the fix. I will look into adding a docker test case. Can you point me to any existing docker test case in the spark project ?

@rxin
Copy link
Contributor

rxin commented Sep 1, 2015

It's here: #8101

@mmfu
Copy link

mmfu commented Sep 1, 2015

Eager to try out this DB2 JDBC dialect. Its changes seem self-contained and affects only DB2 JDBC users. Is there any chance to make this support available in the coming v1.5 release?

@rxin
Copy link
Contributor

rxin commented Sep 1, 2015

No sorry the release candidate has been cut. You can always do a custom build yourself though.

@JoshRosen
Copy link
Contributor

Hey @sureshthalamati, I've merged in #9503 so you should now be able to add Docker tests for this. Take a look at the suites in my PR for examples of how to do this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
6 participants