Unsupported jdbc driver classname error when trying to use as custom driver in Glue

[amichel]

Describe the bug

I'm trying to use this driver as a custom driver for Glue connection, in order to connect to Aurora Postgres.
I'm using software.amazon.jdbc.Driver as a class name and jdbc:aws-wrapper:postgresql: in the URL prefix.
Glue crawler/job fail with the error "Unsupported jdbc driver classname"

Expected Behavior

It should be possible to use Aurora driver in Glue

What plugins are used? What other connection properties were set?

Default. No advanced properties can be set in Glue

Current Behavior

Glue crawler/job fail with the error "Unsupported jdbc driver classname"

Reproduction Steps

Create a new Glue Connection, using aws-advanced-jdbc-wrapper-2.2.4.jar in an S3 bucket, software.amazon.jdbc.Driver as a class name and jdbc:aws-wrapper:postgresql: in the URL prefix.
Create and run new crawler using that connection

Possible Solution

No response

Additional Information/Context

No response

The AWS Advanced JDBC Driver version used

2.2.4

JDK version used

Glue

Operating System and version

Glue

[crystall-bitquill]

Hi @amichel,

Thanks for reaching out and raising this issue.

We'll take a look at this and keep you updated as we investigate.

Thank you for your patience!

[karenc-bq]

Hi, @amichel, thank you for reaching out.

I am able to reproduce your issue but we unfortunately cannot provide a fix for this issue. It seems although AWS Glue say they do support custom drivers, they are checking for very specific driver class names.

It seems like AWS Glue only supports different versions of the supported datasources at the moment.

[amichel]

Yeah, I figured it out. The only workaround is to create a regular network connection in glue and load the driver explicitly in the job. This works for Glue jobs, but doesn't solve the Crawler problem.

…

On Thu, Oct 5, 2023, 16:40 Karen ***@***.***> wrote: Hi, @amichel <https://github.com/amichel>, thank you for reaching out. I am able to reproduce your issue but we unfortunately cannot provide a fix for this issue. It seems although AWS Glue say they do support custom drivers <https://aws.amazon.com/about-aws/whats-new/2023/05/aws-glue-crawler-custom-jdbc-drivers/>, they are checking for very specific driver class names <https://docs.aws.amazon.com/glue/latest/dg/console-connections-jdbc-drivers.html#:~:text=The%20driver%20class%20name%20must%20be%20for%20a%20datasource%20supported%20by%20AWS%20Glue%20crawlers.> . It seems like AWS Glue only supports different versions of the supported datasources <https://docs.aws.amazon.com/glue/latest/dg/console-connections-jdbc-drivers.html#:~:text=own%20JDBC%20driver.-,Supported%20datasources,-Supported%20datasources> at the moment. — Reply to this email directly, view it on GitHub <#653 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAM5BY5UJQIKUI64EU3D6ADX54LMJAVCNFSM6AAAAAA5GMX2LSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBZGYYTAMJWGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[stnorbi]

Hi All,

I have the same issue regarding Teradata jdbc driver in case of the following JDBC Driver Class name: com.teradata.jdbc.TeraDriver

Is there any update or solution since the issue has come up?

Thank you for any reply.

Cheers,
Norbi

[karenc-bq]

Hi @stnorbi, could you please provide more information about your workflow for clarification? Are you attempting to use this AWS JDBC Driver for Glue connection with Teradata JDBC Driver as the underlying driver?

We have previously looked into using the AWS JDBC Driver with Glue, but unfortunately we don't have any further updates as of now. We suggest you reaching out to the AWS Glue team on their repository or by emailing them at glue-connectors@amazon.com for additional support.

[javicamarababel]

Hello

I am trying to use JDBC driver class com.ibm.as400.access.AS400JDBCDriver from https://mvnrepository.com/artifact/net.sf.jt400/jt400, and getting the same "Unsupported jdbc driver classname" when trying to create Glue Crawler.

Does this really mean that one can't use Glue with any JDBC Driver, but Glue is limited to a handful of JDBC Drivers?

[davecramer]

@javicamarababel are you looking to use the aws-advanced driver just for authentication ?

[javicamarababel]

Hi @davecramer

I am not actually using the AWS Advanced Driver, but a custom JDBC Driver, but I came to this thread because I get the same error message. Much like @stnorbi before, I guess.

Therefore I guess my post is somewhat out of topic here, and as @karenc-bq said this is Glue issue that we should report to the Glue team... I will write to glue-connectors@amazon.com as she suggested.

Thanks and best regards