Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [Flink] Flink engine, source is LocalFile, sink is jdbc, get "No suitable driver" Exception #6902

Open
3 tasks done
luoyongsir opened this issue May 25, 2024 · 5 comments
Labels

Comments

@luoyongsir
Copy link

luoyongsir commented May 25, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

Flink engine, source is LocalFile, sink is jdbc, get "No suitable driver" Exception.
Before this, I had already placed the driver in "${SEATUNNEL_HOME}/plugins/jdbc/lib/" directory

SeaTunnel Version

2.3.5

SeaTunnel Config

env {
  job.mode = "BATCH"
  parallelism = 1
}

source {
  LocalFile {
  }
}

transform {
}

sink {
  jdbc {
  }
}

Running Command

${SEATUNNEL_HOME}/bin/start-seatunnel-flink-15-connector-v2.sh --config xxx.conf -e run --target remote

Error Exception

Caused by: org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[JDBC-04], ErrorDescription:[Connector database failed] - unable to open JDBC writer
		at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.open(JdbcOutputFormat.java:69)
		at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.tryOpen(JdbcSinkWriter.java:117)
		at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.write(JdbcSinkWriter.java:128)
		at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.write(JdbcSinkWriter.java:47)
		at org.apache.seatunnel.translation.flink.sink.FlinkSinkWriter.write(FlinkSinkWriter.java:91)
		at org.apache.flink.streaming.api.transformations.SinkV1Adapter$SinkWriterV1Adapter.write(SinkV1Adapter.java:136)
		at org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.processElement(SinkWriterOperator.java:160)
		at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)
		at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)
		at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)
		at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
		at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542)
		at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
		at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
		at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
		at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
		at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
		at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
		at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
		at java.lang.Thread.run(Thread.java:748)
	Caused by: java.lang.RuntimeException: Failed to get driver instance for jdbcUrl=jdbc:mysql://xxxx
		at org.apache.seatunnel.shade.com.zaxxer.hikari.util.DriverDataSource.<init>(DriverDataSource.java:114)
		at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.PoolBase.initializeDataSource(PoolBase.java:331)
		at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.PoolBase.<init>(PoolBase.java:114)
		at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:108)
		at org.apache.seatunnel.shade.com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
		at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.ConnectionPoolManager.lambda$getConnection$0(ConnectionPoolManager.java:45)
		at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
		at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.ConnectionPoolManager.getConnection(ConnectionPoolManager.java:41)
		at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.connection.SimpleJdbcConnectionPoolProviderProxy.getOrEstablishConnection(SimpleJdbcConnectionPoolProviderProxy.java:57)
		at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.open(JdbcOutputFormat.java:67)
		... 19 more
	Caused by: java.sql.SQLException: No suitable driver
		at java.sql.DriverManager.getDriver(DriverManager.java:315)
		at org.apache.seatunnel.shade.com.zaxxer.hikari.util.DriverDataSource.<init>(DriverDataSource.java:106)
		... 28 more

Zeta or Flink or Spark Version

Flink 1.6.1

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@luoyongsir luoyongsir added the bug label May 25, 2024
@ljl568419709
Copy link

你需要把驱动包 和connectorjar包方在seatunnel 的lib目录下,才可以的

@luoyongsir
Copy link
Author

luoyongsir commented May 29, 2024

的lib目录下试过了,没效果。而且官方文档说的就是放 "${SEATUNNEL_HOME}/plugins/jdbc/lib/" 目录

@ljl568419709
Copy link

的lib目录下试过了,没效果。而且官方文档说的就是放 "${SEATUNNEL_HOME}/plugins/jdbc/lib/" 目录

怎么可能呢,我都试过10多种数据源的测试了,虽然有些有问题,但总归是能调起的,还有jdbc的connector包也要丢进去

@luoyongsir
Copy link
Author

请问你的source 是LocalFile,sink是jdbc,并且在Flink上运行吗?我其他场景也用了十来种数据源,几千个任务了,都没问题

@ljl568419709
Copy link

请问你的source 是LocalFile,sink是jdbc,并且在Flink上运行吗?我其他场景也用了十来种数据源,几千个任务了,都没问题
你说的那个localFile 我们没测试过,可能有bug,可以加我vx:lijialing8811 后面一起交流

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants