Skip to content

[Feature][Sinker]Add pre_sql and post_sql for JDBC Sinker#4123

Closed
rewerma wants to merge 3 commits intoapache:devfrom
rewerma:feature/pre_sql
Closed

[Feature][Sinker]Add pre_sql and post_sql for JDBC Sinker#4123
rewerma wants to merge 3 commits intoapache:devfrom
rewerma:feature/pre_sql

Conversation

@rewerma
Copy link
Contributor

@rewerma rewerma commented Feb 14, 2023

Purpose of this pull request

Add pre_sql and post_sql for JDBC Sinker, for example config:

sink {
  jdbc {
      source_table_name = "t_user"
      url = "jdbc:mysql://localhost:3306/test"
      driver = "com.mysql.cj.jdbc.Driver"
      user = "test"
      password = "test"
      pre_sql = ["truncate table t_user"]
      query = "insert into t_user(name,age) values(?,?)"
      post_sql = ["delete from t_user where age>150"]
  }
}

Check list

rewerma and others added 2 commits February 11, 2023 17:40
[Improve][Project] Code format with spotless plugin. (#4101)
@rewerma rewerma changed the title Add pre_sql and post_sql for JDBC Sinker [Feature][Sinker]Add pre_sql and post_sql for JDBC Sinker Feb 14, 2023
Copy link
Member

@liugddx liugddx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

presql should be add in SeaTunnel Plugin LifeCycle#prepare

Copy link
Contributor Author

@rewerma rewerma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The connection get from connectionProvider, it will be closed when JdbcSinkWriter close by the code:

outputFormat.close();

@liugddx
Copy link
Member

liugddx commented Feb 15, 2023

The connection get from connectionProvider, it will be closed when JdbcSinkWriter close by the code:

outputFormat.close();

If the degree of parallelism is >1, does pre_sql need to be executed multiple times?

@rewerma
Copy link
Contributor Author

rewerma commented Feb 15, 2023

The connection get from connectionProvider, it will be closed when JdbcSinkWriter close by the code:

outputFormat.close();

If the degree of parallelism is >1, does pre_sql need to be executed multiple times?

No, the method 'close' and 'open' of sinkWriter be called only once in the task life cycle.


env {
# You can set flink configuration here
execution.parallelism = 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please set execution.parallelism > 1

@rewerma rewerma closed this Feb 15, 2023
@rewerma rewerma deleted the feature/pre_sql branch March 29, 2023 04:01
@XuankuF
Copy link

XuankuF commented Jul 14, 2023

Excuse me, but why hasn't this PR been merged? Or how do I execute delete before sink?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants