Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[pulsar-io][jdbc sink]Support delete and update event for JDBC Sink #4358

Merged
merged 6 commits into from
Jul 7, 2019
Merged

[pulsar-io][jdbc sink]Support delete and update event for JDBC Sink #4358

merged 6 commits into from
Jul 7, 2019

Conversation

tuteng
Copy link
Member

@tuteng tuteng commented May 24, 2019 •
edited
Loading

Motivation

Currently our JDBC Sink not support deletion and update events.
Support for delete and update events.

Modifications

Support for delete and update events.
Add some document for JDBC Sink. #4073

Verifying this change

local Unit Test pass.
Integration test pass

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API: (yes / no)
  • The schema: (yes / no / don't know)
  • The default values of configurations: (yes / no)
  • The wire protocol: (yes / no)
  • The rest endpoints: (yes / no)
  • The admin cli options: (yes / no)
  • Anything that affects deployment: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

@tuteng tuteng changed the title [pulsar-io sink][jdbc]Support delete and update event [pulsar-io][jdbc sink]Support delete and update event May 24, 2019
@tuteng tuteng changed the title [pulsar-io][jdbc sink]Support delete and update event [pulsar-io][jdbc sink]Support delete and update event for JDBC Sink May 24, 2019
@tuteng
Support delete and update event
4eced6c 
Fix some problem of jdbc sink
@tuteng
Replace batch handle
1ece776 
Update integraiont test for jdbc sink test
@tuteng tuteng force-pushed the feature/jdbc-sink-support-select-delete-and-update branch from 52b17f8 to 1ece776 Compare May 25, 2019 00:41
@tuteng
Copy link
Member Author

tuteng commented May 25, 2019

run integraton tests

@tuteng
Copy link
Member Author

tuteng commented May 25, 2019

run Integration Tests

@sijie sijie added area/connector type/feature The PR added a new feature or issue requested a new feature labels May 27, 2019
sijie
sijie reviewed Jun 1, 2019
View reviewed changes
Copy link
Member

@sijie sijie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tuteng

Can you write an example or a documentation on how to use this JDBC sink?

Also is it possible to use debezium source and this JDBC sink to replicate a database to another database?

@tuteng
Copy link
Member Author

tuteng commented Jun 3, 2019

@tuteng

Can you write an example or a documentation on how to use this JDBC sink?

Also is it possible to use debezium source and this JDBC sink to replicate a database to another database?

I will add a use case later.

@sijie
Copy link
Member

sijie commented Jun 9, 2019

@tuteng any updates?

1 similar comment
@sijie
Copy link
Member

sijie commented Jun 18, 2019

@tuteng any updates?

@tuteng
Copy link
Member Author

tuteng commented Jun 18, 2019

@tuteng any updates?
I'm very sorry, I haven't had time to do this recently.

@tuteng
Copy link
Member Author

tuteng commented Jul 3, 2019

@sijie @jiazhai PTAL

@sijie
Copy link
Member

sijie commented Jul 3, 2019

run java8 tests

@sijie
Copy link
Member

sijie commented Jul 5, 2019

@tuteng please rebase this pull request to latest master to address the build errors.

@sijie sijie added this to the 2.5.0 milestone Jul 7, 2019
@sijie sijie merged commit 7d2141b into apache:master Jul 7, 2019
@sijie sijie deleted the feature/jdbc-sink-support-select-delete-and-update branch July 7, 2019 16:04
easyfan pushed a commit to easyfan/pulsar that referenced this pull request Jul 26, 2019
@tuteng @easyfan
[pulsar-io][jdbc sink]Support delete and update event for JDBC Sink (a…
2bc43c4 
…pache#4358)

### Motivation

Currently our JDBC Sink not support deletion and update events.
Support for delete and update events.

### Modifications

Support for delete and update events.
Add some document for JDBC Sink. apache#4073

### Verifying this change

local Unit Test pass.
Integration test pass
@yittg yittg mentioned this pull request Aug 1, 2019
Merged
1 task
sijie pushed a commit that referenced this pull request Aug 2, 2019
@yittg @sijie
[pulsar-io-jdbc] not set action as insert (#4862)
d8356d8 
### Motivation

jdbc sink treat all record as INSERT before #4358 , now it requires an indispensable action property which seems to be a break change, and we can deal records without any action property as INSERT.

### Modifications

treat action not set as INSERT action like before.
@jiazhai jiazhai removed this from the 2.5.0 milestone Aug 28, 2019
@jiazhai jiazhai added this to the 2.4.1 milestone Aug 28, 2019
@jiazhai
Copy link
Member

jiazhai commented Aug 28, 2019

add to 2.4.1 for conflict with #4703

jiazhai pushed a commit that referenced this pull request Aug 28, 2019
@tuteng @jiazhai
[pulsar-io][jdbc sink]Support delete and update event for JDBC Sink (#…
6c6fbfd 
…4358)

### Motivation

Currently our JDBC Sink not support deletion and update events.
Support for delete and update events.

### Modifications

Support for delete and update events.
Add some document for JDBC Sink. #4073

### Verifying this change

local Unit Test pass.
Integration test pass
jiazhai pushed a commit that referenced this pull request Aug 28, 2019
@yittg @jiazhai
[pulsar-io-jdbc] not set action as insert (#4862)
2725ef3 
### Motivation

jdbc sink treat all record as INSERT before #4358 , now it requires an indispensable action property which seems to be a break change, and we can deal records without any action property as INSERT.

### Modifications

treat action not set as INSERT action like before.
(cherry picked from commit d8356d8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sijie sijie sijie approved these changes

Assignees

@tuteng tuteng

Labels
area/connector type/feature The PR added a new feature or issue requested a new feature
Projects
None yet
Milestone
2.4.1
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@tuteng @sijie @jiazhai