Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][connector] Add getSourceConfig method on SourceContext. #16305

Merged
merged 1 commit into from
Jul 13, 2022

Conversation

shibd
Copy link
Member

@shibd shibd commented Jun 30, 2022

Motivation

The configuration of SourceConfig is rich, and the user can flexibly specify it when registering the source. But some sources do not support these configurations. For example, some sources do not support EFFECTIVELY_ONCE guarantees, because the upstream system cannot generate ordered sequence id.

So, add the getSourceConfig method to SourceContext, enables the source to validate the configuration when executing the source.open method.

Modifications

  • Add getSourceConfig method to SourceContext

Documentation

  • doc-not-needed

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Jun 30, 2022
@shibd
Copy link
Member Author

shibd commented Jun 30, 2022

@nlu90 @freeznet @codelipenghui Please help review it, thanks.

@nahguam
Copy link
Contributor

nahguam commented Jul 4, 2022

So, add the getSourceConfig method to SinkContext

Do you mean add it to SourceContext?

@shibd
Copy link
Member Author

shibd commented Jul 4, 2022

Do you mean add it to SourceContext?

Yes.

@codelipenghui codelipenghui added this to the 2.11.0 milestone Jul 13, 2022
@codelipenghui codelipenghui added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages area/connector labels Jul 13, 2022
@codelipenghui codelipenghui merged commit c122928 into apache:master Jul 13, 2022
gaozhangmin pushed a commit to gaozhangmin/pulsar that referenced this pull request Jul 14, 2022
wuxuanqicn pushed a commit to wuxuanqicn/pulsar that referenced this pull request Jul 14, 2022
@nlu90
Copy link
Member

nlu90 commented Jul 22, 2022

Motivation

...
So, add the getSourceConfig method to SinkContext, enables the source to validate the configuration when executing the source.open method.

^^ The PR is good, but the description has some typo. change SinkContext to SourceContext

@shibd
Copy link
Member Author

shibd commented Jul 22, 2022

^^ The PR is good, but the description has some typo. change SinkContext to SourceContext

Sorry, I changed.

zymap added a commit to zymap/pulsar-io-lakehouse that referenced this pull request Sep 15, 2022
---

*Motivation*

Pulsar has added the new method into the SourceContext and SinkContext.
Which will break the current code with the implementation.
Adding the related implementation to avoid building failed.

apache/pulsar#15482
apache/pulsar#16305

*Modification*

- Adding the realted implementation to the class
hangc0276 pushed a commit to streamnative/pulsar-io-lakehouse that referenced this pull request Sep 15, 2022
---

*Motivation*

Pulsar has added the new method into the SourceContext and SinkContext.
Which will break the current code with the implementation.
Adding the related implementation to avoid building failed.

apache/pulsar#15482
apache/pulsar#16305

*Modification*

- Adding the realted implementation to the class
zymap added a commit to streamnative/pulsar-io-lakehouse that referenced this pull request Sep 15, 2022
---

*Motivation*

Pulsar has added the new method into the SourceContext and SinkContext.
Which will break the current code with the implementation.
Adding the related implementation to avoid building failed.

apache/pulsar#15482
apache/pulsar#16305

*Modification*

- Adding the realted implementation to the class

(cherry picked from commit c62df08)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connector doc-not-needed Your PR changes do not impact docs type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants