Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-25795][python][connector/pulsar] Add pulsar sink DataStream API #19682

Closed
wants to merge 2 commits into from

Conversation

a49a
Copy link
Contributor

@a49a a49a commented May 9, 2022

What is the purpose of the change

There is no python pulsar sink API.

Brief change log

Add python pulsar sink.

Verifying this change

This change added tests and can be verified as follows:

(example:)

  • Added test that validates sink configuration

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
  • The S3 file system connector: (n )

Documentation

  • Does this pull request introduce a new feature? (yes)
  • If yes, how is the feature documented? (PyDocs)

@flinkbot
Copy link
Collaborator

flinkbot commented May 9, 2022

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Copy link
Contributor

@dianfu dianfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deadwind4 Thanks for the PR. Have left a few comments. Besides, we should also update the documentation in https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/pulsar/ by adding Python examples.

flink-python/pyflink/datastream/connectors.py Outdated Show resolved Hide resolved
flink-python/pyflink/datastream/connectors.py Outdated Show resolved Hide resolved
flink-python/pyflink/datastream/connectors.py Outdated Show resolved Hide resolved
flink-python/pyflink/datastream/connectors.py Outdated Show resolved Hide resolved
@a49a a49a force-pushed the pulsar-sink branch 2 times, most recently from 037425a to 91d63ca Compare May 15, 2022 04:08
@a49a a49a requested a review from dianfu May 15, 2022 04:24
Copy link
Contributor

@dianfu dianfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deadwind4 Thanks a lot for the update. Have left a few comments. Besides, it seems that the tests didn't passed and there are checkstyle issues. You could run check style check locally under directory flink-python as following: ./dev/lint-python.sh sphinx

flink-python/pyflink/datastream/connectors/pulsar.py Outdated Show resolved Hide resolved
flink-python/pyflink/datastream/connectors/pulsar.py Outdated Show resolved Hide resolved
flink-python/pyflink/datastream/connectors/pulsar.py Outdated Show resolved Hide resolved
flink-python/pyflink/datastream/connectors/pulsar.py Outdated Show resolved Hide resolved
docs/content/docs/connectors/datastream/pulsar.md Outdated Show resolved Hide resolved
Copy link
Contributor

@dianfu dianfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deadwind4 Thanks for the update. LGTM overall.

Example:
::

>>> sink = PulsarSink() \\
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>>> sink = PulsarSink() \\
>>> sink = PulsarSink \\

flink-python/pyflink/datastream/connectors.py Outdated Show resolved Hide resolved
@dianfu dianfu closed this in a70e289 May 19, 2022
JasonLeeCoding pushed a commit to JasonLeeCoding/flink that referenced this pull request May 27, 2022
huangxiaofeng10047 pushed a commit to huangxiaofeng10047/flink that referenced this pull request Jun 6, 2022
zstraw pushed a commit to zstraw/flink that referenced this pull request Jul 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants