Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve][Connector-V2][Kafka] Support extract topic from SeaTunnelRow field #3742

Merged
merged 18 commits into from Jan 20, 2023

Conversation

TaoZex
Copy link
Contributor

@TaoZex TaoZex commented Dec 16, 2022

Purpose of this pull request

Support extract topic from SeaTunnelRow field

For test :

source config :
82fc9f9a865e6e735e84f333b9d7f5e

sink config :
33a8b5602541f7a7c3cce8d612a75ae

send records to topic:
78c1df512b3b569634e89398aa0d7ad

Check list

@TaoZex TaoZex marked this pull request as ready for review December 16, 2022 15:31
Copy link
Contributor

@hailin0 hailin0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add e2e testcase

docs/en/connector-v2/sink/Kafka.md Outdated Show resolved Hide resolved
@TaoZex
Copy link
Contributor Author

TaoZex commented Dec 17, 2022

Add e2e testcase

Writing e2e test for this feature is not easy because the kafka sink connector send records without callback function, so I took a screenshot to prove that the feature is available.

@hailin0
Copy link
Contributor

hailin0 commented Dec 17, 2022

Add e2e testcase

Writing e2e test for this feature is not easy because the kafka sink connector send records without callback function, so I took a screenshot to prove that the feature is available.

  1. Generate fixed data in the data source
  2. Use kafka sink to write topic
  3. Check the message of the topic

@TaoZex
Copy link
Contributor Author

TaoZex commented Dec 17, 2022

Add e2e testcase

Writing e2e test for this feature is not easy because the kafka sink connector send records without callback function, so I took a screenshot to prove that the feature is available.

  1. Generate fixed data in the data source
  2. Use kafka sink to write topic
  3. Check the message of the topic

Thanks for your advice.

Hisoka-X
Hisoka-X previously approved these changes Dec 27, 2022
Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@TaoZex
Copy link
Contributor Author

TaoZex commented Jan 18, 2023

I added e2e test for this feature and refactor the kafkaIT code.PTAL @hailin0 @EricJoy2048 @Hisoka-X

EricJoy2048
EricJoy2048 previously approved these changes Jan 19, 2023
@TyrantLucifer
Copy link
Member

Let's waiting CICD.

Copy link
Member

@TyrantLucifer TyrantLucifer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Hisoka-X Hisoka-X merged commit 8aff807 into apache:dev Jan 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants