-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature][Spark] Support flow control in Spark #5510
Conversation
# Conflicts: # seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/task/SourceSeaTunnelTask.java
# Conflicts: # docs/en/concept/speed-limit.md
|
||
<dependency> | ||
<groupId>org.apache.seatunnel</groupId> | ||
<artifactId>seatunnel-core-starter</artifactId> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why add this dependency?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry, my prolem
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How to test this logic, this e2e is disabled on spark
https://github.com/apache/seatunnel/pull/5509/files#diff-7df030b675701262f339e58e388d7dbeb914e0cb186968f0ff10341df568895dR26
https://github.com/apache/seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mysql-e2e/src/test/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/MysqlCDCIT.java#L58
@hailin0 I tested it with Kafkae2e |
# Conflicts: # seatunnel-core/seatunnel-spark-starter/seatunnel-spark-starter-common/src/main/java/org/apache/seatunnel/core/starter/spark/execution/SourceExecuteProcessor.java
Thanks @zhilinli123 and @hailin0 for review. |
Purpose of this pull request
Check list
SubTask:#5504
New License Guide
release-note
.