-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support key_based batch builder for functions and sources #8523
Support key_based batch builder for functions and sources #8523
Conversation
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
/pulsarbot run-failure-checks |
Signed-off-by: xiaolong.ran <rxl@apache.org>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks good. However, I didn't see any test case. Can you add one?
Signed-off-by: xiaolong.ran <rxl@apache.org>
The current unit tests can be covered. The change of this pull request is to add a batchBuilder field to |
You mean, do we add another integration test on top of the unit test? |
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
@sijie PTAL again thanks. I added the |
### Motivation Currently, we support the Key_Shared subscription mode in Pulsar Function. In order to ensure that when batch is turned on, we can also ensure that messages are distributed to different consumers in the correct order, so we need to support the batch builder of `KEY_BASED` in Pulsar Functions. ### Modifications - Add `--batch-builder` for Pulsar Functions - Add `--batch-builder` for Pulsar Sources - Add test case
### Motivation The changing base on #8523 ### Modifications Support key_base for python function
Signed-off-by: xiaolong.ran <rxl@apache.org> ### Motivation The changing base on #8523 ### Modifications Support key_based batch builder for Go Functions
Signed-off-by: xiaolong.ran <rxl@apache.org> ### Motivation The changing base on apache#8523 ### Modifications Support key_based batch builder for Go Functions
Signed-off-by: xiaolong.ran rxl@apache.org
Motivation
Currently, we support the Key_Shared subscription mode in Pulsar Function. In order to ensure that when batch is turned on, we can also ensure that messages are distributed to different consumers in the correct order, so we need to support the batch builder of
KEY_BASED
in Pulsar Functions.Modifications
--batch-builder
for Pulsar Functions--batch-builder
for Pulsar Sources