add passSourceMessageProperty switch to pulsar functions#6318
Merged
sijie merged 2 commits intoapache:masterfrom Feb 13, 2020
Merged
Conversation
sijie
approved these changes
Feb 13, 2020
codelipenghui
added a commit
that referenced
this pull request
Mar 26, 2020
### Motivation There are some tests that failed in the master branch related to functions. For example https://github.com/apache/pulsar/blob/master/pulsar-broker/src/test/java/org/apache/pulsar/functions/worker/PulsarFunctionE2ESecurityTest.java#L630, the test checks the properties of the message are forwarded, but we don't call `functionConfig.setForwardSourceMessageProperty(true)`, the reason is #6318 introduce `forwardSourceMessageProperty ` to enable or disable message properties forward in functions. But the default value is false, this will break previous behavior. So this PR set it to true to keep forward compatible. And getApiException method (https://github.com/apache/pulsar/blob/master/pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/BaseResource.java#L235) is very confusing, this will cause the async function admin method that #6580 introduced can't complete when met exceptions. BTW, I have no idea why #6318 and #6580 can pass the CI check. This test is there long ago.
huangdx0726
pushed a commit
to huangdx0726/pulsar
that referenced
this pull request
Aug 24, 2020
Fixes apache#5116 ### Motivation Based on the request in apache#5116, adding this flag can help reduce ambiguity and increase flexibility. By default, the function passes source message properties. ### Modifications - add flags in pulsar admin client tools - update functions proto and functionConfig to contain the flag - update the pulsar sink to pass source message property if the flag is set
huangdx0726
pushed a commit
to huangdx0726/pulsar
that referenced
this pull request
Aug 24, 2020
### Motivation There are some tests that failed in the master branch related to functions. For example https://github.com/apache/pulsar/blob/master/pulsar-broker/src/test/java/org/apache/pulsar/functions/worker/PulsarFunctionE2ESecurityTest.java#L630, the test checks the properties of the message are forwarded, but we don't call `functionConfig.setForwardSourceMessageProperty(true)`, the reason is apache#6318 introduce `forwardSourceMessageProperty ` to enable or disable message properties forward in functions. But the default value is false, this will break previous behavior. So this PR set it to true to keep forward compatible. And getApiException method (https://github.com/apache/pulsar/blob/master/pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/BaseResource.java#L235) is very confusing, this will cause the async function admin method that apache#6580 introduced can't complete when met exceptions. BTW, I have no idea why apache#6318 and apache#6580 can pass the CI check. This test is there long ago.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5116
Motivation
Based on the request in #5116, adding this flag can help reduce ambiguity and increase flexibility. By default, the function passes source message properties.
Modifications
Verifying this change
This change is already covered by existing tests, such as
pulsar-functions,pulsar-client-tools.(example:)
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesDocumentation