NIFI-11261 Add Primary Node State handling to GetAzureEventHub #7023
Conversation
- Updated Qpid Proton J from 0.34.0 to 0.34.1
There was a problem hiding this comment.
@exceptionfactory Thanks for adding primary node handling to the processor!
I tested it on a cluster and works as expected. The code also looks good to me.
I have only a minor suggestion. Pls. see below.
|
|
||
| // Set Azure Event Hub Client Identifier using Processor Identifier instead of default random UUID | ||
| final AmqpClientOptions clientOptions = new AmqpClientOptions(); | ||
| clientOptions.setIdentifier(getIdentifier()); |
There was a problem hiding this comment.
The processor's identifier is the same on all cluster nodes so all instances would use the same id.
When the processor is configured to run in "All nodes" mode, it can be confusing to have multiple clients with the same identifier. Also, in "Primary node" mode it would be useful to see which node the client belongs to.
So I suggest adding some node-specific tag in the identifier.
There was a problem hiding this comment.
Thanks for the suggestion, I agree it would be helpful to have the Node Identifier included. I pushed an update to prefix the Client Identifier with the Node Identifier when it is available in a clustered deployment.
- Updated Qpid Proton J from 0.34.0 to 0.34.1 This closes #7023. Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
- Updated Qpid Proton J from 0.34.0 to 0.34.1 This closes apache#7023. Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
Summary
NIFI-11261 Adds Primary Node State Change handling to
GetAzureEventHubto improve Processor behavior in a clustered deployment.Although
ConsumeAzureEventHubprovides the recommend approach for receiving events,GetAzureEventHubcan be configured to run on the cluster primary node only to reduce the potential for duplication.Updates include checking for the Execution Node status and avoiding unnecessary Event Hub Consumer Client creation. The new handler method for Primary Node State changes closes the Consumer Client when a node has its primary status revoked, and creates a new Consumer Client when a node is elected primary.
Additional changes include upgrading the Qpid Proton J dependency from 0.34.0 to 0.34.1 and setting the Consumer Client Identifier to the Processor Identifier for improved connection tracking.
New unit test methods provide basic confirmation of Processor behavior when configured for primary node execution.
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking
Pull Request Tracking
NIFI-00000NIFI-00000Pull Request Formatting
mainbranchVerification
Please indicate the verification steps performed prior to pull request creation.
Build
mvn clean install -P contrib-checkLicensing
LICENSEandNOTICEfilesDocumentation