Skip to content

NIFI-16130 - Fix ReplayLastEventEndpointMerger to count per-node replays correctly#11450

Merged
exceptionfactory merged 2 commits into
apache:mainfrom
pvillard31:NIFI-16130
Jul 22, 2026
Merged

NIFI-16130 - Fix ReplayLastEventEndpointMerger to count per-node replays correctly#11450
exceptionfactory merged 2 commits into
apache:mainfrom
pvillard31:NIFI-16130

Conversation

@pvillard31

@pvillard31 pvillard31 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

NIFI-16130 - Fix ReplayLastEventEndpointMerger to count per-node replays correctly

Provenance event IDs are local counters per node, starting at 0 and incrementing independently. In a multi-node cluster two different nodes can legitimately have the same numeric event ID for entirely different physical provenance events.

ReplayLastEventEndpointMerger was using a HashSet<Long> to accumulate event IDs returned by each node after a "replay last event" request. Because both nodes in a 2-node cluster where each has processed the same number of events will return the same local event ID, the HashSet silently deduplicated them and reported eventsReplayed.size() == 1 instead of 2. This caused ClusteredReplayProvenanceIT.testReplayLastEvent (with ReplayEventNodes.ALL) to fail with expected: <2> but was: <1>.

Change: Replace HashSet<Long> with ArrayList<Long> for the event ID accumulator so each node's replay is counted independently regardless of matching local event IDs.

Also adds: ReplayLastEventEndpointMergerTest — there was no unit test for this merger class. The test covers the colliding-ID case (regression test for this fix), the distinct-ID case, and the partial-failure case.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000
  • Pull request contains commits signed with a registered key indicating Verified status

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using ./mvnw clean install -P contrib-check
    • JDK 21
    • JDK 25

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

…ays correctly

Provenance event IDs are local counters per node and reset to zero on each new
NiFi installation. In a multi-node cluster, two different nodes can legitimately
assign the same numeric event ID to different physical provenance events.

ReplayLastEventEndpointMerger used a HashSet<Long> to accumulate event IDs from
all nodes. When two nodes each replay their last event and happen to have the
same local event ID (which occurs whenever both nodes have processed an equal
number of events), the set deduplicates them and reports eventsReplayed.size()
as 1 instead of 2. This caused ClusteredReplayProvenanceIT to fail with
"expected: <2> but was: <1>".

Change the accumulator to ArrayList<Long> so each node's replay is counted
independently, regardless of colliding local event IDs.

Add ReplayLastEventEndpointMergerTest to cover both the colliding-ID case
(regression test for this fix) and the distinct-ID and partial-failure cases.
@pvillard31 pvillard31 added the type: testing Pull requests for changes to test components label Jul 21, 2026

@exceptionfactory exceptionfactory left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pvillard31, the functional change to List<Long> for Provenance IDs looks good. For the new test, the public modifiers can be removed, and it looks like static values, as well as the NodeIdentifier objects themselves, could be declared once as static final members.

@exceptionfactory exceptionfactory left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pvillard31, the latest version looks good.

@exceptionfactory
exceptionfactory merged commit 15de342 into apache:main Jul 22, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: testing Pull requests for changes to test components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants