NIFI-13779: Missing Some Data Provenance Events from Python

[bobpaulin]

• Use cloned flow file as original
• Transform inbound flow file to ensure it gets picked up in Provenance Events

Summary

NIFI-13779

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using mvn clean install -P contrib-check
  • JDK 21

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[exceptionfactory]

Thanks for proposing this change @bobpaulin. This seems to be a better way to handle the lineage of input FlowFiles, reflecting CONTENT_MODIFIED instead of cloning. However, tagging @markap14 for additional review and consideration.

[markap14]

Hey @bobpaulin thanks for updating! I do think this makes a lot of sense. I had 2 thoughts about the PR though. I commented inline about the name of the transformed and originalCloned variables. Normally I try not to quibble over variable names, but given how frequently they're used in the code I think it's helpful to use clear names.

The other thought is that the Provenance is now going to always show a CLONE event followed by a DROP event if original is auto-terminated, which is the default and very common. But we just recently added a new method to ProcessContext: boolean isAutoTerminated(Relationship relationship);

Perhaps we should not even clone the FlowFile at all if the original FlowFile is auto-terminated. That way, there's no CLONE or DROP event, and the lineage is much clearer.

[markap14]

It feels weird to me to call the FlowFile that we are pulling from an input queue transformed. Perhaps we should name the variable simply flowFile? And then we can call the clone just simply clone rather than originalCloned? I think that would make the code a little easier to read personally.

[exceptionfactory]

I agree with you @markap14, I think just calling this flowFile would be simpler and avoid some confusion.

[markap14]

Thanks for pinging @exceptionfactory. I do agree with the approach but think we should clean it up a bit more, as noted above.

[bobpaulin]

Thanks @markap14 and @exceptionfactory as always for the feedback. This cleaned up nicely with the usage of isAutoTerminated will look forward to using that more!

[exceptionfactory]

Thanks for the input @markap14, and thanks for making the updates @bobpaulin, the latest version looks good! +1 merging