NIFI-12165 Changed the properties "Custom Transformation Class Name" and "Custom Module Directory" to depend on the "Jolt Transformation DSL" property when its value is "Custom"

[dan-s1]

Summary

NIFI-12165

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using mvn clean install -P contrib-check
  • JDK 21

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[ChrisSamo632]

The same problem exists for JoltTransformRecord, that should be updated in this PR too

[dan-s1]

@ChrisSamo632 @exceptionfactory
I understand the history from #7782 that there had to be two processors JoltTransformJSON and JoltTransformRecord. But my question is why is there duplicate code between the two processors? This MR and NIFI-11959 (#7678) are consequences of duplicate code (i.e. the need to make the same fix in both places). Is there something we can do here to refactor and extract the duplicate code to be shared with these two classes?

[ChrisSamo632]

Thanks for the updates @dan-s1 , LGTM 👍 merging

[ChrisSamo632]

@ChrisSamo632 @exceptionfactory I understand the history from #7782 that there had to be two processors JoltTransformJSON and JoltTransformRecord. But my question is why is there duplicate code between the two processors? This MR and NIFI-11959 (#7678) are consequences of duplicate code (i.e. the need to make the same fix in both places). Is there something we can do here to refactor and extract the duplicate code to be shared with these two classes?

@dan-s1 absolutely, I was thinking similar when I made the comment about the *Record processor in this PR, although I didn't think it was worth extending this PR to cover such a refactor. Could you raise a separate Jira ticket to refactor these processors to merge the logic where possible, e.g. the PropertyDescriptors and such, at the very least, could be merged together.

It would be good to explore whether things such as the custom UI available in the JoltTransformJSON could be merged together with JoltTransformRecord in the future, plus the handling of JOLT logic where possible. I'm not overly familiar with these processor, but without spending much time looking at the code, I'd hope that the only real difference should be whether the processor is applying the transform to the entire FlowFile content (i.e. in the *JSON processor) or every Record within (e.g. JSON Object within an Array, or every line of an ndjson/jsonl/json-ld file) - once the appropriate content has been read into memory, presumably the same JOLT transformation logic should be executed, then the result serialised either directly as the FlowFile's content (i.e. for *JSON) or via the configured Record Set Writer (i.e. for *Record).

I've tried to do a similar thing for the PutElastisearch* processors previously (i.e. *JSON and *Record), which both extend an AbstractPutElasticsearch base in an attempt to minimise repetition and avoid maintenance problems/inconsistent behaviour between the processors.