NIFI-11995 Support multiple output formats for message headers in ConsumeAMQP

[umarhussain15]

Summary

NIFI-11995 extends ConsumeAMQP processor and allows the user to configure how to output headers from a received message. It provides three options:

• Output headers to amqp$headers attribute as string joined by the provided separator (Default) -> backward compatible
• Output headers to amqp$headers attribute as a valid JSON string. Headers with null value will be present as well in json with null value
• Output each header key as <Header Key Prefix>.<header key> attribute. It will not put headers which are null in message
  • Property <Header Key Prefix> specifies the prefix value, defaults to consume.amqp

Test cases updated for new changes to the processor

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using mvn clean install -P contrib-check
  • JDK 17

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[nandorsoma]

Thank you, @umarhussain15, for your contribution. I noticed that there are a bunch of unnecessary reformats. Could you revert the ones where you didn't change the logic before we proceed?

[umarhussain15]

Thanks, @nandorsoma, I have updated the pull request.
Can you point me to a resource to configure Java code formatter in IntelliJ IDEA for NiFi project?
My current formatter (google-java-format) is using 4 spaces for indentation, I think which changes the formatting in code.

[nandorsoma]

Thanks @umarhussain15. Unfortunately, we have work to do on this topic. We have a pretty minimal checkstyle.xml in our root folder. In its current form, it is not enough to import using the checkstyle plugin in IDEA that you would typically do. So, the goal is to create a custom format that honors the checkstyle rules and doesn't force reformating existing code. You should be good if you use IntelliJ IDEA's Default with a little tweak setting the max line length to 200.

[exceptionfactory]

Thanks for your work on this feature @umarhussain15.

Do you have any additional feedback on this pull request @nandorsoma?

I noted one concerning about handling failures for JSON formatting.

[umarhussain15]

Hi @nandorsoma /@exceptionfactory, the ci-workflow failed for the MacOS build due to java heap space issue. Do I need to make some changes, or will you re-run the failed stage?

[exceptionfactory]

Thanks for your patience on this pull request @umarhussain15. This looks close to completion, I noted a few implementation details and wording recommendations, and then it should be ready to go.

[umarhussain15]

Thanks, @exceptionfactory. The suggestions look good, and I have applied them to the PR.

[exceptionfactory]

Thanks for the updates @umarhussain15, it looks like the latest commit missed updating a reference to HEADER_PREFIX, resulting in compilation failures. Can you correct that as well?

[codecov-commenter]

Codecov Report

❗ No coverage uploaded for pull request base (main@eb6a5a2). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #7652   +/-   ##
=======================================
  Coverage        ?   50.60%           
  Complexity      ?    35103           
=======================================
  Files           ?     5251           
  Lines           ?   275605           
  Branches        ?    30435           
=======================================
  Hits            ?   139476           
  Misses          ?   126049           
  Partials        ?    10080           

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

[exceptionfactory]

Thanks for working through the feedback @umarhussain15, the latest version looks good! +1 merging