NIFI-6999 - Made changes to load flow.xml files using streams. Update…#4715
NIFI-6999 - Made changes to load flow.xml files using streams. Update…#4715thenatog wants to merge 4 commits intoapache:mainfrom
Conversation
…d tests. NIFI-6999 - Slight change to test to check for WARN message. NIFI-6999 - Removed very large flow file and test that uses it. This test ran for about 2 minutes so was excessive to keep in. The other changed tests to handle streams proves the functionality. A large file can be used on the command line to manually test large flow files. Some other cleanup.
|
This can be tested with the following files: The attached file is 10mb gzipped, and ~1GB unzipped. It has a processor with an encrypted field repeated many times: The command to run is: |
exceptionfactory
left a comment
There was a problem hiding this comment.
The refactored implementation looks good. See individual comments for small improvements. As noted in some of the comments on the unit test, some amount of cleanup there would be helpful.
...oolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
Outdated
Show resolved
Hide resolved
...oolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
Outdated
Show resolved
Hide resolved
...oolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
Outdated
Show resolved
Hide resolved
...oolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
Outdated
Show resolved
Hide resolved
...oolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
Outdated
Show resolved
Hide resolved
...it-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
Outdated
Show resolved
Hide resolved
...it-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
Outdated
Show resolved
Hide resolved
...it-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
Outdated
Show resolved
Hide resolved
...it-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
Outdated
Show resolved
Hide resolved
...it-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
Outdated
Show resolved
Hide resolved
…adability as per code review.
|
Updated PR to remove junk comments and improve readability etc. as per your recommendation. |
exceptionfactory
left a comment
There was a problem hiding this comment.
Thanks for making the adjustments, looks much cleaner. See comments for three additional minor notes.
...it-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
Outdated
Show resolved
Hide resolved
| public String flowXmlPath | ||
| public String outputFlowXmlPath | ||
| public static flowXmlPath | ||
| public static outputFlowXmlPath |
There was a problem hiding this comment.
Is there a reason these values are static?
There was a problem hiding this comment.
These were changed to static to allow static access in static main() for calling loadFlowXml(flowXmlPath). I changed this slightly to take the path as a parameter to allow easier testing for the loadFlowXml method. Not sure how successful it was though.
...it-encrypt-config/src/test/groovy/org/apache/nifi/properties/ConfigEncryptionToolTest.groovy
Outdated
Show resolved
Hide resolved
exceptionfactory
left a comment
There was a problem hiding this comment.
The last round of changes look good. See one additional comment on method parameter naming to avoid shadowing class variable.
...oolkit-encrypt-config/src/main/groovy/org/apache/nifi/properties/ConfigEncryptionTool.groovy
Outdated
Show resolved
Hide resolved
|
@exceptionfactory Updated with latest request. Let me know if there's anything else. |
|
Thanks for making the changes @thenatog. Looks good! |
|
Will merge. |
…d tests. NIFI-6999 - Slight change to test to check for WARN message. NIFI-6999 - Removed very large flow file and test that uses it. This test ran for about 2 minutes so was excessive to keep in. The other changed tests to handle streams proves the functionality. A large file can be used on the command line to manually test large flow files. Some other cleanup. NIFI-6999 - Removed comments and altered the code a little bit for readability as per code review. NIFI-6999 - Removed commented code NIFI-6999 - Renamed variable and removed assert comment. Signed-off-by: Nathan Gough <thenatog@gmail.com> This closes apache#4715.
…d tests. NIFI-6999 - Slight change to test to check for WARN message. NIFI-6999 - Removed very large flow file and test that uses it. This test ran for about 2 minutes so was excessive to keep in. The other changed tests to handle streams proves the functionality. A large file can be used on the command line to manually test large flow files. Some other cleanup. NIFI-6999 - Removed comments and altered the code a little bit for readability as per code review. NIFI-6999 - Removed commented code NIFI-6999 - Renamed variable and removed assert comment. Signed-off-by: Nathan Gough <thenatog@gmail.com> This closes apache#4715.
…d tests.
NIFI-6999 - Slight change to test to check for WARN message.
NIFI-6999 - Removed very large flow file and test that uses it. This test ran for about 2 minutes so was excessive to keep in. The other changed tests to handle streams proves the functionality. A large file can be used on the command line to manually test large flow files. Some other cleanup.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Enables X functionality; fixes bug NIFI-YYYY.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically
main)?Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not
squashor use--forcewhen pushing to allow for clean monitoring of changes.For code changes:
mvn -Pcontrib-check clean installat the rootnififolder?LICENSEfile, including the mainLICENSEfile undernifi-assembly?NOTICEfile, including the mainNOTICEfile found undernifi-assembly?.displayNamein addition to .name (programmatic access) for each of the new properties?For documentation related changes:
Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.