NIFI-8498: Optional removal of fields with UpdateRecord#5121
NIFI-8498: Optional removal of fields with UpdateRecord#5121pgyori wants to merge 1 commit intoapache:mainfrom
Conversation
|
UpdateRecord_field_removal.xml.zip At this point the PR is a draft.
Not yet supported:
The way field removal is triggered will be changed. Currently in UpdateRecord, a new dynamic property needs to be added, the name of the property should be the RecordPath to the field that needs to be removed and the value should be marked empty. Also, Replacement Value Strategy needs to be set to Record Path Value. The unit tests have not yet been pushed. Those will arrive in a future commit. Attached a flow template that can be used for testing. |
| getParentRecord().get().setArrayValue(getField().getFieldName(), getArrayIndex(), newValue); | ||
| } | ||
|
|
||
| @Override |
There was a problem hiding this comment.
Would you explain why this method is empty in a nested comment and/or throw an UnsupportedOperationException?
| */ | ||
| void updateValue(Object newValue, DataType dataType); | ||
|
|
||
| void remove(); |
There was a problem hiding this comment.
Can you please add Javadoc here to be consistent with the other interface methods?
| getParentRecord().get().setMapValue(getField().getFieldName(), getMapKey(), newValue); | ||
| } | ||
|
|
||
| @Override |
| if (parent != null) { | ||
| if (parent.getDataType() instanceof RecordDataType) { | ||
| Optional<RecordField> childFieldOptional = ((RecordDataType) parent.getDataType()).getChildSchema().getField(childName); | ||
| if (childFieldOptional.isPresent()) { |
There was a problem hiding this comment.
If you like functional style - I like that it's more compact - this can be replaced by:
childFieldOptional.map(recordField -> missingChild(fieldValue, recordField)).orElseGet(() -> missingChild(fieldValue));
| if (value == null) { | ||
| return missingChild(fieldValue); | ||
| final Optional<RecordField> childFieldOptional = record.getSchema().getField(childName); | ||
| if (childFieldOptional.isPresent()) { |
There was a problem hiding this comment.
Same here:
childFieldOptional.map(recordField -> missingChild(fieldValue, recordField)).orElseGet(() -> missingChild(fieldValue));
|
|
||
| for (RecordField schemaField : schema.getFields()) { | ||
| Object fieldValue = getValue(schemaField); | ||
| if (fieldValue != null && fieldValue instanceof Record) { |
There was a problem hiding this comment.
I think the null check here is not necessary, instanceof returns false for nulls.
| if (replacementRecordPath.isAbsolute()) { | ||
| record = processAbsolutePath(replacementRecordPath, result.getSelectedFields(), record); | ||
| } else { | ||
| record = processRelativePath(replacementRecordPath, result.getSelectedFields(), record); |
There was a problem hiding this comment.
I don't think the assignment here is necessary. processRelativePath returns the record passed in the argument. The method can be void. Can it be?
|
Closing this PR as I eventually implemented this functionality in #5381 |
2 participants
https://issues.apache.org/jira/browse/NIFI-8498
Description of PR
Enables removing fields from records.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically
main)?Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not
squashor use--forcewhen pushing to allow for clean monitoring of changes.For code changes:
mvn -Pcontrib-check clean installat the rootnififolder?LICENSEfile, including the mainLICENSEfile undernifi-assembly?NOTICEfile, including the mainNOTICEfile found undernifi-assembly?.displayNamein addition to .name (programmatic access) for each of the new properties?For documentation related changes:
Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.