NIFI-12670 JoltTransform processors incorrectly encode/decode text on Windows #8842
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… the Jolt Specification on Windows
exceptionfactory
requested changes
May 16, 2024
There was a problem hiding this comment.
Thanks for proposing this correction and improvement @jrsteinebrey.
The changes in the AbstractJoltTransform class make sense, in terms of using UTF-8, however, we should avoid making changes to nifi-api interfaces. In particular, the createReader() method without a character set specified could easily lead to incorrect usage, falling back to platform-default character encoding.
For that reason, I recommend constructing the Reader with UTF-8 in the AbstractJoltTransform class as a way forward.
nifi-api/src/main/java/org/apache/nifi/components/resource/ResourceReference.java
Outdated
Show resolved
Hide resolved
nifi-api/src/main/java/org/apache/nifi/components/resource/Utf8TextResource.java
Outdated
Show resolved
Hide resolved
exceptionfactory
approved these changes
May 17, 2024
There was a problem hiding this comment.
Thanks for adjusting the approach @jrsteinebrey, the latest version looks good. +1 merging
shubhluck
pushed a commit
to shubhluck/nifi
that referenced
this pull request
Jun 1, 2024
- Specified UTF-8 encoding for reading Jolt Transform to avoid decoding issues on Windows or platforms with different default character sets This closes apache#8842 Signed-off-by: David Handermann <exceptionfactory@apache.org>
shubhluck
pushed a commit
to shubhluck/nifi
that referenced
this pull request
Jun 1, 2024
- Specified UTF-8 encoding for reading Jolt Transform to avoid decoding issues on Windows or platforms with different default character sets This closes apache#8842 Signed-off-by: David Handermann <exceptionfactory@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
NIFI-12670 JoltTransform processors incorrectly encode/decode text on the Jolt Specification on Windows
Summary
NIFI-12670
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking
Pull Request Tracking
NIFI-12670NIFI-12670Pull Request Formatting
mainbranchVerification
Add ran the processors in the UI and confirmed they worked correctly with linux and Windows encoded files.
I also added unit tests.
Build
mvn clean install -P contrib-checkUI Contributions
No UI changes
Licensing
New new dependencies
Documentation
No documentation changes.