Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace instead of ignore so that we can update the offset #1335

Merged
merged 3 commits into from Nov 21, 2016

Conversation

tpetr
Copy link
Contributor

@tpetr tpetr commented Nov 2, 2016

@jhaber what do you think about this tweak? using REPLACE instead of IGNORE allows us to update the offset when we skip over a bytes in the beginning of a chunk.

/cc @ssalinas #1334

*/
@VisibleForTesting
MesosFileChunkObject parseResponseBody(Response response) throws IOException {
// not thread-safe, need to make a new one each time;
CharsetDecoder decoder = StandardCharsets.UTF_8.newDecoder().onMalformedInput(CodingErrorAction.IGNORE);
CharsetDecoder decoder = StandardCharsets.UTF_8.newDecoder().onMalformedInput(CodingErrorAction.REPLACE);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe call replaceWith(REPLACEMENT_CHARACTER) to be more explicit?

@ssalinas ssalinas modified the milestone: 0.13.0 Nov 4, 2016
@tpetr tpetr merged commit dfd0835 into invalid-utf8 Nov 21, 2016
@ssalinas ssalinas modified the milestones: 0.14.0, 0.13.0 Dec 9, 2016
@ssalinas ssalinas deleted the invalid-utf8-2 branch February 8, 2017 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants