Partially implement translate_newlines config in JVM backend's DecoderInstance #724
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a boolean member for it (defaults to false) and set it based on the config hash if given. Currently only takeAllChars uses it, because that's all that was needed to get both t/nqp/019-file-opt.t and t/nqp/116-streaming-decoder.t to pass on Windows.
MasterDuke17
changed the title
jnthn
reviewed
May 25, 2021
| @@ -156,7 +159,8 @@ public synchronized String takeAllChars(ThreadContext tc) { | |||
| decoder.flush(target); | |||
| decoder.reset(); | |||
| } | |||
| return Normalizer.normalize(decodedBuffer(target), Normalizer.Form.NFC); | |||
| String normalized = Normalizer.normalize(decodedBuffer(target), Normalizer.Form.NFC); | |||
| return translate_newlines ? normalized.replace("\r\n", "\n") : normalized; | |||
There was a problem hiding this comment.
I think this should happen in takeChars and takeAvailableChars too, probably by factoring out this line into a method. Otherwise it seems to be doing the right thing.
There was a problem hiding this comment.
Cool, I'll give that a shot and see what happens.
There was a problem hiding this comment.
Done. No change in tests.
Also pull it out into a method.
jnthn
approved these changes
May 26, 2021
MasterDuke17
deleted the
implement_translating_newlines_for_Decoder.takeAllChars_on_jvm
branch
usev6
added a commit
to Raku/roast
that referenced
this pull request
May 30, 2021
Fixed with Raku/nqp#724.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add a boolean member for it (defaults to false) and set it based on the
config hash if given.
Currently only takeAllChars uses it, because that's all that was needed
to get both t/nqp/019-file-opt.t and t/nqp/116-streaming-decoder.t to
pass on Windows with the JVM backend.
This will allow us to turn back on the JVM-on-Windows NQP test job in CI, which I accidentally did in #719, so after a rebase that PR should all pass.