Add missing unicode whitespace support to String methods #10367
Conversation
String#split, String#strip and String#to_i etc. only recognized ascii whitespace, this adds support for unicode whitespace.
straight-shoota
added
kind:bug
| while ptr.value.unsafe_chr.ascii_whitespace? | ||
| ptr += 1 | ||
| end | ||
| ptr += calc_excess_right |
There was a problem hiding this comment.
Is this right? The loop make ptr skip a bunch of whitespaces from the current position and now it will jump exactly the amount of whitespaces at the end of the string. I might be missing a reason why that would lead to the same result in this particular situation.
There was a problem hiding this comment.
The purpose of this piece is to determine if the entire string was consumed (including trailing whitespace). If we add the size of trailing whitespace to the pointer and end up at the end of string, it's all good. But if there's some non-whitespace character between the pointer position and the end of string, adding the size of trailing white space will not move the pointer to the end of string, thus it can be detected as invalid input (if strict == true).
There was a problem hiding this comment.
I suppose the entire unless ptr.value == 0 could be made conditional on strict as well because the trailing whitespace detection is unnecessary without strict.
String#split, String#strip and String#to_i etc. only recognized ascii whitespace, this adds support for unicode whitespace.
Resolves #10312