You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.
Unlike readLines(), readr::read_lines() doesn't have a skipNul=TRUE option. Hence it either chokes on null characters, or strings get truncated at the null character. Could a skipNul=TRUE option be implemented?
The text was updated successfully, but these errors were encountered:
It looks like the signature of the Token constructor changed, but the
calls in TokenizerLine were not properly updated for the change. It
wasn't causing a compilation error because the hasNull boolean was being
implicitly coerced to an integer, and the 0 was being coerced to a
boolean, (very bad!).