Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid byte sequence treating #27

Open
johnd0e opened this issue Jan 4, 2020 · 1 comment
Open

Invalid byte sequence treating #27

johnd0e opened this issue Jan 4, 2020 · 1 comment
Labels
bug

Comments

@johnd0e
Copy link

@johnd0e johnd0e commented Jan 4, 2020

I found that luautf8 in some cases treats some invalid sequences as correct utf8.
For example, utf8 string cannot consist of single char in range 128..255.
So when we execute following code in lua 5.3 we get no output:

for i=128,255 do
  local c = string.char(i)
  if utf8.len(c) then
    print(i)
  end
end

But luautf8 considers all such strings as valid.

Even more: luautf8 treats some invalid chars as punctuation:

for i=128,255 do
  local c = string.char(i)
  print(
    i,
    utf8.match(c,'%p'),
    string.match(c,'%p')
  )
end
@johnd0e

This comment has been minimized.

Copy link
Author

@johnd0e johnd0e commented Jan 4, 2020

Perhaps, this is duplicate of #13

@starwing starwing added the bug label Jan 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.