Plugin is very fragile with respect to UTF-8

[kittylyst]

If a .adoc contains incorrect UTF-8, it throws exceptions and does not display properly. Could this be made less of a hard crash - getting some output so I can find the bad UTF-8 in the document would be useful.

[bodiam]

@kittylyst Do you have a testcase for this? I'm not sure how to reproduce this.

[kittylyst]

@bodiam It's been months since I looked at this, but if memory serves, the problem was caused by a \n character being incorrectly inserted by stupid line-wrapping software partway through a multibyte UTF-8 sequence.

I don't have the test case to hand, as it came up during some client work which I don't have access to any more. A probable minimal test case is:

1. Take a 3-byte UTF-8 sequence
2. Insert a \n in between offset 1 and offset 2 in the 3-byte sequence
3. Profit (?)

[bodiam]

Hi @kittylyst , I've tried a simple test by opening this document: http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt

I get a NPE in at apple.awt.CAccessible.getAccessibleContext(CAccessible.java:74), I only get partial rendering, but I don't see a crash. Would this be good enough for you?

[kittylyst]

That is a significant torture test - if you're not seeing a crash & getting partial rendering from this, feel free to close this bug.

[bodiam]

It was the worst I could find ;-). I doesn't crash on my Mac, only partial rendering, but that seems good enough to fix the document. I'm in the process of releasing 0.2, which should be approved in 1-2 days. If you have a different OS than Mac, would you care to retest it?

[mojavelinux]

Thanks for the torture test. That's super useful. I'm going to see if I can get Asciidoctor core to drop invalid characters and only issue a warning so that it at least processes the file.