emoji characters cause issues such as offset to search highlighting as well as hyperlink anomalies

[floufleobli]

(for what it's worth, testing shows this issue does not affect gnote ...)

emoji characters (e.g. 😁) inserted into a tomboy note cause search highlighting to be offset for matches appearing somewhere after the emoji character. for example, say you have a note with the following content:

test1
😁
test2

if you search for "test", the string "test" will be highlighted in the first line, whereas the string "est2" will be highlighted in the third line.

emoji characters also cause anomalies in hyperlinking. after inserting e.g. "😁" into a note, if you type "http://google.com" somewhere after the emoji, once you type "http://", the text will appear as a hyperlink, but once you type "g" the leading "h" will no longer appear as part of the hyperlink; once you type "o", the leading "h" will again appear as part of the hyperlink (this alternation continues as you continue to type).

Note that I originally assumed that all multibyte characters cause this issue, but that doesn't seem to be the case, as the ellipsis character (…), as an example, doesn't cause the same issue.

[alex-ter]

Sounds like a bug to me, thanks for reporting.

[alex-ter]

So apparently this is buffer.GetSlice() at line 1230 of NoteWindow.c returning two question marks instead of one symbol when it has such a symbol and that screws further indexing. I haven't yet found what exactly is causing this, but I'm looking into this.

[alex-ter]

So at the end of the day it turned out a slightly different problem and actually even two (with the same root cause). Both search highlighting and URL activation were using .NET's string indexes, which apparently are not covering all Unicode symbols as symbols and fall back to bytes for things like these emojis.

I've fixed both places to use a different approach to determining the area to highlight/activate and it passed my simple tests, so I've got this in. Feel free to test it out and let me know if you'd want an engineering build for Windows to give it a try.

[floufleobli]

thanks! :-)