Highlight should not rely on normalized identifiers

[cjxgm]

With the following source code

import strutils
echo "".t_o_l_o_w_e_r

nimsuggest gives the following highlight for the t_o_l_o_w_e_r:

highlight       skProc  2       8       7

which should really be

highlight       skProc  2       8       13

I suppose that's because in the library it's written as toLower, or the highlight is based on normalized identifier tolower, thus the wrong length of 7.

[Araq]

I suppose that's because in the library it's written as toLower, or the highlight is based on normalized identifier tolower, thus the wrong length of 7.

Indeed and it's much work to fix. We need to look at the original text file to fix it much like an editor could. Maybe we shouldn't output the token length at all? ;-)

[cjxgm]

I don't know about the internals of nim compiler/nimsuggest, but how about storing the original identifier along side the nomalized one in the symbol table?

We need to look at the original text file to fix it much like an editor could. Maybe we shouldn't output the token length at all? ;-)

Well that's what I did in my highlighter: ignore the length and search the original text file. But that feels awkward. I prefer a fix in nim compiler/nimsuggest.