Syntax highlighting is off with unicode characters

[CamilleScholtz]

micro index.html 
Micro encountered an error: runtime error: index out of range
runtime.errorString runtime error: index out of range
/usr/lib/go/src/runtime/panic.go:443 (0x46d329)
    gopanic: reflectcall(nil, unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz))
/usr/lib/go/src/runtime/panic.go:27 (0x46b7b5)
    panicindex: panic(indexError)
/src/pkgmk/sources/micro/cmd/micro/cursor.go:21 (0x40f148)
/src/pkgmk/sources/micro/cmd/micro/highlighter.go:425 (0x415f57)
/src/pkgmk/sources/micro/cmd/micro/view.go:386 (0x434be2)
/src/pkgmk/sources/micro/cmd/micro/micro.go:276 (0x41937c)
/usr/lib/go/src/runtime/proc.go:188 (0x46ee40)
    main: main_main()
/usr/lib/go/src/runtime/asm_amd64.s:1998 (0x49e0d1)
    goexit: BYTE    $0x90   // NOP

Happens both when scrolling and when using the arrows, it always seems to be the same line in this particular file: https://github.com/onodera-punpun/rice/blob/gh-pages/index.html#L42

Disabling syntax highlighting stops making it crash.

[zyedidia]

Commit f730864 fixes the crash, but the underlying problem still exists, which is that unicode characters cause the indexing in Go's regexp engine to give different indices than I expect. This is related to #134.

Go gives matches in byte indices (so I know the number of the byte that matches) but I want it in character indices so I know which character matches. Generally this is not a problem because in ASCII 1 byte = 1 character, but when using more unicode characters this doesn't hold true anymore.

In this case the crash is caused by the double width character in the title of the document (this character: 。).

[CamilleScholtz]

Ahh yes, it seems like all syntax highlighting and selections are off by a few characters now.

[CamilleScholtz]

Have you looked at https://github.com/gdamore/tcell/blob/815826d8a7207fcc8fd33e01d9679b1cb966b2d3/_demos/unicode.go & https://github.com/mattn/go-runewidth ?

I don't really know anything about this more advanced stuff, but it seems to be about these double width characters.