Text highlight of wide characters (i.e. chinese) doesn't select right

[pkerichang]

Brief summary of issue / Description of requested feature:

When you try to select text in the text area while using wide character encoding (like Big5), the text selection/highlighting seems to treat wide characters as narrow characters.

For example, if a line contains 12 chinese characters, and you move your mouse to the beginning of the sentence and start click-and-dragging, you'll notice that the text highlighting move twice as far as your mouse movement, so you only need to move your mouse 6 characters to the right to select the entire sentence.

Steps to reproduce the issue / Reasons for adding feature:

1. Connect to a MUD with traditional chinese encoding (I use wfmud.org:7788)
2. Change character encoding to Big5.
3. Try to select any text in the welcome message to notice the behavior.

Error output / Expected result of feature

Text highlight should follow where the mouse is, just like in normal GUI.

Extra information, such as Mudlet version, operating system and ideas for how to solve / implement:

I think it's pretty clear that somewhere someone made an assumption about character width. If someone wants to give me some pointers I'm happy to investigate this myself.

[vadi2]

@SlySven could you help with the pointers?

[SlySven]

It is a "known" issue and is something that I have identified as something to be sorted out - and I have mentioned it as such in the recent past. The stalled #1633 was a hulking great clean-up to the TBuffer code that I wanted to get in place before I started on fixing this. *sigh*

[vadi2]

Anything that @pkerichang can start working on in the meantime while you do the Discord stuff?

[vadi2]

@pkerichang are you able to work on this?

[pkerichang]

Hello,

It seems like the last status is that there's some clean-up to do with the TBuffer code first? Is that finished? If someone can give me some pointers I'll see what I can do.

[vadi2]

That cleanup hasn't been moving much and blocking this work, which isn't really healthy for the project :/ so might as well do this and the other PR can be adapted.

[vadi2]

@pkerichang interested in having a look at this?

[seikichin]

Brief summary of issue / Description of requested feature:

When you try to select text in the text area while using wide character encoding (like Big5), the text selection/highlighting seems to treat wide characters as narrow characters.

For example, if a line contains 12 chinese characters, and you move your mouse to the beginning of the sentence and start click-and-dragging, you'll notice that the text highlighting move twice as far as your mouse movement, so you only need to move your mouse 6 characters to the right to select the entire sentence.

Steps to reproduce the issue / Reasons for adding feature:

1. Connect to a MUD with traditional chinese encoding (I use wfmud.org:7788)
2. Change character encoding to Big5.
3. Try to select any text in the welcome message to notice the behavior.

Error output / Expected result of feature

Text highlight should follow where the mouse is, just like in normal GUI.

Extra information, such as Mudlet version, operating system and ideas for how to solve / implement:

I think it's pretty clear that somewhere someone made an assumption about character width. If someone wants to give me some pointers I'm happy to investigate this myself.

good，You explained my intention correctly.

[SlySven]

From what I recall the selection code is based upon counting the QChar/TChar in the string and working out where the mouse click is along the line. My stalled code actually measured how much across the line each grapheme (visual character) took when it was drawn in the TTextEdit code - which allowed the use of proportional fonts, but instead we went down the (simpler, I guess) route of using the wcwidth code which effective reports whether a grapheme will take one or two spaces.

Note that currently when we do draw wide characters I thought that things sometimes goes wrong and we do not allow enough space so that the next character is drawn partially over the previous one. I suspect we may have to check that the character painting code is working as we expect then we can be sure that we can determine whereabouts in a partially selected line the mouse click point is so that we highlight the right number of characters (at a minimum we should never stop a selection to the right on a high surrogate and to the left on a low surrogate QChar - mind you I think we have that in place already but I do no have the code/application open in front of me at this precise moment to check)...

[cantona]

the issue exist on a UTF8 chinese MUD, FYI

[SlySven]

Looking at this now - note that it is a known issue...

[cantona]

Looking at this now - note that it is a known issue...

Thanks, look forward to hearing your good news!

[vadi2]

@pkerichang @lostsnow @seikichin @cantona can you please test if #2790 fixes this issue?

[lostsnow]

@pkerichang @lostsnow @seikichin @cantona can you please test if #2790 fixes this issue?

LGTM

[lostsnow]

lua echo("测试测试测试\t测试测试测试")

When Chinese and tab are on the same line, the selection is still incorrect.

[vadi2]

[vadi2]

Unrelated - we now have a QQ group, join! https://forums.mudlet.org/viewtopic.php?f=14&p=45201

[vadi2]

Tab display in Mudlet does not work well it would seem:

So text selection will not work right.