[BUG] Arabic code editor incorrectly editing text when using text selection with backspace and/or delete

[alaaeddin-swidan]

Describe the bug
When editing code in the Arabic language: specifically trying to change some text using simple text selection with the backspace and delete buttons, it is then impossible to determine where the cursor is going to take the action and therefore the editing of the textual code is hindered.

Paste the Hedy code & level
Applies to all levels. But let's take the following as an example: Level 2 -> introduction -> first example code:
الاسم هو هيدي
العمر هو ١٥
قول الاسم وعمرها الان العمر سنة

**Video screen comparing Arabic and English code

2022-12-09_12h30_42.mp4

The actions above show that a shift is happening to the left whenever a selection is done on a letter or a word, the first operations were a backspace: what they do is they remove a character ahead (left of the selection by 2 spaces (I think)) rather than going backward to the right while deleting the character.. Another one is when selecting the whole word الان followed by a backspace: the word itself stays as is (not deleted) but the deletion happens also somewhere to the left (ahead of the word).

2022-12-09_12h32_12.mp4

In the English editor the actions are as expected.

What machine and browser you were using (optional)
Windows. Firefox browser.

[Felienne]

Hi @alaaeddin-swidan!

Is this a new issue (in my experience this has always been the case) Just trying to understand if this is something we broke or that is most likely in Ace.

[vzsky]

I'm not sure whether the arabic one is the same as this case. But with Thai, AceAjax edit seems not to fully support the language.

for example, try the word "คำที่มีปัญหา" in the official(?) ace editor here. When you move the cursor to the back of the word, it extends beyond the actual position of the end of the visual word as there are 0 length characters.
Try putting some English alphabet at where the cursor lands, and it should be append to the word, verifying that the extra spaces are not whitespaces.

For example, here is the problematic word. The actual cursor is right after the word Hedy, as in if you were to type anything, it would append immediately to "Hedy", but the cursor is shown to be further.

I suspect something with this issue on ace : ajaxorg/ace#4512
Or this one in the case of arabic? : ajaxorg/ace#460

In the past, I had used codemirror, where there were no problem with Thai. But still have no up-to-date information.

[alaaeddin-swidan]

Hi @Felienne no this is not a new issue.
Thanks for the input @vzsky. I tried the official Ace editor, the problem shows also there. There seem to be a 10 character shift to the left, this is evident from their text-input-debugger window - i show here another screen video of the issue.
I will investigate further whether a solution exists somehow.

2023-03-06_16h52_22.mp4

[Felienne]

I understand this is very annoying, but at a first glance this seems to be out of our control. We might be able to patch it with some CSS as we have in other cases, but if this is an issue in Ace it will be hard to fix (other then from submitting an issue and maybe a PR to Ace).

Or, as suggested, we can fix it by swapping out the entire editor to CodeMirror. That is a thing we have considered before but of course that'd be a large project that we do not start lightly.

So hopefully you will find an issue in Ace with some setting or other, otherwise this will probably stay on the log for a while.

[Felienne]

In ajaxorg/ace#2548 someone mentions this could be fixed by using a " fixed width font". I am not sure we have to for Arabic or Thai (or whether that even exists?) but it could be worth a try. Diving into ajaxorg/ace#2548 and its many (many!) duplicates might be a good start:

Some things to try:

[Felienne]

The problem is that it is hard for me to try since I don't really know what success/failure looks like.

[reedspool]

Took a look at this. I think if #4090 doesn't get this to a suitable place, then I think the only reasonable alternative is moving to CodeMirror. I know this is a big project so I wouldn't vote that way lightly.

First, to explain the issue concisely, read the first post of this issue from 2013, 10 years ago.

Then later, in 2016, tomerm explains exactly why this is a fundamental flaw in Ace

The short version is that Ace uses a bespoke text rendering engine, and it's apparently been western-text-focused for over 10 years. All these issues you've collected show a major lack of support from the primary contributor, nightwing. The issues generally trail off into "does anyone have a fix for this?" with no supportive replies. More than one of the threads I read mentioned this exact issue being a deciding factor for moving to CodeMirror.

So, either Ace already supports this, and it should be as easy as getting the CSS/font choice right, or the support is not there and it might be another decade before Ace gets it right.

[Felienne]

We are going to (ultimately) replace Ace with CodeMirror (see: #4339) so this specific issue will be addressed but it might take a while.

[Felienne]

Update: #4479 will fix this (hopefully) so this can be removed from our working list.

[jpelay]

Hello, @alaaeddin-swidan! We worked on this issue and eventually replaced Ace with CodeMirror. This change is live on hedy.org. Can you try it a bit and tell us if it solves this issue or if any other issue arises? Thanks!!

[vzsky]

Thanks a lot for the improvement. Although I am not sure if its work correctly in arabic, I have tested that it is now perfectly working in Thai, unlike earlier with aceajax.

[jpelay]

Thanks a lot for the improvement. Although I am not sure if its work correctly in arabic, I have tested that it is now perfectly working in Thai, unlike earlier with aceajax.

Great to hear! I do am fixing #4806 which does seem to be an annoying issue

[jpelay]

@Felienne can we close this one?

[Felienne]

Yes I think so! I have not heard back from https://github.com/alaaeddin-swidan but it seems to work for other languages with issues!