Keyboard for Arabic and languages with Arabic script, initial release #5569

WaseemAlkurdi · 2019-11-06T11:59:27Z

This is an Arabic character keyboard layout. It is based in part on the PC keyboard layout, and in part on the Google keyboard.
I have had to add some keys to the first and second rows.
I haven't tested this, because attempting to compile the build from source results in an error:

make[1]: *** No rule to make target 'luafilesystem/src/lfs.c', needed by 'build/x86_64-linux-gnu-debug/libs/libkoreader-lfs.so'.  Stop.

Diacritic marks haven't been added, but they could be added in a couple of minutes, should this be tested and proven to work.

This change is

Frenzie · 2019-11-06T12:48:36Z

On which system is that? You can use the extracted AppImage (for just testing Lua stuff) or the Docker image to have a tailored build environment, see https://github.com/koreader/koreader/blob/master/doc/Building.md#setting-up-a-build-environment-for-koreader

Also on systems other than Android it's generally simple to just change a few Lua files.

WaseemAlkurdi · 2019-11-06T12:50:07Z

On which system is that? You can use the extracted AppImage (for just testing Lua stuff) or the Docker image to have a tailored build environment, see https://github.com/koreader/koreader/blob/master/doc/Building.md#setting-up-a-build-environment-for-koreader

Also on systems other than Android it's generally simple to just change a few Lua files.

I'm building on desktop Linux.
So is it okay to just copy the source tree over to the /mnt/us/koreader directory on my Kindle?
And do I have to edit options to enable the Arabic keyboard?

Frenzie · 2019-11-06T12:51:30Z

You still need to add it to this table:

koreader/frontend/ui/widget/virtualkeyboard.lua

Lines 526 to 535 in f1f75c5

    
           lang_to_keyboard_layout = { 
        
               el = "el_keyboard", 
        
               en = "en_keyboard", 
        
               es = "es_keyboard", 
        
               fr = "fr_keyboard", 
        
               he = "he_keyboard", 
        
               ja = "ja_keyboard", 
        
               pt_BR = "pt_keyboard", 
        
               ko_KR = "ko_KR_keyboard", 
        
           },

Frenzie · 2019-11-06T12:56:00Z

So is it okay to just copy the source tree over to the /mnt/us/koreader directory on my Kindle?

Sort of. As long as you have the libraries in place that should be fine, but it's normally faster and easier to just overwrite the couple of Lua files you edited.

WaseemAlkurdi · 2019-11-06T13:08:58Z

@Frenzie I did that ... KOReader now crashes. Why is that?
The Lua scripts are error-free, except for warnings about unused variables.

Frenzie · 2019-11-06T13:09:10Z

frontend/ui/data/keyboardlayouts/ar_keyboard.lua

@@ -60,7 +60,7 @@ local alef = ar_popup.alef
 local baa = ar_popup.baa
 local jeem = ar_popup.jeem
 local daal = ar_popup.daal
-local h-aa = ar_popup.h-aa
+local haa2 = ar_popup.haa2


Incidentally, underscore (_) is fine in a variable name.

@Frenzie I know, but I removed them in order to isolate an issue. I'll return them in the next round of cleanups. But for now, the keyboard as is would crash KOReader for some reason.

Frenzie · 2019-11-06T13:13:56Z

I did that ... KOReader now crashes. Why is that?

I have no idea, because all you've got here is two completely independent files that literally can't affect anything at all. ^_^ The crash.log will tell you more if you want to find out, or you can just overwrite the whole thing with the latest nightly from http://build.koreader.rocks/download/nightly/v2019.10-33-gf1f75c5_2019-11-06/ so you've got a working copy.

Frenzie · 2019-11-06T13:17:56Z

Anyway, I can push a few fix-up commits to your branch if you like.

WaseemAlkurdi · 2019-11-06T13:18:44Z

Anyway, I can push a few fix-up commits to your branch if you like.

It would be great if you do!

Frenzie · 2019-11-06T13:29:01Z

@WaseemAlkurdi See the two individual commits for the (minor) changes.

Frenzie · 2019-11-06T13:30:29Z

frontend/ui/widget/virtualkeyboard.lua

@@ -377,6 +377,7 @@ function VirtualKeyPopup:init()
                virtual_key.onHoldReleaseKey = function()
                    virtual_key:onTapSelect(true)
                    UIManager:close(self)
+                    return true


Whoops, I didn't mean to commit this line but that's okay. This solves the worst part of #5555.

poire-z · 2019-11-06T13:31:28Z

KOReader now crashes. Why is that? The Lua scripts are error-free,

See your crash.log, you should have a stack trace that will tell you (or us) why it crashes, and what must have happened for it to crash even if your lua files are error-free (truncated transfer, other file touched...)

About AppImage: https://github.com/koreader/koreader/wiki/Installation-on-desktop-linux
(@Frenzie : Looks like this #4304 got lost in the Wiki reorganisation.)

Also, we'll have to think about these HE/AR keyboards: is it fine that you start with querty, and you have to it the little Globa icon key to have the HE/AR keys? Or should we start with these, and get qwerty with that Globe icon key instead. See #5414 (comment) and around.

Frenzie · 2019-11-06T13:32:59Z

@poire-z Not lost, that's at the top of https://github.com/koreader/koreader/blob/master/doc/Building.md#setting-up-a-build-environment-for-koreader

Frenzie · 2019-11-06T13:35:03Z

@WaseemAlkurdi Now you'll need to git pull my commits into your branch before you continue working.

poire-z · 2019-11-06T13:42:10Z

frontend/ui/data/keyboardlayouts/ar_AA_keyboard.lua

+        -- fourth row
+        {
+            { "Sym",     "Sym",  "ABC",  "ABC",  "Sym",  "رمز",    "رمز",  "رمز",  "Sym",  "Sym",  "ABC",  "ABC",
+              width = 1.5},


You could try reducing this width to = 1 to not have these keys overflow the keyboard :)

NiLuJe · 2019-11-06T15:04:08Z

The original build error vaguely looks like you haven't pulled the submodules? One kodev step should take care of that, see the docs ;). Otherwise, I'm going to need more context.

Also, I've posted the one-liner I use to scp updated files to my devices a couple of times around. I'm unfortunately not on my desktop right now, but it's basically based on the list that git diff --name-only outputs ;).

Frenzie · 2019-11-06T15:04:56Z

A ./kodev run should do that automatically though.

See #5573

poire-z · 2019-11-06T21:53:25Z

frontend/ui/data/keyboardlayouts/ar_AA_keyboard.lua

+        },
+        -- third row
+        {  --  1           2       3       4       5       6          7       8       9       10      11      12
+            { label = "بَدِّل",


Looks like you need to leave it as "Shift" for it to work.

poire-z · 2019-11-06T21:54:34Z

frontend/ui/data/keyboardlayouts/ar_AA_keyboard.lua

+            { ",",        ",",    "’",    "↓",    "ֽ ",    waw,   "Ю",    "ю",    "Ç",    "ç",    "…",    "¨", },
+            { ".",        ".",    "’",    "↓",    "ֽ ",    zay,   "Ю",    "ю",    "Ç",    "ç",    "…",    "¨", },
+            { "/",        "/",    "’",    "↓",    "ֽ ",    thaa,   "Ю",    "ю",    "Ç",    "ç",    "…",    "¨", },
+            { label = "مَسح",


Same here: must stay Backspace (that's not what displayed, the icon= takes precedence, but I guess the label (albeit displayed if no icon) serves as recognizing it should do shift or backspace.

poire-z · 2019-11-06T22:01:09Z

frontend/ui/data/keyboardlayouts/ar_AA_keyboard.lua

+        },
+        -- fourth row
+        {
+            { "Sym",     "Sym",  "ABC",  "ABC",  "Sym",  "رمز",    "رمز",  "رمز",  "Sym",  "Sym",  "ABC",  "ABC",


Looks like these رمز", "رمز", "رمز" (raaah, so hard to cut and paste and wrap, bidi hell!) need to be added to:
symbolmode_keys = {["Sym"] = true, ["ABC"] = true, ["رمز"] = true}, on line 91, for them to work.

(That's going to be my new swear word : that bidi-mary is bidi disgusting on this bidi sunday :)

poire-z · 2020-01-17T16:04:14Z

frontend/ui/data/keyboardlayouts/ar_AA_keyboard.lua

+            { diacritic_tanween_damm,     qaf,     "#",    "3", },
+            { "ﻹ",     fah,     "+",    _eq, },


Might keep them all aligned the same way with the 1 2 3 4, for readability :)

@poire-z Can I do that from the web interface, or do I have to push again?

@poire-z Turns out that I can, but it would be considered another commit. Not a problem!

poire-z · 2020-01-17T16:05:06Z

frontend/ui/data/keyboardlayouts/ar_AA_keyboard.lua

+            { "ﻹ",     fah,     "+",    _eq, },
+            { "إ",   ghayn,    "€",    "(", },
+            { "`",                       ayin,    "‰",    ")", },
+-- h_aa


You can indent that comment (including the --) up to where the {"ه", label="ه‍"} (I assume it says what's that thing is :)

@poire-z Fixing that!

poire-z · 2020-01-17T16:08:48Z

frontend/ui/data/keyboardlayouts/keypopup/ar_AA_popup.lua

+        "ز",
+        north = "ژ",
+    },
+    ha = { -- الحاء


You could translate that arabic word to english -- الحاء (english translation) if that means something other than "ha" :) so we don't have to wonder if we're missing something, and to improve the culture of non arabic readers :)
(I wonder the same above for h_aa, but it's shorter, so it would just be h_aa in arabic :)

@poire-z I didn't know how to "transcribe" the two letters, so I used 'h_aa' for هـ and 'ha' for ح. Honestly, this could be improved, and I myself mix up the two "transciptions".
Please do listen to the first 5 seconds of this YouTube video: https://www.youtube.com/watch?v=YeyVjaWD6-U. The first letter that the announcer pronounces is 'هـ' (my 'h_aa') and the second letter is 'ح' (my ha).
Edit: Had them in reverse.

The word 'الحاء' is simply the name of the letter, roughly meaning "the letter ha". like how "double u" is sort of a name of the letter W in English.

poire-z · 2020-01-17T16:10:20Z

frontend/ui/data/keyboardlayouts/keypopup/ar_AA_popup.lua

+-- الحركات
+   diacritics = {


I guess that arabic word means "diacritics" :)

@poire-z Precisely.

WaseemAlkurdi · 2020-01-18T09:46:56Z

@WaseemAlkurdi Can these just be removed?

    frontend/ui/data/keyboardlayouts/ar_AA_keyboard.lua:11:7: unused variable 'h_aa'
    frontend/ui/data/keyboardlayouts/ar_AA_keyboard.lua:38:7: unused variable 'taamarbouta'

There's a slight problem here. I had to hardcode these two letters (instead of defining them as variables) because I had to use label=, as you can see in ar_AA_keyboard.lua).
Do you suggest removing these variables, or is there a way in which I could use label= with ar_popup.lua?
Never mind. I tried using label= directly in ar_AA_popup.lua directly, and it worked. I mean, it should, since what we were doing is defining a Lua table inline, so why should it not work if importing that table as a variable from another file?

Fixed the two missing variables by moving the `label=` part from the main keyboard to this file.

- Removed the label. - Fixed the hardcoded labels (and the two missing variables).

Translated label.

Fixed the missing variables ... for good?

poire-z · 2020-01-18T10:18:20Z

Regarding the trick with the zero-width-joiner/non-joiner stuff mentionned in #5569 (comment) :
The sad thing is that we don't see it in the code :)
I had the same issue some weeks ago with zero width space, and I used \x utf8 bytes instead. You could do that, with a comment explaining why.
Or just add a comment mentionning there is an invisible ZWNJ or ZWJ.
Or, I think arabic has another range in unicode for "arabic presentational forms", which I think are the pre-shaped glyphs, so may be the one you want might be there:
https://en.wikipedia.org/wiki/Arabic_Presentation_Forms-A
https://en.wikipedia.org/wiki/Arabic_Presentation_Forms-B

WaseemAlkurdi · 2020-01-18T10:20:22Z

@poire-z Sheesh, I've mistyped it while moving. It was supposed to have a ZWJ, but the ZWJ isn't directly accessible on the keyboard, unlike a non-zero width joiner (an easy Shift-J away). Will fix this instant.

Replaced the non-zero width joiner with a zero width joiner.

WaseemAlkurdi · 2020-01-18T10:24:45Z

The sad thing is that we don't see it in the code :)

GitHub's web UI, at least here on Firefox on Linux, uses a red dot to indicate it instead of the shaped character.

WaseemAlkurdi · 2020-01-18T10:30:15Z

Or, I think arabic has another range in unicode for "arabic presentational forms", which I think are the pre-shaped glyphs, so may be the one you want might be there.

No, these (from what I see here: https://en.wikipedia.org/wiki/Arabic_Presentation_Forms-A) are the Arabic characters used by non-Arabic languages with Arabic scripts, which are not "legal Arabic". Something like how some French and German characters, while using Latin script, are not "legal Latin".
But the B block (https://en.wikipedia.org/wiki/Arabic_Presentation_Forms-B) is in fact legal Arabic, and is probably the block you're referring to. These are the shaped ligatures ... I could see where you're going with this! Amazing.

@poire-z

Replaced the two letters that have a non-zero width joiner with their preformed ligature forms as per @poire-z's suggestion. Really neat.

WaseemAlkurdi · 2020-01-18T10:44:05Z

Replaced as per your suggestion. It's a much neater solution, having the whole thing as one character, instead of bothering with invisible characters.
Looks like this is done then! The build tests check out.

A small note though. The Wikipedia article (https://en.wikipedia.org/wiki/Arabic_Presentation_Forms-B) says:

The presentation forms are present only for compatibility with older standards, and are not currently needed for coding text.

Does this mean that these might be deprecated? If so, wouldn't this be a "hack" that might stop working with unpredictable consequences?

poire-z · 2020-01-18T10:44:47Z

And in B ? :) Isn't it this one:

Unicodecode point	character	UTF-8encoding(hex)	Unicode character name	Unicode 1.0 character name (deprecated)
U+FEE9	ﻩ	ef bb a9	ARABIC LETTER HEH ISOLATED FORM	GLYPH FOR ISOLATE ARABIC HA
U+FEEA	ﻪ	ef bb aa	ARABIC LETTER HEH FINAL FORM	GLYPH FOR FINAL ARABIC HA
U+FEEB	ﻫ	ef bb ab	ARABIC LETTER HEH INITIAL FORM	GLYPH FOR INITIAL ARABIC HA
U+FEEC	ﻬ	ef bb ac	ARABIC LETTER HEH MEDIAL FORM	GLYPH FOR MEDIAL ARABIC HA

from https://unicode-search.net/unicode-namesearch.pl?term=HEH

poire-z · 2020-01-18T10:46:31Z

Oh, allright, you found it :)
I don't think Unicode can deprecate or remove codepoints :) by contract, I guess, they can just add stuff.

Is it ok that it does not look the same in GH UI ?

poire-z · 2020-01-18T11:13:05Z

Looks like this is done then! The build tests check out.

Stil the alignment with 1 2 3 4 to fix with GH UI :)

- Fixed keyboard row spacing ... should be more organized now. - Added a couple of labels to explain some Arabic strings and transliteration choices.

WaseemAlkurdi · 2020-01-18T17:54:09Z

Looks like this is done then! The build tests check out.

Stil the alignment with 1 2 3 4 to fix with GH UI :)

Done! Should look really tidy now.

Fixed the missing space

WaseemAlkurdi · 2020-01-18T17:59:56Z

I think you accidentally lost a space here.

What's a space between friends? :-D

poire-z

100% conform to my knowledge of Arabic ! (which would give 0, but hey, approved anyway :)

poire-z · 2020-01-18T18:06:20Z

frontend/ui/data/keyboardlayouts/keypopup/ar_AA_popup.lua

+--  Diacritics (al-Harakat الحركات)
+   diacritics = {


Last nitpick: have the comment indented as what follows it.

@poire-z Done.

Indented a comment.

NiLuJe · 2020-01-18T20:58:13Z

I'd possibly have gone with IPA for those latest (interesting!) comments, but that's just me ;).

Frenzie · 2020-01-18T21:35:42Z

No, I definitely would've gone with IPA. But I didn't know IPA prior to the university, so I wouldn't really expect anybody else to unless they had a degree in something language-related.

)

Added Arabic keyboard, initial release

ed2c014

WaseemAlkurdi changed the title ~~Added Arabic keyboard, initial release~~ Keyboard for Arabic and languages with Arabic script, initial release Nov 6, 2019

Frenzie added this to the 2019.11 milestone Nov 6, 2019

Frenzie added the UX label Nov 6, 2019

WaseemAlkurdi added 2 commits November 6, 2019 14:47

Fixed tab characters

464409e

Fixed tab characters (2)

63be139

Fixed tab characters (3)

b3e4cb9

Error fixes

bdaab10

Frenzie reviewed Nov 6, 2019

View reviewed changes

Frenzie added 2 commits November 6, 2019 14:24

Rename and add to list

5c3fdc9

Add missing comma

40f3f79

Frenzie reviewed Nov 6, 2019

View reviewed changes

poire-z reviewed Nov 6, 2019

View reviewed changes

poire-z mentioned this pull request Nov 6, 2019

Keyboard: avoid additional key stroke on hold release #5573

Merged

remove return true

9350e15

See #5573

poire-z reviewed Nov 6, 2019

View reviewed changes

poire-z reviewed Jan 17, 2020

View reviewed changes

WaseemAlkurdi added 4 commits January 18, 2020 11:54

Update ar_AA_popup.lua

e9808ab

Fixed the two missing variables by moving the `label=` part from the main keyboard to this file.

Update ar_AA_keyboard.lua

3ff2a9c

- Removed the label. - Fixed the hardcoded labels (and the two missing variables).

Update ar_AA_popup.lua

c2e8c91

Translated label.

Update ar_AA_keyboard.lua

06d975f

Fixed the missing variables ... for good?

Update ar_AA_popup.lua

75498c1

Replaced the non-zero width joiner with a zero width joiner.

Update ar_AA_popup.lua

854819e

Replaced the two letters that have a non-zero width joiner with their preformed ligature forms as per @poire-z's suggestion. Really neat.

Update ar_AA_keyboard.lua

1e53c0a

- Fixed keyboard row spacing ... should be more organized now. - Added a couple of labels to explain some Arabic strings and transliteration choices.

Update ar_AA_keyboard.lua

88b6743

Fixed the missing space

poire-z approved these changes Jan 18, 2020

View reviewed changes

Update ar_AA_popup.lua

286ddcc

Indented a comment.

Frenzie added 2 commits January 18, 2020 22:38

indentation nits for merge

1255f47

minor premerge nit

5ff0621

Frenzie merged commit c8b3942 into koreader:master Jan 18, 2020

WaseemAlkurdi deleted the arabic_keyboard branch January 19, 2020 08:45

yparitcher mentioned this pull request Jan 24, 2020

fix arabic keyboard to conform to #5639 #5793

Merged

mwoz123 pushed a commit to mwoz123/koreader that referenced this pull request Mar 29, 2020

[UX] Keyboard for Arabic and languages with Arabic script (koreader#5569

07a3c5b

)

		{ diacritic_tanween_damm, qaf, "#", "3", },
		{ "ﻹ", fah, "+", _eq, },

Keyboard for Arabic and languages with Arabic script, initial release #5569

Keyboard for Arabic and languages with Arabic script, initial release #5569

Conversation

WaseemAlkurdi commented Nov 6, 2019 • edited by Frenzie Loading

Frenzie commented Nov 6, 2019

WaseemAlkurdi commented Nov 6, 2019

Frenzie commented Nov 6, 2019

Frenzie commented Nov 6, 2019

WaseemAlkurdi commented Nov 6, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Frenzie commented Nov 6, 2019

Frenzie commented Nov 6, 2019

WaseemAlkurdi commented Nov 6, 2019

Frenzie commented Nov 6, 2019

Choose a reason for hiding this comment

poire-z commented Nov 6, 2019

Frenzie commented Nov 6, 2019

Frenzie commented Nov 6, 2019

Choose a reason for hiding this comment

NiLuJe commented Nov 6, 2019 • edited Loading

Frenzie commented Nov 6, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WaseemAlkurdi Jan 18, 2020 • edited Loading

Choose a reason for hiding this comment

WaseemAlkurdi Jan 18, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WaseemAlkurdi commented Jan 18, 2020 • edited Loading

poire-z commented Jan 18, 2020 • edited Loading

WaseemAlkurdi commented Jan 18, 2020

WaseemAlkurdi commented Jan 18, 2020

WaseemAlkurdi commented Jan 18, 2020 • edited Loading

WaseemAlkurdi commented Jan 18, 2020

poire-z commented Jan 18, 2020

poire-z commented Jan 18, 2020 • edited Loading

poire-z commented Jan 18, 2020

WaseemAlkurdi commented Jan 18, 2020

WaseemAlkurdi commented Jan 18, 2020

poire-z left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NiLuJe commented Jan 18, 2020 • edited Loading

Frenzie commented Jan 18, 2020

WaseemAlkurdi commented Nov 6, 2019 •

edited by Frenzie

Loading

NiLuJe commented Nov 6, 2019 •

edited

Loading

WaseemAlkurdi Jan 18, 2020 •

edited

Loading

WaseemAlkurdi Jan 18, 2020 •

edited

Loading

WaseemAlkurdi commented Jan 18, 2020 •

edited

Loading

poire-z commented Jan 18, 2020 •

edited

Loading

WaseemAlkurdi commented Jan 18, 2020 •

edited

Loading

poire-z commented Jan 18, 2020 •

edited

Loading

NiLuJe commented Jan 18, 2020 •

edited

Loading