Add initial support for UTF-8 text input

[ihhub]

As of now only CP-1251 is being implemented (as @Branikolog was eating my brain too much). You can test changes by using your OS keyboard layout and try to type one of CP-1251 languages. For example, Player Name in the multiplayer mode.

Note: SDL interprets Ctrl + Key combination as not a text input and falls back to English keyboard.

relates to #7933, #6845 and #6207

[github-actions]

⚠️ Clang-Tidy found issue(s) with the introduced code (1/1)

[github-actions]

⚠️ Clang-Tidy found issue(s) with the introduced code (1/1)

[Branikolog]

Hi, @ihhub !
Russian and Belarussian works well.
But when I select English as game language and try to add some symbols, while my system input language is set as Russian or Belarussian - nothing happens. In master branch I can still type, no matter what language input was chosen. We should have an opened issue for MAC users only in issues section. With this PR the issue is valid for Win too now.

[oleg-derevenetz]

Hi @Branikolog

But when I select English as game language and try to add some symbols, while my system input language is set as Russian or Belarussian - nothing happens.

When you select English as game language, no Russian or Belarussian fonts exist, so when Russian or Belarussian symbols are coming from the keyboard, they are ignored.

In master branch I can still type

In master branch UTF-8 does not come from the keyboard.

[Branikolog]

Hello, @oleg-derevenetz .

In master branch UTF-8 does not come from the keyboard.

I'm just speaking as a common user. :)
For me personally current implementation for English is less convenient, than the one from master branch.

If the game language is set as English, it is implied, that no other languages could be used to type texts. So I expect only Latin symbols to appear. In that case I feel there's no need to select exactly English within OS to type, since no other symbols are allowed. We've already had some complains on MAC, as I mentioned above, that game forces user to switch OS input language to be able to add text.
To sum up, I suggest for "English" game language add always Latin symbols, no matter what text input was set within OS. If it's possible, for sure.
For such languages as Russian and Belarussian everything works fine for me in this PR.

[oleg-derevenetz]

Hi @ihhub there are my observations:

1. This doesn't work on Linux. No "key press" events are generated for non-English languages, thus, if ( le.KeyPress() ) { checks doesn't work;
2. On Android, when SDL_StartTextInput() is called, the Android native virtual keyboard appears on the screen. But you still cannot enter characters using this virtual keyboard (even English ones!) because "key press" events are not generated either.

[ihhub]

Hi @oleg-derevenetz , thanks for the notes! I will try to test on Android and on Linux. Most likely these OSes behave differently from Windows.

[oleg-derevenetz]

@Branikolog

To sum up, I suggest for "English" game language add always Latin symbols, no matter what text input was set within OS. If it's possible, for sure.

Please see my notes above. If no key information comes for non-English symbols (only their UTF-8 symbol codes) there is no way to match these UTF-8 symbols to corresponding English symbols on the same keyboard keys (i.e. й->q, ц->w) unless you have an information about keyboard layout, and the ways to obtain this information are highly system-dependent (and maybe even flavor-dependent in case of Linux, because some flavors use X Window and some use Wayland), therefore, it's hardly worth it.

[oleg-derevenetz]

Hi @ihhub

I will try to test on Android and on Linux. Most likely these OSes behave differently from Windows.

Also please note that, although I couldn't verify it, there will be issues with national symbols in file names on Linux. Although Linux filesystems (such as EXT3) are encoding-agnostic (i.e. they treat file names just as an array of bytes), particular apps (e.g. shell) will treat file names according to their locale settings (set via LC_* environment variables) and most likely they won't understand CP1251 file names, because most likely their locales are set to UTF-8.

[ihhub]

Hi @oleg-derevenetz , I believe we shouldn't allow to enter non-English names for files at the moment.

[Branikolog]

@oleg-derevenetz and @ihhub

@Branikolog

Please see my notes above. If no key information comes for non-English symbols (only their UTF-8 symbol codes) there is no way to match these UTF-8 symbols to corresponding English symbols on the same keyboard keys (i.e. й->q, ц->w) unless you have an information about keyboard layout, and the ways to obtain this information are highly system-dependent (and maybe even flavor-dependent in case of Linux, because some flavors use X Window and some use Wayland), therefore, it's hardly worth it.

I actually don't know how exactly things happen in the code regarding to symbol codes.
I just express my opinion as a potential user of fheroes2 engine.
If things I was talking above are not possible to implement by creating a proper logic or using a combination of new code for non-English languages selected and the old code related to text input for English game language only, then I can approve this PR, as It brings more benefits, than losses. 💯
But I believe, there could be complains from Windows users on forcing them to switch OS language input, same as I've already had from MAC users.

[oleg-derevenetz]

But I believe, there could be complains from Windows users on forcing them to switch OS language input, same as I've already had from MAC users.

Complaints like "I just told my OS that I want to enter non-English letters, but your program should magically turn these non-English letters into English letters" seem absurd to me. It's not worth the effort. If someone wants this behavior, they can implement it themselves and offer a pull request. Good luck to them with the support of the entire zoo of devices and OS for this.

[Districh-ru]

Hi, @ihhub. The CP1251 letters works good for me (Win10), but I cannot input some symbols:

[ihhub]

Hi @Branikolog , what was before doesn't mean it was correct. We will see what we could do about this.

@Districh-ru , most likely it is because out CP-1251 font does not contain all symbols.

[sonarcloud]

Quality Gate failed

Failed conditions

1 New Major Issues (required ≤ 0)
1 New Critical Issues (required ≤ 0)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint