Input method support on Linux

[Edgaru089]

Subject of the issue

Input methods (I'm using Fcitx) does not work with SFML on Linux.

I looked into the source code and found that SFML is simply generating TextEntered events on key presses.

A input method is a piece of software used to input characters not present on the keyboard, mostly for ideographic scripts which is not even composed of a alphabet, and must be entered by imaginative ways, like most significantly the famous CJK Ideographs.

Your environment

Arch Linux, KDE Plasma, SFML 2.5.1

Fcitx4 with aur/fcitx-sogoupinyin

Steps to reproduce

Sadly, for those who don't speak Chinese or Japanese, using a input method (and the fact that we even need it) is not very meaningful and thus difficult to reproduce. Even if you do know a little about one of those it's quite complicated to set up a working input method on Linux for those who don't use it on a daily basis.

If you have a working IM however, simply create a window and dump the TextEntered and KeyPressed events.

Behaviour

You will find they closely resemble each other and the window does not respond to the installed input method at all.

[Edgaru089]

The sad thing about input methods on Linux is that there are so many frameworks, all of which are incompatible which each other. Take a look at the chart on the Arch Wiki page if you're interested. Each column is a input method framework patching the already cacophonous X input stack, and each row is a actual input method doing the real work, translating your keystrokes to characters.

X has a protocol called XIM (X Input Method), but it predates Unicode and uses 16-bit code pages, and is only there for legacy compatibility. I can hardly find any documentation on it, other than the offical X Specification, which is notoriously hard to understand.

[Edgaru089]

Just went digging the SDL source, supporting Fcitx and ibus.

Turns out they both require (or at least SDL is using) a dbus connection to the local daemon! Simple grepping shows dbus is only used for RealtimeKit1 (thread priority), UPower (battery status), and Fcitx/ibus.

Using dbus might be too much a overhead for fixing a bug (or maybe, implementing a feature) not meaningful to many. Or at least, copy-pasting works with Unicode, and input method support on Linux is so scarce that people are getting used to it, with even utilities for entering text on an overlay, and copy-pasting them via clipboard with a hotkey.

[Edgaru089]

Should we use DBus, or just leave it be? Or should we investigate using XIM?

[eXpl0it3r]

Input method is certainly not a trivial feature to implement, but maybe I'm wrong and it's actually simpler than expect.
Either way, choosing an implementation option without having done some deeper analysis on how this is setup and how it works and then also keep in mind a cross-platform API design (if an additional API is required).

On Windows I've seen quite some hacks and undocumented API calls that SDL uses to get IME working...

[Edgaru089]

For API changes, I don't think the current API should be changed, or at most something like window.hintTextCursorPosition() can be added for giving a hint for the IME to place its overlay near the text cursor.

Other features are too platform, font/coloring or IME dependent (given that SFML can't even access system fonts), requires much more integration from the application (very large API change), and too difficult to fit all the habits of different, and arguably few, users noticing the change.

The Windows implementation of window.hintTextCursorPosition() can be as simple as

if (HIMC himc = ImmGetContext(windowHandle)) {
    COMPOSITIONFORM cf;
    cf.ptCurrentPos.x = CursorX; // The upper-left corner of the IME overlay window
    cf.ptCurrentPos.y = CursorY; // Relative to the top-left corner of the window
    cf.dwStyle = CFS_FORCE_POSITION; // Don't let the IME adjust the pos
    ImmSetCompositionWindow(himc, &cf);
    ImmReleaseContext(windowHandle, himc);
}

---

Most (Simplified) Chinese input methods translate one keystroke sequence into multiple candidates to select with a number key (or the convenient space bar for the 1st one). Before the raw input (called pre-edit data) is long enough to form the desired character or word (a sequence of characters with a given meaning), it is stored temporarily by the IME and displayed, with the candidates, in a overlay.

In the example you can see the text cursor, and the white overlay positioned under it, with pre-edit (the 1st line) and candidates (the second) inside. I only typed the Latin letters and the IME kindly sliced the sequence up representing the 4 characters by a given rule.

---

And, instead of the IME displaying the pre-edit in the overlay, the application also displays it or even in more imaginative ways, as in this example.

This is good-old Windows Notepad.exe with the Chinese IME built into Windows 10 (the one you get simply by installing Simplified Chinese language pack). Notepad.exe told the IME its cursor position, and even the font and colors it's using, and the IME simply rendered pre-edit over existing text, displaying only candidates in the overlay.

---

Most Traditional Chinese and Japanese IMEs behave completely differently, but I'm not familiar with these.

[Edgaru089]

For IME on Linux, the question is: is it worth adding a dependency of libdbus to simply implement this little feature?

[Edgaru089]

Fixed by #1850.

[Neko-Box-Coder]

Hi,

I stumbled upon this post when trying to see if SFML supports IME.
Are there any plans on implementing "hinting the IME for the caret/text cursor position"?
@eXpl0it3r @Edgaru089
Thanks.

For API changes, I don't think the current API should be changed, or at most something like window.hintTextCursorPosition() can be added for giving a hint for the IME to place its overlay near the text cursor.

Other features are too platform, font/coloring or IME dependent (given that SFML can't even access system fonts), requires much more integration from the application (very large API change), and too difficult to fit all the habits of different, and arguably few, users noticing the change.

The Windows implementation of window.hintTextCursorPosition() can be as simple as

if (HIMC himc = ImmGetContext(windowHandle)) {
    COMPOSITIONFORM cf;
    cf.ptCurrentPos.x = CursorX; // The upper-left corner of the IME overlay window
    cf.ptCurrentPos.y = CursorY; // Relative to the top-left corner of the window
    cf.dwStyle = CFS_FORCE_POSITION; // Don't let the IME adjust the pos
    ImmSetCompositionWindow(himc, &cf);
    ImmReleaseContext(windowHandle, himc);
}

Most (Simplified) Chinese input methods translate one keystroke sequence into multiple candidates to select with a number key (or the convenient space bar for the 1st one). Before the raw input (called pre-edit data) is long enough to form the desired character or word (a sequence of characters with a given meaning), it is stored temporarily by the IME and displayed, with the candidates, in a overlay.

In the example you can see the text cursor, and the white overlay positioned under it, with pre-edit (the 1st line) and candidates (the second) inside. I only typed the Latin letters and the IME kindly sliced the sequence up representing the 4 characters by a given rule.

And, instead of the IME displaying the pre-edit in the overlay, the application also displays it or even in more imaginative ways, as in this example.

This is good-old Windows Notepad.exe with the Chinese IME built into Windows 10 (the one you get simply by installing Simplified Chinese language pack). Notepad.exe told the IME its cursor position, and even the font and colors it's using, and the IME simply rendered pre-edit over existing text, displaying only candidates in the overlay.

Most Traditional Chinese and Japanese IMEs behave completely differently, but I'm not familiar with these.