ibus predict window is shown at the previous cursor position #243

Closed
GoogleCodeExporter opened this Issue Apr 22, 2015 · 8 comments

Projects

None yet

2 participants

@GoogleCodeExporter
What steps will reproduce the problem?
1. Open a html file or access a page which has textarea in Firefox(a sample 
html file is attached).

2. Type the following keys to input "今日の天気は晴れ" in the textarea.
"kyouno<space>tenkiha<space>hare"

3. See the position of ibus predict window for "hare".

What is the expected output?
ibus predict windows is shown at the position of "hare".

What do you see instead?
ibus predict window is shown at the position of "tenkiha".
See attached screenshot.

What version of the product are you using? On what operating system?

Ubuntu 14.10 Utopic Unicorn (development branch)
ibus-mozc:
  Installed: 1.15.1857.102-1
firefox:
  Installed: 31.0~b6+build2-0ubuntu1

Please provide any additional information below.

Original issue reported on code.google.com by nob...@nobuto-murata.org on 7 Aug 2014 at 12:56

Attachments:

@GoogleCodeExporter
Thank you for the report.  I'm able to reproduce this issue with Ubuntu 14.04 
with Mozc-1.13.1651.102.  However, I no longer have enough spare time to work 
on Linux IBus client of Mozc.  Therefore this issue will not be fixed until 
someone volunteers to fix it.  Please feel free to contact us if you find the 
root cause or come up with some patches to address this issue.

Thank you for your understandings.

Original comment by yukawa@google.com on 7 Aug 2014 at 4:24

  • Changed state: VolunteersNeeded
  • Added labels: OpSys-Linux, Priority-Low
  • Removed labels: Priority-Medium
@GoogleCodeExporter
Note: this issue is also reproducible with GEdit on Ubuntu 14.04.

BTW, I just took a look at ibus-mozc code and had a feeling that we are not 
able to fix this issue in ibus-mozc in a reliable way.  This probably came from 
a limitation of GTK+ immodule.  If I understood correctly, there is no reliable 
way to retrieve the screen coordinates of composing characters in Linux 
desktop.  In the case of your report, GTK+ immodule doesn't provide the way to 
retrieve the screen coordinates of "はれ".

To work around against this unfortunate limitation, IBus-mozc has used 
gtk_im_context_set_cursor_location signal to estimate the location of the 
composing text.  Unfortunately, this isn't perfect.
https://developer.gnome.org/gtk3/stable/GtkIMContext.html#gtk-im-context-set-cur
sor-location

If you hope a reliable fix of this behavior, I'd suggest you to file a feature 
request for GTK+ immodule and IBus upstream to enable IMEs to retrieve the 
screen coordinates of the composing characters.  AFAIK such limitation doesn't 
exist in Windows, OSX, and Chromium OS.  For example, following Windows APIs 
are powerful enough for Mozc to show suggestion windows at the expected 
location on Windows OS.  We absolutely need similar APIs in Linux desktop.
http://msdn.microsoft.com/ja-jp/library/windows/desktop/dd318634.aspx
http://msdn.microsoft.com/en-us/library/windows/desktop/ms538781.aspx

Original comment by yukawa@google.com on 7 Aug 2014 at 6:08

@GoogleCodeExporter
Revised the summary because this issue is not specific to Firefox

Original comment by yukawa@google.com on 7 Aug 2014 at 6:11

  • Changed title: ibus predict window is shown at the previous cursor position
@GoogleCodeExporter
Correction for my comment #2.

wrong:   http://msdn.microsoft.com/en-us/library/windows/desktop/ms538781.aspx
correct: http://msdn.microsoft.com/en-us/library/windows/desktop/ms538782.aspx

Original comment by yukawa@google.com on 7 Aug 2014 at 6:14

@GoogleCodeExporter
Ah right, with the keystroke "kyouno<space>tenkiha<space>hare", this issue is 
reproducible both on firefox and gedit.

However with the keystroke "kyouno<space><enter>tenkiha<space><enter>hare", 
it's reproducible only on firefox. The behavior in gedit is slightly better.

Original comment by nob...@nobuto-murata.org on 8 Aug 2014 at 5:15

@GoogleCodeExporter
> However with the keystroke "kyouno<space><enter>tenkiha<space><enter>hare", 
it's reproducible only on firefox. The behavior in gedit is slightly better.

It isn't surprising at all.  I guess ibus-mozc is calling IBus APIs (which 
eventually mapped into GTK+ immodule APIs) in the same way on both 
applications, but it does not mean that each application calls back 
gtk_im_context_set_cursor_location in the same way.  For instance, if the 
target application never calls back gtk_im_context_set_cursor_location, 
ibus-mozc never receives the screen coordinates of the cursor.  In this 
scenario, of course, it is impossible for ibus-mozc to show 
suggestion/candidate window at the appropriate position. 

You can find some related comments in the IBus itself.
https://github.com/ibus/ibus/blob/130d3108902ed3e7c61930e3a24ee957c3d3e658/ui/gt
k3/propertypanel.vala#L108
        /* FIXME: set_cursor_location() has a different behavior
         * in embedded preedit by applications.
         * GtkTextView applications, e.g. gedit, always call
         * set_cursor_location() with and without preedit
         * but VTE applications, e.g. gnome-terminal, and xterm
         * do not call set_cursor_location() with preedit.
         * firefox and thunderbird do not call set_cursor_location()
         * without preedit.
         * This may treat GtkIMContext and XIM with different ways.
         * Maybe get_preedit_string() class method.
         */

Let me summarize the situation for IME developers on Linux desktop.
1. Only cursor position can be retrieved via IME APIs.  There is no common way 
to retrieve the screen coordinates of an arbitrary character.
2. The application may or may not provide the cursor position. It even depends 
on the timing and application-internal state.

Perhaps we could improve the behavior of ibus-mozc when it is used with Firefox 
31, but I'm not sure if we can call it as a proper "Fix".

Original comment by yukawa@google.com on 8 Aug 2014 at 2:09

@GoogleCodeExporter

Original comment by yukawa@google.com on 21 Apr 2015 at 6:16

  • Now blocked on: #287
@yukawa yukawa added a commit that referenced this issue Aug 15, 2015
@yukawa yukawa Stop working around a limitation in GTK IM-module.
There is a long standing limitation in GTK IM-module that IMEs cannot retrieve
the screen coordinates of each composing character, which is definitely needed
to align suggestion window to the left edge of composing text.

In ibus-mozc, we have worked around this limitation by recording the cursor
rectangle in the mozc server rather and simulating the character screen
coordinates from those cursor rectangles since OSS Mozc 1.3.911.102
(a1fae21).

This emulation has, however, never been perfect.  Following issues are
actually edge cases of the emulation.
- #243: ibus predict window is shown at the previous cursor position
- https://bugzilla.mozilla.org/show_bug.cgi?id=1120851

Therefore we decided to remove the above emulation from ibus-mozc and live
in more robust but unsophisticated world instead.  With this CL, the
suggestion window will show up just under the cursor location rather than
being aligned with composing text.

This clean-up also enables us to refactor mozc-server without bothering future
ibus-mozc maintainers because that emulation code that is implemented in
mozc-server.  In subsequent CLs we can remove the emulation code without
breaking existing ibus-mozc client.

Closes #243.

BUG=#243
TEST=manually done on Ubuntu 14.04.
9fbcdd5
@yukawa yukawa added a commit that closed this issue Aug 15, 2015
@yukawa yukawa Stop working around a limitation in GTK IM-module.
There is a long standing limitation in GTK IM-module that IMEs cannot retrieve
the screen coordinates of each composing character, which is definitely needed
to align suggestion window to the left edge of composing text.

In ibus-mozc, we have worked around this limitation by recording the cursor
rectangle in the mozc server rather and simulating the character screen
coordinates from those cursor rectangles since OSS Mozc 1.3.911.102
(a1fae21).

This emulation has, however, never been perfect.  Following issues are
actually edge cases of the emulation.
- #243: ibus predict window is shown at the previous cursor position
- https://bugzilla.mozilla.org/show_bug.cgi?id=1120851

Therefore we decided to remove the above emulation from ibus-mozc and live
in more robust but unsophisticated world instead.  With this CL, the
suggestion window will show up just under the cursor location rather than
being aligned with composing text.

This clean-up also enables us to refactor mozc-server without bothering future
ibus-mozc maintainers because that emulation code that is implemented in
mozc-server.  In subsequent CLs we can remove the emulation code without
breaking existing ibus-mozc client.

Closes #243.

BUG=#243
TEST=manually done on Ubuntu 14.04.
9fbcdd5
@yukawa yukawa closed this in 9fbcdd5 Aug 15, 2015
@yukawa yukawa self-assigned this Aug 15, 2015
@yukawa
Member
yukawa commented Sep 1, 2015

I confirmed that 9fbcdd5 does fix the case of gedit but the original issue in Firefox persists as of Firefox Nightly 2015-08-31. Probably there were two different issues and the firefox-specific one cannot be fixed in ibus-mozc side. I've just reported as Mozilla Bug 1200506 for firefox-specific one.

@yukawa yukawa added a commit that referenced this issue Dec 19, 2015
@yukawa yukawa Remove dead code from mozc_server.
This is a follow up for 9fbcdd5,
with which we stopped working around a limitation in GTK IM-module
and removed the ibus-mozc side logic.

This CL removes the backend logic implemented in mozc_server.  This
should not have any user-visible behavior change because that logic
in question is no longer used.

BUG=#243
TEST=unittest
REF_BUG=19395322
REF_CL=91766128
5d5cd4e
@yukawa yukawa added a commit that referenced this issue Dec 19, 2015
@yukawa yukawa Remove dead code from mozc_server.
This is a follow up for 9fbcdd5,
with which we stopped working around a limitation in GTK IM-module
and removed the ibus-mozc side logic.

This CL removes the backend logic implemented in mozc_server.  This
should not have any user-visible behavior change because that logic
in question is no longer used.

BUG=#243
TEST=unittest
REF_BUG=19395322
REF_CL=91766128
ea2b0ab
@yukawa yukawa added a commit that referenced this issue Dec 19, 2015
@yukawa yukawa Remove dead code from mozc_server.
This is a follow up for 9fbcdd5,
with which we stopped working around a limitation in GTK IM-module
and removed the ibus-mozc side logic.

This CL removes the backend logic implemented in mozc_server.  This
should not have any user-visible behavior change because that logic
in question is no longer used.

BUG=#243
TEST=unittest
REF_BUG=19395322
REF_CL=91766128
778eeba
@yukawa yukawa added a commit that referenced this issue Dec 20, 2015
@yukawa yukawa Remove dead code from mozc_server.
This is a follow up for 9fbcdd5,
with which we stopped working around a limitation in GTK IM-module
and removed the ibus-mozc side logic.

This CL removes the backend logic implemented in mozc_server.  This
should not have any user-visible behavior change because that logic
in question is no longer used.

BUG=#243
TEST=unittest
REF_BUG=19395322
REF_CL=91766128
2e83a44
@yukawa yukawa added a commit that referenced this issue Dec 20, 2015
@yukawa yukawa Remove dead code from mozc_server.
This is a follow up for 9fbcdd5,
with which we stopped working around a limitation in GTK IM-module
and removed the ibus-mozc side logic.

This CL removes the backend logic implemented in mozc_server.  This
should not have any user-visible behavior change because that logic
in question is no longer used.

BUG=#243
TEST=unittest
REF_BUG=19395322
REF_CL=91766128
7a129e6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment