Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure IME works on Android devices #2580

Merged
merged 2 commits into from
Jul 6, 2023
Merged

Ensure IME works on Android devices #2580

merged 2 commits into from
Jul 6, 2023

Conversation

RobinMalfait
Copy link
Collaborator

@RobinMalfait RobinMalfait commented Jul 6, 2023

We had an issue #2409 where typing using an IME (Input Method Editor, E.g.: Japanese — Romaji) already submitted characters while the user was still composing the characters together.

  1. Type wa
  2. Expected result:
  3. Actual result: wあ (where is the character a)

This was solved by not triggering change events at all until the compositionend event was triggered. This worked fine for this use case. However this also meant that only at the end of your typing session (when you press enter/space) the actual value was submitted.

Fast forward to today, we received a new issue #2575 where this behaviour completely broke on Android devices. Android always use the IME APIs for handling input... if we think about our solution form above, it also means that while you are typing on an Android device no options are being filtered at all. The moment you hit enter/space the combobox will open and results will be filtered.

This is where this fix comes in. The goals are simple:

  1. Make it work
  2. Try to make the current code simpler

I started digging to see why this wあ was even submitted. A normal input field doesn't do that?! We have some code that does the following things:

  1. Sync the selected value with the input such that if you update the value from the outside, then the value in the input is up-to-date with the displayValue of that incoming value.

  2. A fix for macOS VoiceOver to improve the VoiceOver experience when opening the Combobox component. This is done by manually resetting the value of the input field.

    1. Keep track of the current value
    2. Keep track of the current selection range (start/end) state
    3. Reset the input to an empty string ''
    4. Restore the value to the captured value
    5. Restore the selection range

When you are typing, the input field doesn't have to update yet because typing doesn't cause an option to become the selected option, therefore it doesn't have to sync the value yet. So (1.) isn't the issue here.

However, when you start typing, the Combobox should open and then we trigger the macOS VoiceOver fix. This is touching the input field because we change the value & selection.

Because we touched the input while the user was still in a composing mode, it bailed and submitted whatever characters it had. This is the part that we don't want. Not applying the macOS VoiceOver fix while typing solves this issue. In addition, because we are touching the input field, VoiceOver is acting normally.

In hindsight, the solution is very simple: do not touch the input field when the user is typing.

We still keep track whether the user isComposing so that we can bail on the default Enter behaviour (marking the current option as the selected option) because pressing Enter while composing should get out of the IME.

Fixes: #2575
Fixes: #2409 (this should still work after this change)

Before:

I start typing "Tom" but it is only the moment I press space that "Tom Cook" is filtered.

Screen.Recording.2023-07-06.at.12.59.49.mov

After:

Now, when I start typing "Tom", the options are already being filtered.

Screen.Recording.2023-07-06.at.13.01.51.mov

We had an issue #2409 where typing using an IME (Input Method Editor,
E.g.: Japanese — Romaji) already submitted characters while the user was
still composing the characters together.

1. Type `wa`
2. Expected result: `わ`
3. Actual result: `wあ` (where `あ` is the character `a`)

This was solved by not triggering change events at all until the
`compositionend` event was triggered. This worked fine for this use
case. However this also meant that only at the end of your typing
session (when you press `enter`/`space`) the actual value was submitted.

Fast forward to today, we received a new issue #2575 where this
behaviour completely broke on Android devices. Android _always_ use the
IME APIs for handling input... if we think about our solution form
above, it also means that while you are typing on an Android device no
options are being filtered at all. The moment you hit enter/space the
combobox will open and results will be filtered.

This is where this fix comes in. The goals are simple:

1. Make it work
2. Try to make the current code simpler

I started digging to see _why_ this `wあ` was even submitted. A normal
input field doesn't do that?! We have some code that does the following
things:

1. Sync the selected value with the `input` such that if you update the
   value from the outside, then the value in the `input` is up-to-date
   with the `displayValue` of that incoming value.
2. A fix for macOS VoiceOver to improve the VoiceOver experience when
   opening the `Combobox` component. This is done by manually resetting
   the value of the `input` field.

   1. Keep track of the current value
   2. Keep track of the current selection range (start/end) state
   3. Reset the input to an empty string `''`
   4. Restore the value to the captured value
   5. Restore the selection range

When you are typing, the input field doesn't have to update yet because
typing doesn't cause an option to become the `selected` option,
therefore it doesn't have to sync the value yet. So (1.) isn't the issue
here.

However, when you start typing, the Combobox should open and then we
trigger the macOS VoiceOver fix. This is touching the `input` field
because we change the value & selection.

Because we touched the `input` while the user was still in a composing
mode, it bailed and submitted whatever characters it had. This is the
part that we don't want. Not applying the macOS VoiceOver fix while
typing solves this issue. In addition, because _we_ are touching the
input field, VoiceOver is acting normally.

In hindsight, the solution is very simple: do not touch the input field
when the user is typing.

We still keep track whether the user `isComposing` so that we can bail
on the default `Enter` behaviour (marking the current option as the
selected option) because pressing `Enter` while composing should get out
of the IME.

Fixes: #2575
@vercel
Copy link

vercel bot commented Jul 6, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
headlessui-react ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 6, 2023 10:56am
headlessui-vue ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 6, 2023 10:56am

Copy link
Contributor

@thecrypticace thecrypticace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. I tried to break it but wasn't able to. Seems like this works as I'd expect. Well done! 💪

@RobinMalfait RobinMalfait merged commit 00fdb7e into main Jul 6, 2023
7 checks passed
@RobinMalfait RobinMalfait deleted the fix/issue-2575 branch July 6, 2023 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants