Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the key events sent from the embedders to Servo #20331

Open
fabricedesre opened this issue Mar 18, 2018 · 7 comments
Open

Improve the key events sent from the embedders to Servo #20331

fabricedesre opened this issue Mar 18, 2018 · 7 comments

Comments

@fabricedesre
Copy link
Contributor

@fabricedesre fabricedesre commented Mar 18, 2018

The key events sent by the embedder to Servo are based on Glutin ones (https://doc.servo.org/src/msg/constellation_msg.rs.html#22-147), and are then mapped to DOM Key events in components/script/dom/keyboardevent.rs. This is not ideal because Glutin only has a really limited subset of possible DOM Keys (eg. most of the keys in https://w3c.github.io/uievents-key/ are missing).

Would that make sense to have the embedders do themselves the mapping from their input stack to DOM Keys instead? The other option would be to add keys to the current enum, but that doesn't seem very scalable.

@paulrouget ?

@jdm
Copy link
Member

@jdm jdm commented Mar 18, 2018

It does seem like a good idea to define Servo's key types in terms of what web standards describe and require embedders to communicate with Servo using those keys.

@paulrouget
Copy link
Contributor

@paulrouget paulrouget commented Mar 19, 2018

Would that make sense to have the embedders do themselves the mapping from their input stack to DOM Keys instead?

Absolutely.

@modal17
Copy link

@modal17 modal17 commented Apr 2, 2018

May I look into this?

@paulrouget
Copy link
Contributor

@paulrouget paulrouget commented Apr 4, 2018

@modal17 absolutely. Let me know if you have any questions.

@paulrouget
Copy link
Contributor

@paulrouget paulrouget commented Aug 8, 2018

Do DOM key events cover enough to print all that is needed for an IME?

I'm thinking about Chinese characters, or dead keys combo. For example when I type: Option+e a, I see an accented underscore which is then replaced by á,or ni1 in Pinyin will show ni underscored then replaced with , or pressing lll Return will show 丿丿丿 then be replaced by with a chinese keyboard. Also, navigating the temporary string with arrow keys is controlled by the input method. I'm wondering if this is all covered by https://w3c.github.io/uievents-key/#keys-composition

@jdm jdm removed the C-assigned label Aug 8, 2018
@fabricedesre
Copy link
Contributor Author

@fabricedesre fabricedesre commented Aug 8, 2018

Yes, what we really need is full composition support. showIME/hideIME are needed just to trigger the virtual keyboard, and then we need something similar to Gecko's nsITextInputProcessor.

@pyfisch
Copy link
Contributor

@pyfisch pyfisch commented Sep 29, 2018

I am working on this. It mostly works. I just need to refactor the built-in keyboard shortcuts.

pyfisch added a commit to pyfisch/servo that referenced this issue Oct 6, 2018
Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of servo#20331
pyfisch added a commit to pyfisch/servo that referenced this issue Oct 6, 2018
Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of servo#20331
bors-servo added a commit that referenced this issue Oct 6, 2018
WIP Use keyboard-types crate

Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of #20331

What this PR does:

* allow the use non-ASCII keyboards for text input
* decouple keyboard event "key" from "code" (key meaning vs location)

What this PR does not do:

* completely improve keyboard events send from winit and webdriver
* add support for CompositionEvent or IME

Notes:

* The winit embedder does not send keyup events for printable keys (this is a regression)

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21881)
<!-- Reviewable:end -->
pyfisch added a commit to pyfisch/servo that referenced this issue Oct 6, 2018
Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of servo#20331
bors-servo added a commit that referenced this issue Oct 6, 2018
WIP Use keyboard-types crate

Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of #20331

What this PR does:

* allow the use non-ASCII keyboards for text input
* decouple keyboard event "key" from "code" (key meaning vs location)

What this PR does not do:

* completely improve keyboard events send from winit and webdriver
* add support for CompositionEvent or IME

Notes:

* The winit embedder does not send keyup events for printable keys (this is a regression)
* keyboard-types is on crates.io because I believe it to be useful outside of servo. If you prefer I can add a copy in this repo.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21881)
<!-- Reviewable:end -->
pyfisch added a commit to pyfisch/servo that referenced this issue Oct 7, 2018
Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of servo#20331
pyfisch added a commit to pyfisch/servo that referenced this issue Oct 7, 2018
Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of servo#20331
pyfisch added a commit to pyfisch/servo that referenced this issue Oct 7, 2018
Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of servo#20331
pyfisch added a commit to pyfisch/servo that referenced this issue Oct 7, 2018
Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of servo#20331
pyfisch added a commit to pyfisch/servo that referenced this issue Oct 7, 2018
Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of servo#20331
pyfisch added a commit to pyfisch/servo that referenced this issue Oct 7, 2018
Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of servo#20331
bors-servo added a commit that referenced this issue Oct 7, 2018
Use keyboard-types crate

Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of #20331

What this PR does:

* allow the use non-ASCII keyboards for text input
* decouple keyboard event "key" from "code" (key meaning vs location)

What this PR does not do:

* completely improve keyboard events send from winit and webdriver
* add support for CompositionEvent or IME

Notes:

* The winit embedder does not send keyup events for printable keys (this is a regression)
* keyboard-types is on crates.io because I believe it to be useful outside of servo. If you prefer I can add a copy in this repo.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21881)
<!-- Reviewable:end -->
pyfisch added a commit to pyfisch/servo that referenced this issue Oct 7, 2018
Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of servo#20331
bors-servo added a commit that referenced this issue Oct 7, 2018
Use keyboard-types crate

Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of #20331

What this PR does:

* allow the use non-ASCII keyboards for text input
* decouple keyboard event "key" from "code" (key meaning vs location)

What this PR does not do:

* completely improve keyboard events send from winit and webdriver
* add support for CompositionEvent or IME

Notes:

* The winit embedder does not send keyup events for printable keys (this is a regression)
* keyboard-types is on crates.io because I believe it to be useful outside of servo. If you prefer I can add a copy in this repo.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21881)
<!-- Reviewable:end -->
bors-servo added a commit that referenced this issue Oct 13, 2018
Use keyboard-types crate

Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of #20331

What this PR does:

* allow the use non-ASCII keyboards for text input
* decouple keyboard event "key" from "code" (key meaning vs location)

What this PR does not do:

* completely improve keyboard events send from winit and webdriver
* add support for CompositionEvent or IME

Notes:

* The winit embedder does not send keyup events for printable keys (this is a regression)
* keyboard-types is on crates.io because I believe it to be useful outside of servo. If you prefer I can add a copy in this repo.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21881)
<!-- Reviewable:end -->
bors-servo added a commit that referenced this issue Oct 15, 2018
Use keyboard-types crate

Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of #20331

What this PR does:

* allow the use non-ASCII keyboards for text input
* decouple keyboard event "key" from "code" (key meaning vs location)

What this PR does not do:

* completely improve keyboard events send from winit and webdriver
* add support for CompositionEvent or IME

Notes:

* The winit embedder does not send keyup events for printable keys (this is a regression)
* keyboard-types is on crates.io because I believe it to be useful outside of servo. If you prefer I can add a copy in this repo.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21881)
<!-- Reviewable:end -->
bors-servo added a commit that referenced this issue Oct 17, 2018
Use keyboard-types crate

Have embedders send DOM keys to servo and use a strongly typed KeyboardEvent
from the W3C UI Events spec. All keyboard handling now uses the new types.

Introduce a ShortcutMatcher to recognize key bindings. Shortcuts are now
recognized in a uniform way.

Updated the winit port.
Updated webdriver integration.

part of #20331

What this PR does:

* allow the use non-ASCII keyboards for text input
* decouple keyboard event "key" from "code" (key meaning vs location)

What this PR does not do:

* completely improve keyboard events send from winit and webdriver
* add support for CompositionEvent or IME

Notes:

* The winit embedder does not send keyup events for printable keys (this is a regression)
* keyboard-types is on crates.io because I believe it to be useful outside of servo. If you prefer I can add a copy in this repo.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21881)
<!-- Reviewable:end -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.