Key events do not map to physical keys #114

Closed
dougbinks opened this Issue Aug 13, 2013 · 27 comments

Comments

Projects
None yet
6 participants
@dougbinks
Contributor

dougbinks commented Aug 13, 2013

Issue

Whilst investigating implementation of key tokens to text I discovered that whilst the key tokens refer to physical keys on OS X, they refer to locale translated keys on Windows 8 (and potentially other versions of windows, as yet untested).

This creates a potential problem in setting up the default keys for an application, such as WASD for an FPS. Whilst on OS X these can be tested as the known physical key W on Windows they cannot.

Reproduction steps

  1. On Windows, set up a French (AZERTY) and English (QWERTY) keyboard in Control Panel->Language->[Your Language]->Options->Input Method->Add an input method.
  2. Run the events test project.
  3. First set the QWERTY English keyboard active and press the W key (Z if your actual keyboard is French).
  4. Set the AZERTY French keyboard active and press the same key.
  5. Look at text output.

Results

The following is the output from the Events test project on Windows when pressing the W key on a keyboard once, then switching to a French keyboard layout:

000000ea at 38.355: Key 0x0057 Scancode 0x0011 (W) was pressed
000000eb at 38.362: Character 0x00000077 (w) input
000000ec at 38.459: Key 0x0057 Scancode 0x0011 (W) was released
000000ed at 48.835: Key 0x005a Scancode 0x0011 (Z) was pressed
000000ee at 48.841: Character 0x0000007a (z) input
000000ef at 48.907: Key 0x005a Scancode 0x0011 (Z) was released

As can be seen the scancode remains the same but the key code returned switches from 0x57 to 0x5a.

Expected

The documentation states: "The key functions deal with physical keys, with key tokens named after their use on the standard US keyboard layout."

Thus we would expect to see:

000000ea at 38.355: Key 0x0057 Scancode 0x0011 (W) was pressed
000000eb at 38.362: Character 0x00000077 (w) input
000000ec at 38.459: Key 0x0057 Scancode 0x0011 (W) was released
000000ed at 48.835: Key 0x0057 Scancode 0x0011 (W) was pressed
000000ee at 48.841: Character 0x0000007a (z) input
000000ef at 48.907: Key 0x0057 Scancode 0x0011 (W) was released

Note that here the char callback changes but the key code does not.

Resolution

This behaviour is the same in both GLFW 2 and 3. Thus changing this to match documentation is likely problematic and additionally may involve some overhead in translating the virtual key codes to US layout specific key codes. I thus recommend that for now the documentation is changed to reflect the difference on Windows vs other OSs.

Optionally, a translation of physical key codes to keyboard locale keys could be implemented. I may be able to undertake this as part of the glfwGetKeyName work.

@dougbinks

This comment has been minimized.

Show comment Hide comment
@dougbinks

dougbinks Aug 15, 2013

Contributor

I'm working on a fix for this, which loads the US keyboard and then uses this to get the US virtual key from the scan code. This will also need a change in my vkey to text function once complete.

I don't think the overhead will be substantial since I can load the kb at init time, but there may be unwanted side effects since I'm seeing the US keyboard appear in my list of keyboards once I load it (even though the docs say it's only loaded for the process).

Contributor

dougbinks commented Aug 15, 2013

I'm working on a fix for this, which loads the US keyboard and then uses this to get the US virtual key from the scan code. This will also need a change in my vkey to text function once complete.

I don't think the overhead will be substantial since I can load the kb at init time, but there may be unwanted side effects since I'm seeing the US keyboard appear in my list of keyboards once I load it (even though the docs say it's only loaded for the process).

@dougbinks

This comment has been minimized.

Show comment Hide comment
@dougbinks

dougbinks Aug 16, 2013

Contributor

I have a first draft of a fix which seems to work well on my glfw fork, branch windows-input.

This is a 'breaking' change in that any keys set up for Windows non-US keyboards will now change, so it may be a good idea to have this solution handled with a define - which I can implement if required (thought his adds legacy issues into the code-base so a clean break may be thought best).

This doesn't use the keyboard loading solution mentioned above, so has no side effects nor extra runtime overhead.

Contributor

dougbinks commented Aug 16, 2013

I have a first draft of a fix which seems to work well on my glfw fork, branch windows-input.

This is a 'breaking' change in that any keys set up for Windows non-US keyboards will now change, so it may be a good idea to have this solution handled with a define - which I can implement if required (thought his adds legacy issues into the code-base so a clean break may be thought best).

This doesn't use the keyboard loading solution mentioned above, so has no side effects nor extra runtime overhead.

@ghost ghost assigned elmindreda Aug 22, 2013

@elmindreda

This comment has been minimized.

Show comment Hide comment
@elmindreda

elmindreda Aug 22, 2013

Owner

Verified. This needs to be fixed. The Windows port will have to be changed to conform to the documentation.

I will review your fix once 3.0.2 is out, which it will hopefully be today.

Owner

elmindreda commented Aug 22, 2013

Verified. This needs to be fixed. The Windows port will have to be changed to conform to the documentation.

I will review your fix once 3.0.2 is out, which it will hopefully be today.

@dougbinks

This comment has been minimized.

Show comment Hide comment
@dougbinks

dougbinks Aug 22, 2013

Contributor

Good luck with 3.0.2!

Contributor

dougbinks commented Aug 22, 2013

Good luck with 3.0.2!

@bakercp bakercp referenced this issue in openframeworks/openFrameworks Aug 23, 2013

Closed

pointsAsTextures example & AZERTY keyboards #2501

@NathanSweet

This comment has been minimized.

Show comment Hide comment
@NathanSweet

NathanSweet Aug 25, 2013

Contributor

Some games prefer using the user's keyboard locale, eg RTS games. Other games prefer the physical keyboard layout, eg first person shooters. However, using the physical layout makes it impossible to tell the user what keys to press. Eg, if I tell users to use WSAD and they aren't using a US keyboard then my instructions are wrong and the user is confused. Worse, there is no workaround.

I would argue that all games should use the user's keyboard locale. First person shooters can solve the problem by providing the user with a button remapping screen. I think we should keep the Windows backend as it is and "fix" the Mac backend so the user's keyboard locale is used. To do this we need to look at [NSEvent charactersIgnoringModifiers] and map the Unicode character to a GLFW key instead of using [NSEvent keyCode].

I would also find it acceptable to have an option to choose between physical and user locale, though I do think using the locale is best.

Contributor

NathanSweet commented Aug 25, 2013

Some games prefer using the user's keyboard locale, eg RTS games. Other games prefer the physical keyboard layout, eg first person shooters. However, using the physical layout makes it impossible to tell the user what keys to press. Eg, if I tell users to use WSAD and they aren't using a US keyboard then my instructions are wrong and the user is confused. Worse, there is no workaround.

I would argue that all games should use the user's keyboard locale. First person shooters can solve the problem by providing the user with a button remapping screen. I think we should keep the Windows backend as it is and "fix" the Mac backend so the user's keyboard locale is used. To do this we need to look at [NSEvent charactersIgnoringModifiers] and map the Unicode character to a GLFW key instead of using [NSEvent keyCode].

I would also find it acceptable to have an option to choose between physical and user locale, though I do think using the locale is best.

@dougbinks

This comment has been minimized.

Show comment Hide comment
@dougbinks

dougbinks Aug 25, 2013

Contributor

This new feature for translating Key Tokens to text might help solve the RTS issue you mention by searching for the key which matches a given character code at initialization or configuration time, though for some keyboard layouts you'd probably not get Latin characters as the default translation.

A configurable solution might be possible, with both physical and virtual keys able to be used. This would take more effort though, as would moving to virtual keys on all platforms.

Contributor

dougbinks commented Aug 25, 2013

This new feature for translating Key Tokens to text might help solve the RTS issue you mention by searching for the key which matches a given character code at initialization or configuration time, though for some keyboard layouts you'd probably not get Latin characters as the default translation.

A configurable solution might be possible, with both physical and virtual keys able to be used. This would take more effort though, as would moving to virtual keys on all platforms.

@NathanSweet

This comment has been minimized.

Show comment Hide comment
@NathanSweet

NathanSweet Sep 10, 2013

Contributor

I was wrong in my suggestion to use charactersIgnoringModifiers and not keyCode. keyCode is already device independent, so we just need to map the keyCode to GLFW keys. This should be easy, then we just need Linux to use the keyboard locale and I would be happy.

Contributor

NathanSweet commented Sep 10, 2013

I was wrong in my suggestion to use charactersIgnoringModifiers and not keyCode. keyCode is already device independent, so we just need to map the keyCode to GLFW keys. This should be easy, then we just need Linux to use the keyboard locale and I would be happy.

@elmindreda

This comment has been minimized.

Show comment Hide comment
@elmindreda

elmindreda Sep 10, 2013

Owner

Key codes will continue to map to physical keys. If you want to find which, if any, physical key maps to a given letter in the current layout then the planned feature @dougbinks mentions above will be useful.

Owner

elmindreda commented Sep 10, 2013

Key codes will continue to map to physical keys. If you want to find which, if any, physical key maps to a given letter in the current layout then the planned feature @dougbinks mentions above will be useful.

@dougbinks

This comment has been minimized.

Show comment Hide comment
@dougbinks

dougbinks Sep 12, 2013

Contributor

Elmindreda - if you need me to submit a pull request for the proposed change I have to resolve this let me know.

Also, feel free to ask for any modifications or changes to the code you need if it has issues.

Contributor

dougbinks commented Sep 12, 2013

Elmindreda - if you need me to submit a pull request for the proposed change I have to resolve this let me know.

Also, feel free to ask for any modifications or changes to the code you need if it has issues.

@elmindreda

This comment has been minimized.

Show comment Hide comment
@elmindreda

elmindreda Sep 12, 2013

Owner

@dougbinks I'm afraid I won't have time for this issue until 3.0.4 and I barely have time for 3.0.3 at the moment.

Owner

elmindreda commented Sep 12, 2013

@dougbinks I'm afraid I won't have time for this issue until 3.0.4 and I barely have time for 3.0.3 at the moment.

@dougbinks

This comment has been minimized.

Show comment Hide comment
@dougbinks

dougbinks Sep 13, 2013

Contributor

@elmindreda No problem - let me know if there's anything I can do to take some load off, particularly on Windows/Mac/Linux (I'm currently not developing for Android or iOS).

Contributor

dougbinks commented Sep 13, 2013

@elmindreda No problem - let me know if there's anything I can do to take some load off, particularly on Windows/Mac/Linux (I'm currently not developing for Android or iOS).

@elmindreda

This comment has been minimized.

Show comment Hide comment
@elmindreda

elmindreda Sep 13, 2013

Owner

@dougbinks Thank you! Testing of the OS X cursor input code, once I have bug fixes to test, would be very helpful.

Owner

elmindreda commented Sep 13, 2013

@dougbinks Thank you! Testing of the OS X cursor input code, once I have bug fixes to test, would be very helpful.

@dougbinks

This comment has been minimized.

Show comment Hide comment
@dougbinks

dougbinks Sep 13, 2013

Contributor

Can do. Just point me at it when you're ready. I'm on Central European Time
so there may be some delay.

On 13 September 2013 12:33, Camilla Berglund notifications@github.comwrote:

@dougbinks https://github.com/dougbinks Thank you! Testing of the OS X
cursor input code, once I have bug fixes to test, would be very helpful.


Reply to this email directly or view it on GitHubhttps://github.com/glfw/glfw/issues/114#issuecomment-24386057
.

Contributor

dougbinks commented Sep 13, 2013

Can do. Just point me at it when you're ready. I'm on Central European Time
so there may be some delay.

On 13 September 2013 12:33, Camilla Berglund notifications@github.comwrote:

@dougbinks https://github.com/dougbinks Thank you! Testing of the OS X
cursor input code, once I have bug fixes to test, would be very helpful.


Reply to this email directly or view it on GitHubhttps://github.com/glfw/glfw/issues/114#issuecomment-24386057
.

@bilderbuchi bilderbuchi referenced this issue in openframeworks/openFrameworks Sep 13, 2013

Closed

Wrong keys are returned by GLFW on keypress #2562

@elmindreda

This comment has been minimized.

Show comment Hide comment
@elmindreda

elmindreda Oct 7, 2013

Owner

I've taken this and your key name work, rebased it and put it into the keyboard branch of this repository. I'll keep working on both from there.

Owner

elmindreda commented Oct 7, 2013

I've taken this and your key name work, rebased it and put it into the keyboard branch of this repository. I'll keep working on both from there.

@arturoc

This comment has been minimized.

Show comment Hide comment
@arturoc

arturoc Dec 4, 2013

Contributor

What's the state of this? i see that there's now a glfwSetCharCallback but that only reports key presses, it would be useful to have the same parameters as glfwSetKeyCallback.

For example an application might want to increase a value every frame while the '+' key is pressed and stop updating it when it's released but right now we can only detect reliably if '+' has been pressed but not when it's been released since '+' is mapped differently depending on the keyboard

Another possibility would be to make public the _glfwKeySym2Unicode function so it can be called from the key callback.

Contributor

arturoc commented Dec 4, 2013

What's the state of this? i see that there's now a glfwSetCharCallback but that only reports key presses, it would be useful to have the same parameters as glfwSetKeyCallback.

For example an application might want to increase a value every frame while the '+' key is pressed and stop updating it when it's released but right now we can only detect reliably if '+' has been pressed but not when it's been released since '+' is mapped differently depending on the keyboard

Another possibility would be to make public the _glfwKeySym2Unicode function so it can be called from the key callback.

@webhat

This comment has been minimized.

Show comment Hide comment
@webhat

webhat Dec 15, 2013

@dougbinks what's the state of this? Will it be merged?

webhat commented Dec 15, 2013

@dougbinks what's the state of this? Will it be merged?

@dougbinks

This comment has been minimized.

Show comment Hide comment
@dougbinks

dougbinks Dec 15, 2013

Contributor

@webhat I can't merge this as I'm not an admin on glfw. @elmindreda is aiming for milestone 3.1 it seems. If you need this merged in your own code you can create your own local branch or fork of glfw to merge it.

Contributor

dougbinks commented Dec 15, 2013

@webhat I can't merge this as I'm not an admin on glfw. @elmindreda is aiming for milestone 3.1 it seems. If you need this merged in your own code you can create your own local branch or fork of glfw to merge it.

@elmindreda

This comment has been minimized.

Show comment Hide comment
@elmindreda

elmindreda Dec 15, 2013

Owner

Yes, this will be merged with 3.1.

Owner

elmindreda commented Dec 15, 2013

Yes, this will be merged with 3.1.

@webhat

This comment has been minimized.

Show comment Hide comment
@webhat

webhat Dec 15, 2013

@dougbinks @elmindreda cool thanks!

webhat commented Dec 15, 2013

@dougbinks @elmindreda cool thanks!

@elmindreda

This comment has been minimized.

Show comment Hide comment
@elmindreda

elmindreda Mar 21, 2014

Owner

The Windows physical key fix part of this has been merged with dc987ed.

Owner

elmindreda commented Mar 21, 2014

The Windows physical key fix part of this has been merged with dc987ed.

@dougbinks

This comment has been minimized.

Show comment Hide comment
@dougbinks

dougbinks Mar 21, 2014

Contributor

Fix confirmed, now get correct results when switching from a US to French keyboard:

00000064 to 1 at 4.991: Key 0x0057 Scancode 0x0011 (W) was pressed
00000065 to 1 at 4.999: Character 0x00000077 (w) input
00000066 to 1 at 5.080: Key 0x0057 Scancode 0x0011 (W) was released

// switch to French keyboard here:

00000067 to 1 at 13.398: Key 0x0057 Scancode 0x0011 (W) was pressed
00000068 to 1 at 13.406: Character 0x0000007a (z) input
00000069 to 1 at 13.458: Key 0x0057 Scancode 0x0011 (W) was released

Am closing as per normal submitter closes rules, let me know if that's not the appropriate action for the glfw repo.

Contributor

dougbinks commented Mar 21, 2014

Fix confirmed, now get correct results when switching from a US to French keyboard:

00000064 to 1 at 4.991: Key 0x0057 Scancode 0x0011 (W) was pressed
00000065 to 1 at 4.999: Character 0x00000077 (w) input
00000066 to 1 at 5.080: Key 0x0057 Scancode 0x0011 (W) was released

// switch to French keyboard here:

00000067 to 1 at 13.398: Key 0x0057 Scancode 0x0011 (W) was pressed
00000068 to 1 at 13.406: Character 0x0000007a (z) input
00000069 to 1 at 13.458: Key 0x0057 Scancode 0x0011 (W) was released

Am closing as per normal submitter closes rules, let me know if that's not the appropriate action for the glfw repo.

@dougbinks dougbinks closed this Mar 21, 2014

@elmindreda

This comment has been minimized.

Show comment Hide comment
@elmindreda

elmindreda Mar 21, 2014

Owner

@dougbinks Perfect, thank you.

Owner

elmindreda commented Mar 21, 2014

@dougbinks Perfect, thank you.

@elmindreda

This comment has been minimized.

Show comment Hide comment
@elmindreda

elmindreda Mar 21, 2014

Owner

I'll open another issue for the key name feature.

Owner

elmindreda commented Mar 21, 2014

I'll open another issue for the key name feature.

@EmJayJay

This comment has been minimized.

Show comment Hide comment
@EmJayJay

EmJayJay Oct 30, 2014

Is this the same problem I'm experiencing where I'm just getting key presses with the glfwGetKey(...) but they don't match my keyboards keys?
I have made a function which gets all the key states and adds the result if true as binary operation as shown below

if(glfwGetKey(this->window, GLFW_KEY_D))
{
keyStates->ah = keyStates->ah | 00010000;
}

keyStates is a struct which holds physical buttons represented as char hence the 00000000
Since I'm from Europe and have åäö letters on my keyboard, it doesn't match the US keyboard or its layout.
GLFW_KEY_B matches my keyboard's F letter and Escapce and F5 are the same as well.
Mouse buttons work normally but mouse button 1 and 2 are the same in my mouse.
Using Ubuntu 12.04.
Should I use something else to get the keyboards inputs?

Is this the same problem I'm experiencing where I'm just getting key presses with the glfwGetKey(...) but they don't match my keyboards keys?
I have made a function which gets all the key states and adds the result if true as binary operation as shown below

if(glfwGetKey(this->window, GLFW_KEY_D))
{
keyStates->ah = keyStates->ah | 00010000;
}

keyStates is a struct which holds physical buttons represented as char hence the 00000000
Since I'm from Europe and have åäö letters on my keyboard, it doesn't match the US keyboard or its layout.
GLFW_KEY_B matches my keyboard's F letter and Escapce and F5 are the same as well.
Mouse buttons work normally but mouse button 1 and 2 are the same in my mouse.
Using Ubuntu 12.04.
Should I use something else to get the keyboards inputs?

@dougbinks

This comment has been minimized.

Show comment Hide comment
@dougbinks

dougbinks Oct 30, 2014

Contributor

The keys returned by glfwGetKey are physical keys, i.e. they correspond to
the position on the keyboard and not the character from the locale. Names
are based on the US keyboard layout:
http://www.glfw.org/docs/latest/group__keys.html

If you want to get the translated character (for instance so that the user
can type text) you should use the Unicode char callback
via glfwSetCharCallback:
http://www.glfw.org/docs/latest/group__input.html#ga1103f1876518acecb5976f6b307c51d1

Which you use would depend on which you require - for buttons to perform
actions such as moving a spaceship you should use the physical key callback
or glfwGetKey function, and for text input you should use the Unicode
character callback.

The glfwGetKeyName feature #117 was
intended to add support for mapping physical keys to key names to present
these in the UI. I don't know the present state of this API.

On 30 October 2014 16:13, EmJayJay notifications@github.com wrote:

Is this the same problem I'm experiencing where I'm just getting key
presses with the glfwGetKey(...) but they don't match my keyboards keys?
I have made a function which gets all the key states and adds the result
if true as binary operation as shown below

if(glfwGetKey(this->window, GLFW_KEY_D))
{
keyStates->ah = keyStates->ah | 00010000;
}

keyStates is a struct which holds physical buttons represented as char
hence the 00000000
Since I'm from Europe and have åäö letters on my keyboard, it doesn't
match the US keyboard or its layout.
GLFW_KEY_B matches my keyboard's F letter and Escapce and F5 are the same
as well.
Mouse buttons work normally but mouse button 1 and 2 are the same in my
mouse.
Using Ubuntu 12.04.
Should I use something else to get the keyboards inputs?


Reply to this email directly or view it on GitHub
#114 (comment).

Contributor

dougbinks commented Oct 30, 2014

The keys returned by glfwGetKey are physical keys, i.e. they correspond to
the position on the keyboard and not the character from the locale. Names
are based on the US keyboard layout:
http://www.glfw.org/docs/latest/group__keys.html

If you want to get the translated character (for instance so that the user
can type text) you should use the Unicode char callback
via glfwSetCharCallback:
http://www.glfw.org/docs/latest/group__input.html#ga1103f1876518acecb5976f6b307c51d1

Which you use would depend on which you require - for buttons to perform
actions such as moving a spaceship you should use the physical key callback
or glfwGetKey function, and for text input you should use the Unicode
character callback.

The glfwGetKeyName feature #117 was
intended to add support for mapping physical keys to key names to present
these in the UI. I don't know the present state of this API.

On 30 October 2014 16:13, EmJayJay notifications@github.com wrote:

Is this the same problem I'm experiencing where I'm just getting key
presses with the glfwGetKey(...) but they don't match my keyboards keys?
I have made a function which gets all the key states and adds the result
if true as binary operation as shown below

if(glfwGetKey(this->window, GLFW_KEY_D))
{
keyStates->ah = keyStates->ah | 00010000;
}

keyStates is a struct which holds physical buttons represented as char
hence the 00000000
Since I'm from Europe and have åäö letters on my keyboard, it doesn't
match the US keyboard or its layout.
GLFW_KEY_B matches my keyboard's F letter and Escapce and F5 are the same
as well.
Mouse buttons work normally but mouse button 1 and 2 are the same in my
mouse.
Using Ubuntu 12.04.
Should I use something else to get the keyboards inputs?


Reply to this email directly or view it on GitHub
#114 (comment).

@elmindreda

This comment has been minimized.

Show comment Hide comment
@elmindreda

elmindreda Oct 30, 2014

Owner

@dougbinks The state is as follows.

The only subset that I've found support for on all platforms is the names of printable keys according to the current layout. I'm trying to decide between making a function that only does this and adding static strings for the remaining named keys to glfwGetKeyName, neither of which addresses unnamed keys like Play, Email or Super-Duper-Extra-Gamer-Key-7.

Owner

elmindreda commented Oct 30, 2014

@dougbinks The state is as follows.

The only subset that I've found support for on all platforms is the names of printable keys according to the current layout. I'm trying to decide between making a function that only does this and adding static strings for the remaining named keys to glfwGetKeyName, neither of which addresses unnamed keys like Play, Email or Super-Duper-Extra-Gamer-Key-7.

@EmJayJay

This comment has been minimized.

Show comment Hide comment
@EmJayJay

EmJayJay Oct 30, 2014

Closer look into the code revealed that my code had the problem, not GLFW. For some reason the | 00100000 didn't work. It might have read it as integer.

Closer look into the code revealed that my code had the problem, not GLFW. For some reason the | 00100000 didn't work. It might have read it as integer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment