Keyboard Issues

[rainergericke]

Describe the bug
A clear and concise description of what the bug is.
Want I want: use key strokes to start functions in an VSG application. A simple handler has been written. Its only function is to give a response to key stroke 'i'. It is part of the example program 'KeyboardIssue'.

To Reproduce
Steps to reproduce the behavior:

1. Build the example program
2. Start it from a terminal window
3. Hit key 'i'

Expected behavior
A clear and concise description of what you expected to happen.
On all operation systems I expect for one key hit exactly one response line in the terminal window.

Screenshots
If applicable, add screenshots to help explain your problem.

On MacOS one key hit leads to two responses.
On Windows any key hit leads no response.
On Linux the key hit leads to one response, as it should be.

Desktop (please complete the following information):

• OS: macOS Big Sur 11.0.1 X86_64
• OS: Windows 10 Home 20H2
• OS: Linux latest Ubuntu Desktop Intel 20.10
  KeyboardIssue.zip

[robertosfield]

Thanks for the detailed report and test program. Unfortunately, I don't personally have access to Windows or macOS so will have to defer to others for testing/help resolving platform specific issues on these OS's. I'll ping the vsg-users later if a quick resolution isn't possible, Doe the window having focus make any difference to the results for you? Could you add code to your test program that reports both the KeyEvent::keyModified, keyBase and keyModified values so we can see what is happening for all key presses. For instance It may be that 'i' is a modified 'I' under Windows. It might also be useful to have keyPress and keyRelease tracked. Perhaps it would be best for me to create a vsgExample to output these values. I'm currently working on implementing dynamic text so this might be a good place to add UI test as well.

[rainergericke]

On Mac and Windows the VSG window has to have the focus. I added some lines to track also keyRelease events. They are not recognized on the Mac. On Windows neither keyPress events nor keyRelease events can be seen.
I'm not a great expert in deep MacOS programming, but it could be the the keyRelease event that makes the second output, not correctly recognized as keyRelease event. Let the experts look at it.

The other method is to ask for keyEvents and report it. There is a difference between Mac and Windows:
On the Mac modifiers are only recognized together with a printing key like 'I'.
On Windows modifiers are recognized as single keystrokes. Printing keys to double events like on the Mac.
KeyboardIssue2.zip

[robertosfield]

@tomhog does any of these Windows/macOS issues ring any bells?

[robertosfield]

On Sat, 21 Nov 2020 at 18:01, Antoine Hoarau ***@***.***> wrote: @robertosfield <https://github.com/robertosfield> Hello Robert, github actions could allow to easily test on all platforms

I'm not familiar with github actions. Does it support interactive vulkan applications? That's what we'd need in this instance.

[ahoarau]

Sorry robert I misunderstood the issue + you already setup the ci.

[robertosfield]

@rainergericke I've now wrapped up the work on dynamic text so will now create a vsgExample that outputs the keyboard and mouse values, perhaps along the lines of the old osgkeyboard example.

First up I've run your KeyboardIssue example and confirm that it's working under Linux but without Windows or macOS that's all I can do in terms of testing. To help find out what might be happening with the events I've extended the AppKeyboardHandler to test of 'I' and well as 'i', and test keyBase and keyModified:

class AppKeyboardHandler : public vsg::Inherit<vsg::Visitor, AppKeyboardHandler>
{
public:
AppKeyboardHandler(vsg::Viewer* viewer)
: m_viewer(viewer)
{
}
void apply(vsg::KeyPressEvent& keyPress) override
{
if(keyPress.keyBase == 'i') {
std::cout << "keyBase 'i' pressed " << ++m_i_keyBase_counter << " times.\n";
}
if(keyPress.keyModified == 'i') {
std::cout << "keyModified 'i' pressed " << ++m_i_keyModified_counter << " times.\n";
}
if(keyPress.keyBase == 'I') {
std::cout << "keyBase 'I' pressed " << ++m_I_keyBase_counter << " times.\n";
}
if(keyPress.keyModified == 'I') {
std::cout << "keyModified 'I' pressed " << ++m_I_keyModified_counter << " times.\n";
}
}

private:
size_t m_i_keyBase_counter = 0;
size_t m_i_keyModified_counter = 0;
size_t m_I_keyBase_counter = 0;
size_t m_I_keyModified_counter = 0;
vsg::observer_ptrvsg::Viewer m_viewer;
};

When I press 'i' then add the shift key and press i again for 'I' I get the follow results under Linux:
$ bin/KeyboardIssue
keyBase 'i' pressed 1 times.
keyModified 'i' pressed 1 times.
keyBase 'i' pressed 2 times.
keyModified 'I' pressed 1 times.

Which is what I would expect. I'm wondering if Windows and macOS have the meaning flipped. FYI, I wrote the Linux Xcb support, while @tomhog wrote the Windows and macOS support so we may have ended up with different interpretations of what is meant by keyBase and ketModified.

[robertosfield]

@rainergericke I have been reviewing the Linux (Xcb_Window.cpp +.h) and the Windows (Win32_Window.cpp +.h) and it looks to me like in Xcb the unmodified 'i' key reports 'i', while under Windows the unmodified 'i' reports 'I'.

At a guess I'd think that perhaps Windows keyModified will report 'i' when you press the 'i' key. Could you test this as I can't as I don't have a Windows machine to test against.

I haven't looked at macOS, but am wondering if it's a similar story - the default unmodified might be 'I' and modified is 'i'.

If the above is correct then we are left with a decision to make about whether we standardize all keyBase values to work in the same case, or just leave it as the native value. For portability of code I think it would best if keyBase behaved the same for all platforms, even if that means us changing it from the native interpretation.

[rainergericke]

I tested your code changes, to my surprised, it worked also on windows, but it's a good surprise. As you proposed, I started the program and pressed 'i' one time and then 'I' one time. Here is the windows response:

On the Mac I get this, everything seems to be doubled:

[robertosfield]

Thanks for the testing. This starts to make the picture clearer. I would like to have the VSG work the same on each platform as much as possible so it would make sense to standardize the interpretation of what keyBase would be. Modified vs unmodified keys would normally be interpreted as the keycode you get when press shift to get 'A' ad unmodified i.e. no shift you'd get 'a' - which is what I'm getting with Xcb and what your assumption was.

I'm curious, what made you assume 'i' would be the keyBase rather than 'I'?

I haven't looked at the MacOS side yet, t don't want to confuse my little brain too much with different combinations of problems at once. I will look at the next.

[robertosfield]

I have checked in the beginnings of vsginput example that will report event data. Right now it just outputs keyboard and mouse events to the console. It's get near end of working day so I wont' be able to do too much more.

https://github.com/vsg-dev/vsgExamples/tree/Input

[rainergericke]

Sorry for this lack of clearness, I wanted to write lowercase I plus shift, but my browser messes up my input!

[robertosfield]

I have started looking into the double reporting of events in macOS and haven't spotted an obvious reason why VulkanSceneGraph/src/vsg/platform/macos/MacOS_Window.mm would issue two events for each key press/release.

I can't think of anything more I can do for macOS side, the only way to work out what is going on is to put a break point in and use a debugger to figure how the code behaves in repose to user events, or put some debugging into the bool MacOS_Window::handleNSEvent(NSEvent* anEvent) method to see how often each part of the switch gets called.

Do you know if mouse events are also getting double handled?

[rainergericke]

I have found the problem and the solution.

Could you please apply this change?

What happened? VSG got wrong event information. A keyRelease event was marked as a keyPress event. With my proposed change the program response seem to be correct.

[robertosfield]

@rainergericke Well spotted. Could you generate a PR for this fix? Thanks.