Virtual keyboard missing keys and not showing up in any emulator

[rsn8887]

Description

When binding any key to "on-screen keyboard" in input->hotkey binds, the virtual keyboard only shows up in the gui, but not in the emulators. In addition, the virtual keyboard does not show all the keys that are present on a physical keyboard. For example, keypad keys, control, alt etc. are all missing.

Expected behavior

The virtual keyboard should not only appear in the gui, but also show up inside the emulators such as Dosbox and Vice when I press the hotkey. Furthermore, it should allow me to select and press keys using a game controller. Those should then be send to the emulator core. Basically the virtual keyboard should act as if a real physical keyboard was connected and those keys were pressed on it at any place within Retroarch (gui, emulators etc.)

Actual behavior

see description

Steps to reproduce the bug

1. bind hotkey to on-screen keyboard toggle
2. press hotkey inside gui, virtual keyboard appears but is missing keys like alt, ctrl etc.
3. press hotkey inside emulator, virtual keyboard does not appear

Bisect Results

The virtual keyboard basically never worked for me in any useful way apart from limited functionality inside the gui itself.

Version/Commit

You can find this information under Information/System Information

• RetroArch: 1.7.0 nightly

Environment information

• OS: RPi, Vita, PC (tested on those three)

[Ferk]

The virtual keyboard should probably be a special case of overlay that can be navigated with the gamepad and can have a hotkey to bring it up, even while in the GUI.

It makes sense for it to be customizable just like other overlays, so people in non-english languages can set a custom overlay as osk to add their own symbols. Or to to use core-specific config to set specific keyboards for the particular system.

The support for overlays is already in retroarch, the tricky part would probably be allowing navigation with the retropad.

[i30817]

Bumping this, because it would be really useful in some cores for phone devices. I mean, some cores require you press buttons but not necessarily in a frenetic way that would require button mapping.

For instance imagine a Adventure Game Studio core, or even the scummvm core. It would be very good if the virtual keyboard could seamlessly plugin as the device without any focus bugs, and would easily reduce port complexity (unless the phones already do this on their virtual keyboards... something i rather doubt for RA cores, but may happen).

The usecase i'm thinking is:
user presses button, RA pauses cores, ups VK.
user enters line (should multiline be supported and how?)
on carriage return (or another method if multi-line), unpause the core and send the string into it to be handled by the core focus system.

[Zlika]

llibretro-uae and libretro-vice use a custom "VKDB" virtual keyboard feature (cf. libretro/vice-libretro@d74caf5). I think it would be interesting to add this feature to libretro-common to make it easily available for all cores.

[rsn8887]

Yes, thanks to the heroic effort of @sonninnos who worked tirelessly on Vice and PUAE, we now have really good virtual keyboards in those cores.

My issue here was created before all that work was done. I was hoping for a good universal virtual keyboard provided by the frontend. Such a virtual keyboard could work in any core, even those that don't have their own vkbd, for example in dosbox-svn. It could be controlled via touch and/or dpad, and it could provide all the keys found on a regular PC, with the mapping to native keys just as a regular physical keyboard would be mapped.

I noticed that Amstrad CPC and Fusion Spectrum cores already have nice vkbd implementations, too. So I guess the only core that really needs a vkbd is dosbox_svn (and dosbox_core).

I think it would be easier now to just implement a vkbd in dosbox, be done with it, and not worry about a universal keyboard provided by the frontend any more.

[Spawk-HS]

Any progress on a dosbox VKBD? Almost every one of my games require typing at least at some point... enter your name, save game name! Being able to load them is great but the core is not actually "usable" at this point.

[rsn8887]

Well, you can use a BT keyboard on Vita and a USB keyboard on Switch for Dosbox SVN, but, yeah, not very usable on such platforms.