Add support for GNOME 43

[archerallstars]

Describe the bug
The extension doesn't work on GNOME 43.

To Reproduce
Steps to reproduce the behavior:

1. Update to GNOME 43.
2. The extension can't be enabled on GNOME 43.

Expected behavior
The extension works on GNOME 43.

Screenshots

Environment (please complete the following information):

• Distro: openSUSE Tumbleweed
• Gnome version: 43

Additional context

• Have you been running your system for a while, or is it a fresh installation?
  I have been running my system for a while.
• Are you running any other extensions that might interfere with on-screen keyboard?
  No.
• Does journalctl /usr/bin/gnome-shell -f -o cat show any errors?
  No.
• Can you see any errors in looking glass? (Press Alt + F2, enter lg, switch to "extensions" tab and click "show errors" for Improved OSK extension)
  No.

[Tuxman2]

👍

Works great on Gnome 42. Waiting support for Gnome 43.

Thanks.

[lacereation]

I got it "working" on 43. I did a couple of things, I am not exactly sure what got it to work. And I am wondering if I really got it work in 43 or if the ctrl alt tab + arrow keys I now have are maybe an update to the stock osk for Gnome 43.

I did do the following + maybe they will work for others.

1. Disable Version checking. Here is a guide: https://www.addictivetips.com/ubuntu-linux-tips/disable-the-version-check-for-gnome-extensions/
2. Edit ~/.local/share/gnome-shell/extensions/improved-osk@nick-shmyrev.dev/metadata.json
   After line 9, which reads
   "42,"
   add
   "43"
   The stock osk will come up, the directional keys, etc only will show if you click within the terminal. Then they could be obscured by the size of the osk. You can adjust in the ext's settings. Anyways it is usable, tho I get no F keys or ESC.

[nick-shmyrev]

I got it "working" on 43. I did a couple of things, I am not exactly sure what got it to work. And I am wondering if I really got it work in 43 or if the ctrl alt tab + arrow keys I now have are maybe an update to the stock osk for Gnome 43.

Yep, good guess, that was the built-in OSK. Gnome 43 added a terminal mode to the default OSK that includes Ctrl, Alt, Tab and arrow keys, but it only shows when you're using terminal.

This extension used to rewrite quite a few methods of the default OSK, and that made maintaining it rather difficult. At this point, I'm considering rewriting it to make use of the "terminal mode" and extended keyboard layout that was added in Gnome 43. However, I'd have to test everything, and manually create and package extended OSK layouts for each language, so it'll take some time to get it working.

If you really need an extended OSK in Gnome 43 right now, there's a WIP branch here: https://github.com/nick-shmyrev/improved-osk-gnome-ext/tree/gnome_43_rewrite. Basically, it replaces the default US layout with an extended one made for a terminal mode. I've briefly tested it in Fedora 37 Beta, and it seems to be working. However, it currently lacks F1-12 buttons, Escape and Super keys, doesn't support any key combinations using Shift, and only shows additional keys for US layout. To install it, you'll need to clone that branch, and copy the files into the extension dir (~/.local/share/gnome-shell/extensions/improved-osk@nick-shmyrev.dev), then reboot or log out and back in. Again, that's an untested work-in-progress branch, so use at your own risk.

[Tuxman2]

@nick-shmyrev: Thanks for your work. I'm still on Gnome 42 because my distribution hasn't provided the Gnome 43 version yet. I'm not in a hurry because I'm waiting for all the gnome extensions I use to be available under Gnome 43, including yours.

[rduivenvoorde]

hi, on Debian testing here, Gnome-shell 43.1
Tried your WIP, installed ibus, even editted metadata.json to have 43.1
But the improved one doesnot show up? Any hints to investigate?

[LeSnake04]

Is there an ETA?

Fedora 37 stable just got released, but I would like to wait until this gets updated since I have a thinkpad yoga and really need this extension.

[HughR]

I am typing this on a tablet with Fedora 37. Using a physical keyboard.
Using the tables without a physical keyboard isn't functional with the Gnome 43 OSK.
Maybe your extension would be enough to redeem is.

I would like the keyboard to have the extra keys I'm used to, and not only for input to Gnome Terminal. For example, I use them in FireFox. Your extension seems to do this.

I would like the keyboard to be smaller, sized to the actual buttons, and to be able to be dragged around. That would interfere less with viewing what's on the screen. ChromeOS seems to have this.

Thanks!

[ruralTom]

Man, I should not have upgraded to 43. The default osk is wet hot trash. Difficult to use on my tablet, numbers are buried and it works poorly. I am eagerly waiting for your osk to update, it's a ferrari where as the ubuntu osk is a goat pulled cart with square wheels. My surface 3 pro is hamstrung because the default osk is soooo bad

[nick-shmyrev]

I would like the keyboard to be smaller, sized to the actual buttons, and to be able to be dragged around. That would interfere less with viewing what's on the screen. ChromeOS seems to have this.

Thanks!

Check out Onboard, if I recall correctly it was draggable.

[HughR]

Check out Onboard, if I recall correctly it was draggable.

Thanks for the pointer. https://github.com/schuhumi/gnome-shell-extension-onboard-integration

Unfortunately it was last updated six years ago. It supports Gnome 3.20 so I imagine that it won't work now.

It looks as if the meat of it is only 80 lines of Javascript so it might be easy to fix but I know nothing about how Gnome Shell Extensions work.

[nick-shmyrev]

There's an updated version of the extension available at https://github.com/nick-shmyrev/improved-osk-gnome-ext/tree/gnome_43_rewrite. It's still a work-in-progress, but it now has extended keys for US, FR, DE, ES, RU and UA layouts. If the layout you're using is not included, let me know, and I'll add it. Or, feel free to modify it yourself (see /src/data/osk-layouts dir) and make a PR.

To install extension, clone the gnome_43_rewrite branch, run package-extension.sh to create a .zip file. Then open terminal and run this from the directory with the .zip file:

gnome-extensions install improvedosk@nick-shmyrev.dev.shell-extension.zip

Known issues:

• Numbers layer won't latch. This is an upstream issue, see https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/5763. Hopefully Gnome devs fix it soon, but I might also take a stab at it.
• Key combinations using Shift only work in terminal for the time being.
• Currently, terminal mode doesn't show capital letters even when Shift is pressed. This is purely cosmetic, and is a side effect of being able to use Shift in key combinations.

[ShaunTheSalvo]

Hello,

I've just now installed the Gnome 43 rewrite of Improved OSK on my Arch system, using it on my (rather elderly) Dell Venue 11 Pro tablet. It certainly lives up to its name - I realise it's still WIP, but it's much better than the default Gnome OSK (which is itself quite nice)! Just the fact of being able to make the keyboard bigger on landscape mode is an absolute godsend; my relatively fat fingers say "Thank you!" :)

Just a couple of things I wanted to say.

Firstly - Nick: this extension is awesome! I wanted to encourage you to keep up the great work; happy to help in any way I can (testing etc). Sadly I don't have any coding experience to speak of, otherwise I'd help with that too.

Check out Onboard, if I recall correctly it was draggable.

That is correct, however Onboard is pretty clunky on Gnome, particularly on Wayland. Which is a pity really, as in itself it's a fantastic on-screen keyboard. Input from Onboard does not come up in a Wayland session.

[christophhagemann]

Hi,

thanks a lot for your work. I installed gnome_43_rewrite. While being very helpful, I just wanted to notify you, that ctrl-click (e.g. to select multiple mails in evolution) doesn't work currently. It did in Gnome 42. Would be very nice if this functionality could come back. Let me know if I can help you with testing.

Thanks a lot!

[sughero]

I don't know what's wrong with what I am doing:

• I clone the 43 branch
  cd ~/.local/share/gnome-shell/extensions
git clone --branch gnome_43_rewrite https://github.com/nick-shmyrev/improved-osk-gnome-ext.git ~/.local/share/gnome-shell/extensions/improvedosk@nick-shmyrev.dev

• I build the package running
  package-extension.sh

• I install it with
  gnome-extensions install improvedosk@nick-shmyrev.dev.shell-extension.zip

and the only thing I get is the keyboard icon on the traybar.
Nothing works here :-(
Maybe because IT layout is missing? I tried myself to understand what file to edit without success,
Maybe I need you to write the italian layout as you did with other languages?
Thanks

[yfreund]

@sughero aside from the IT layout not being fully supported yet, as you correctly mentioned, you should also avoid cloning the 43 branch into ~/.local/share/gnome-shell/extensions/improvedosk@nick-shmyrev.dev, since this is the path where the extension will be installed to in the third step. Please use a different folder on your system for these files, which could be deleted after the extension is installed (e.g. mkdir ~/git && cd ~/git && git clone --branch gnome_43_rewrite https://github.com/nick-shmyrev/improved-osk-gnome-ext.git)

[sughero]

@yfreund thanks, you're right!
syncing in the same directory where extension is installed is not a good idea and I need to delete that folder to let the extension install properly.
I installed both US and IT layouts so that when I need CTRL, ALT or other keys I switch keyboard.
I noticed that, differently from previous versions, I need to enable the OSK in Gnome accessibility settings even though I am in tablet mode.

[yfreund]

@sughero I'm glad you've found a solution that works for you, at least for now.

Which device are you using the extension on?

• With a tablet/2-in-1 detachable device (where the keyboard completely detaches from the screen), the accessibility setting should not be required, though I have no device to test with.

• With a flip device (where the keyboard does not detach, but turns ~360 degrees behind the screen), because the physical keyboard does not get disconnected, you'd need to ensure that the OSK gets triggered regardless, which could be done in 3 different ways:

1. Enabling the OSK accessibility setting, which also opens the OSK on mouse/touch-pad click.
2. Using the Force Show OSK extension, which only opens the keyboard on touch input (preferred IMO), but is no longer maintained.
3. Trying my fork of this extension (pull request pending), where I added the functionality from the second option as an additional toggle (after installation, open the extension settings, and turn on Force touch-input). If you decide to go with this option, please use the same process/branch as with this extension, and report back on whether this solved your problem (I only tested this on my old Lenovo Yoga, which I use as a secondary device. My main device doesn't have a touch screen and runs KDE).

[sughero]

@sughero I'm glad you've found a solution that works for you, at least for now.

Well, that's far from being a "solution", let's say it's a work around

Which device are you using the extension on?

I am using it on a Microsoft Surface book 2. Everything was perfect on gnome 42 and accessibility set on on was not required

• With a tablet/2-in-1 detachable device (where the keyboard completely detaches from the screen), the accessibility setting should not be required, though I have no device to test with.
• With a flip device (where the keyboard does not detach, but turns ~360 degrees behind the screen), because the physical keyboard does not get disconnected, you'd need to ensure that the OSK gets triggered regardless, which could be done in 3 different ways:

You can completely detach screen and use it as a tablet but you can also connect screen reverse-wise using keyboard as a "standing support"

1. Enabling the OSK accessibility setting, which also opens the OSK on mouse/touch-pad click.
2. Using the Force Show OSK extension, which only opens the keyboard on touch input (preferred IMO), but is no longer maintained.
3. Trying my fork of this extension (pull request pending), where I added the functionality from the second option as an additional toggle (after installation, open the extension settings, and turn on Force touch-input). If you decide to go with this option, please use the same process/branch as with this extension, and report back on whether this solved your problem (I only tested this on my old Lenovo Yoga, which I use as a secondary device. My main device doesn't have a touch screen and runs KDE).

I will definitevely try your code and let you know what works, thanks again.

[yfreund]

Well, that's far from being a "solution", let's say it's a work around

Fair enough.

I am using it on a Microsoft Surface book 2. Everything was perfect on gnome 42 and accessibility set on on was not required
You can completely detach screen and use it as a tablet but you can also connect screen reverse-wise using keyboard as a "standing support"

I am familiar with the Surface Book series, though I've never owned one. I am a bit surprised that this extension isn't working for you "as is" when used without the keyboard attached, but unfortunately there are constant changes to upstream GNOME (specifically mutter) which change the default OSK behavior, pleasing one group of users while disappointing another (google at your own peril, especially if you enjoy reading very angry comments).
This isn't something that could be fully dealt with within the scope of this extension, but perhaps a better workaround could be provided (hence my contribution).

I should note that the OSK behavior is also inconsistent between Xorg and Wayland (a bit more on that later), so switching between them is another thing you could experiment with. At the very least, knowing which one you're using would be helpful.

I will definitevely try your code and let you know what works, thanks again.

That's great - the more testing to my alterations, the better. I really hope this works better for yo than the accessibility setting, as I was having a similar issue with my setup before I ran into the additional extension, and prompted me to add the code to this extension once it was no longer maintained.
To be frank, I don't need the extra keys as much as I need a reliable way to open the OSK, and having the keyboard icon in conjunction with the ability to force the OSK is the main reason I use this extension, since there are still issues automatically opening the OSK in some application under Wayland (specifically, and most annoyingly, web-browsers), and falling back to Xorg isn't an option for me due to other bugs, which are less likely to be fixed.
The extra keys are a very nice plus, though. 😉

If you don't mind, I'd suggest we move this discussion to the pull request once you give my fork a try - it feels like we're going a bit off-topic in our discussion.

[nick-shmyrev]

I installed both US and IT layouts so that when I need CTRL, ALT or other keys I switch keyboard. I noticed that, differently from previous versions, I need to enable the OSK in Gnome accessibility settings even though I am in tablet mode.

@sughero I've updated IT layout to also include extended keys, so at least you won't have to change layouts to use CTRL and ALT anymore.
Interesting, I might have to figure a way to explicitly enable Gnome's default OSK when this extension is enabled.

[sughero]

I installed both US and IT layouts so that when I need CTRL, ALT or other keys I switch keyboard. I noticed that, differently from previous versions, I need to enable the OSK in Gnome accessibility settings even though I am in tablet mode.

@sughero I've updated IT layout to also include extended keys, so at least you won't have to change layouts to use CTRL and ALT anymore. Interesting, I might have to figure a way to explicitly enable Gnome's default OSK when this extension is enabled.

thanks :)

[nick-shmyrev]

A version supporting Gnome 43 is now available at https://extensions.gnome.org/extension/4413/improved-osk/.

Currently, the following layouts have extended keys: CH+FR, CH, DE, ES, FR, IT, RU, UA, US. If the layout you're using does not have the extended keys, let me know, and I'll add them. Or, feel free to modify it yourself (see /src/data/osk-layouts dir) and make a PR.

It also includes a fix for Numbers layer not properly latching.

[LeSnake04]

Can most of the rewrite be reused for the gnome 44 release. Gnome 44 comes out on the 22.

It would be great If I can the latest fedora when it comes out.

According to this, The planned release date of fedora 38 is 25.04.2023.

[JamiKettunen]

https://os.gnome.org/ is already available and should allow testing the extension on 44 right away