[WIP] On-screen keyboard support #892

dcz-purism · 2018-04-23T18:07:53Z

This pull request combines work on:

input-method protocol #856
text-input protocol #776
rootston glue

Its purpose is to develop these in sync, and to remove the need to juggle different long-running feature branches as different parts are being developed. This has proven to take up effort and lead to awkward situations, e.g. many of the rebased rootston commits were developed against an earlier version of wlr_input_method.h and are currently unbuildable.

One interesting thing is the addition of code duplicating some functionality of layer surfaces (panels, unfocusable windows) in rootston views, in order to support input-method-panel.

ddevault · 2018-04-23T18:24:34Z

include/rootston/osk.c

+	panel->panel = wl_global_create(display, &zwp_input_panel_v1_interface, 1,
+		panel, input_panel_bind);
+	return panel;
+}


This should not be named osk.c, input-method supports more than OSKs

agx · 2018-04-23T18:26:17Z

Now that we have layer-surfaces can't OSK use just that?

dcz-purism · 2018-04-23T18:33:32Z

@agx: I wanted to have a reasonably complete implementation of the existing protocol first, especially because all existing software, namely weston-keyboard uses it. The plan is to update the protocol and remove the panel in its current form in favor of layer-surfaces in the future.

agx · 2018-04-23T18:35:46Z

@dcz-purism with existing you mean weston-keyboard using zwp_input_panel_surface_v1 ?
Looking at the protocol the panel part looks pretty much bolted onto generic input_method stuff.

ddevault

I'll come back around for another round of review when input-method is disentangled from rootston views

ddevault · 2018-04-23T18:36:06Z

include/rootston/osk.c

+
+struct roots_input_panel *roots_input_panel_create(struct wl_display *display) {
+	struct roots_input_panel *panel = calloc(1,
+											 sizeof(struct roots_input_panel));


ddevault · 2018-04-23T18:36:06Z

include/rootston/osk.c

+
+#include "input-method-unstable-v1-protocol.h"
+
+static void noop() {}


Add newlines and such and something to the effect of /* This space deliberately left blank */

ddevault · 2018-04-23T18:36:06Z

include/rootston/osk.h

+#include "rootston/desktop.h"
+
+struct roots_input_panel {
+	struct roots_desktop *desktop; // FIXME - remove when views come


If you expect this TODO to get merged, can you elaborate a bit? If you will resolve prior to merge no worries

ddevault · 2018-04-23T18:36:06Z

include/rootston/text_input.h

+    struct wlr_text_input *input;
+	struct wlr_input_method_context *context;
+
+    struct wl_list link;


Wrong indent here and sprinkled throughout this struct

ddevault · 2018-04-23T18:36:06Z

include/rootston/text_input.h

+	struct wlr_input_method_context *context;
+
+    struct wl_list link;
+	struct wl_listener text_input_enable; // uint32_t show_input_panel


Put the docs near the signal, not the listener

ddevault · 2018-04-23T18:36:06Z

include/wlr/types/wlr_input_method.h

+	struct wl_resource *resource;
+	struct {
+		struct wl_signal new_context; // struct wlr_input_method_context*
+		// TODO: create roots_keyboard (seat_add_keyboard) on this; modify roots_keyboard to support backend-less


I think this should be more along the lines of wlr_input_method creating a wlr_keyboard which roots_keyboard should just be able to pick up without changes. Maybe some silly changes to automatically associate it with the relevant seat if wlr_keyboard_is_input_method returns true.

I think this is what I did but forgot to update the comment. Thanks for the seat input.

ddevault · 2018-04-23T18:36:06Z

include/wlr/types/wlr_input_method.h

+
+void wlr_input_method_send_deactivate(struct wlr_input_method *input_method,
+	struct wlr_input_method_context *context);
+#endif


minor nit: extra newline above the #endif

ddevault · 2018-04-23T18:36:06Z

rootston/cursor.c

-			roots_seat_set_focus(seat, view);
+			if (!view || !view->special || view->features.focusable) {
+				roots_seat_set_focus(seat, view);
+            }


Weird indent

ddevault · 2018-04-23T18:36:06Z

rootston/desktop.c

+	view->x = x;
+	view->y = y;
+	view_damage_whole(view);
+}


Yeah all of this stuff can go if you don't try to hack it into views.

I'll be slow to address it, but it will get addressed eventually. I want to keep the whole stack working at any time, so I'll get rid of the current panel implementation once there's a keyboard that doesn't use it. It probably means implementing it on my own, so it will take time.

ddevault · 2018-04-23T18:36:06Z

rootston/input.c

@@ -43,7 +43,7 @@ struct roots_seat *input_get_seat(struct roots_input *input, char *name) {
 	return seat;
 }

-static void handle_new_input(struct wl_listener *listener, void *data) {
+void handle_new_input(struct wl_listener *listener, void *data) {


dcz-purism · 2018-04-23T18:50:11Z

@agx: yes, it's using input-method-panel-surface, which is definitely bolted, but it covers some things that layer-shell doesn't:

<SirCmpwn> input-method popups are shown by the compositor near the cursor of the client with keyboard focus
<SirCmpwn> xdg-shell and layer-shell popups are shown where the client asks them to be

and also the function of keyboards that they may need to be visible even when the lock screen is active.

Shugyousha

Mostly superficial things

Shugyousha · 2018-04-23T19:54:14Z

rootston/output.c

@@ -333,6 +336,9 @@ static bool has_standalone_surface(struct roots_view *view) {
 	case ROOTS_XWAYLAND_VIEW:
 		return wl_list_empty(&view->xwayland_surface->children);
 #endif
+	case ROOTS_INPUT_PANEL_VIEW:
+		printf("Stub: has_standalone_surface on OSK_VIEV returns true\n");
+		return true;


The printed text should be updated to correspond to the constant. There is also a typo in the current text (s/OSK_VIEV/OSK_VIEW/).

What's a standalone surface here?

It's used for fullscreen to know if the view has only one surface.

Shugyousha · 2018-04-23T19:54:14Z

rootston/seat.c

-		first_seat_view->link.next, next_seat_view, link);
+	// Focus the next view which is focusable
+	struct roots_seat_view *next_seat_view = first_seat_view;;
+	do {


Shugyousha · 2018-04-23T19:54:14Z

types/wlr_text_input.c

+	}
+	if (text_input->pending.surrounding.text) {
+		free(text_input->pending.surrounding.text);
+	}


Since free(NULL) is a noop, we don't need these two "if" statements here.

Shugyousha · 2018-04-23T19:54:14Z

types/wlr_text_input.c

+	if (text_input->pending.surrounding.text) {
+		free(text_input->pending.surrounding.text);
+	}
+	text_input->pending.surrounding.text = strdup(text);


Shugyousha · 2018-04-23T19:54:14Z

types/wlr_text_input.c

+	if (text_input->current.surrounding.text) {
+		free(text_input->pending.surrounding.text);
+	}
+	text_input->current = text_input->pending;


Shugyousha · 2018-04-23T19:54:14Z

types/wlr_text_input.c

+struct wlr_text_input_manager *wlr_text_input_manager_create(
+		struct wl_display *wl_display) {
+	struct wlr_text_input_manager *manager =
+		calloc(1, sizeof(struct wlr_text_input_manager));


Add out of memory check

agx · 2018-04-24T04:25:32Z

@dcz-purism my main concern is growing a third way to handle popups and overlay surfaces. layer-shell already has overlap with views and it would be nice if either one could be reused.

On-screen keyboard will appear whenever an application using the text-input protocol wants to edit text. Rootston will pass text composition events from the keyboard to the application, and will accept keysym events from the keyboard, and create a virtual keyboard device in order to pass them to the application. The keyboard will restrict the space available for maximized applications. rootston/text_input is the relay of messages between wlr_text_input and wlr_input_method. To test with older GTK installed: ``` cd gtk PKG_CONFIG_PATH=.../wayland_protocols_prefix/share/pkgconfig ./configure --enable-wayland-backend --enable-x11-backend --prefix=.../gtk-install make && make install chrpath -d .../gtk-install/lib/gtk-3.0/3.0.0/immodules/im-wayland.so WAYLAND_DEBUG=1 GDK_BACKEND=wayland GTK_IM_MODULE=wayland GTK_IM_MODULE_FILE=.../gtk-install/lib/gtk-3.0/3.0.0/immodules.cache yad --entry ```

agx · 2018-04-26T20:46:49Z

examples/text-input.c

+
+/**
+ * Usage: idle-inhibit
+ * Creates a xdg-toplevel using the idle-inhibit protocol.


It's rather text-input and does s.th. different?

Thanks. I didn't realize this push would send out notifications.

agx · 2018-04-26T20:47:52Z

rootston/input_method.c

+	*im_pid = fork();
+	if (*im_pid == 0) {
+		sleep(1);
+		execl("/bin/sh", "/bin/sh", "-c", im_cmd, (void *)NULL);


Why does it need a shell?

The -E option pipes the command through a shell, so in order to maintain consistency, I made it the same. Personally, I don't see the need for a shell in either case, but maybe I'm missing something.

agx · 2018-04-27T09:17:28Z

On Thu, Apr 26, 2018 at 09:22:57PM +0000, dcz-purism wrote: dcz-purism commented on this pull request. > @@ -134,3 +144,55 @@ void handle_input_method_context_destroy(struct wl_listener *listener, } update_maximized_views(desktop); } + +static void spawn_im_process() { + *im_pid = fork(); + if (*im_pid == 0) { + sleep(1); + execl("/bin/sh", "/bin/sh", "-c", im_cmd, (void *)NULL); The -E command accepts a shell, so in order to maintain consistency, I made it the same. Personally, I don't see the need for a shell in either case, but maybe I'm missing something.

Shell uses $PATH but I'd rather not rely on that for the OSK.

agx · 2018-04-27T16:32:19Z

include/rootston/text_input.h

+
+	bool enabled; // client requested state
+	bool sent_enter; // server requested state
+	bool active; // active focus


nitpick: Indentation is off here

ddevault · 2018-07-29T00:51:44Z

How're are things in this department going on your end? Should we expect an update to this soon?

dcz-purism · 2018-07-29T06:49:37Z

Yes indeed! Currently I'm busy upstreaming text-input to GTK, so after the merge window closes and I'm free to do other things again, I'll push my changes to get some formal feedback.

ddevault · 2018-07-29T14:50:46Z

Sweet, keep up the good work!

dcz-purism · 2018-08-06T16:49:24Z

The first replacement of the old input method was just posted to gather feedback today:

https://lists.freedesktop.org/archives/wayland-devel/2018-August/039255.html

dcz-purism · 2018-08-29T16:54:05Z

I rewrote this code to updated protocols and submitted it as a separate (out of forgetfulness) PR here.

dcz-purism added 10 commits Mar 29, 2018

Add support for zwp_text_input_unstable_v3

Loading status checks…

117a749

Make focusability optional for views

c0cd19d

Allow views to stick to display frame

ff01c3b

Make maximized views respect panels

379708a

Prevent switching to unfocusable views with keyboard shortcut

6c87fc4

Fix crash on desktop click

29e0e99

Fix resource tracking and destructoin issues

80aae6c

Add input method protocol

f00fa27

Merge branch 'text_input_v3' into input_method_1

c1d0051

Merge branch 'views' into input_method_2

f262e96

This was referenced Apr 23, 2018

input-method protocol #856

Closed

WIP: Add partial support for zwp_text_input_unstable_v3 #776

Closed

ddevault reviewed Apr 23, 2018

View changes

dcz-purism force-pushed the dcz-purism:osk branch from b3ba5cd to 9916665 Apr 23, 2018

ddevault requested changes Apr 23, 2018

View changes

dcz-purism force-pushed the dcz-purism:osk branch from 9916665 to 57911f2 Apr 23, 2018

dcz-purism force-pushed the dcz-purism:osk branch 2 times, most recently from 4391ad1 to fa21d66 Apr 23, 2018

Shugyousha reviewed Apr 23, 2018

View changes

dcz-purism force-pushed the dcz-purism:osk branch from fa21d66 to f70082c Apr 24, 2018

dcz-purism added 4 commits Apr 24, 2018

Add cursor position to input_method

Loading status checks…

7925bf4

Prevent multiple input method clients

bc73647

Fix crash when keyboard drops while deactivating

0abdf95

Use single input-method instance

33cbf0e

dcz-purism added 2 commits Apr 26, 2018

Fix handling disordered text-input and input-method events

f97a96d

Spawn and restart keyboard

Loading status checks…

f07737c

agx reviewed Apr 26, 2018

View changes

Fix comment on text-input example

Loading status checks…

680949d

agx changed the title ~~WIP: On-screen keyboard support~~ [WIP] On-screen keyboard support Apr 27, 2018

agx reviewed Apr 27, 2018

View changes

include/rootston/text_input.h

bool enabled; // client requested state

bool sent_enter; // server requested state

bool active; // active focus

This comment has been minimized.

Sign in to view

agx Apr 27, 2018
Contributor

nitpick: Indentation is off here

ddevault mentioned this pull request May 27, 2018

Support virtual keyboard protocol #999

Merged

dcz-purism closed this Aug 29, 2018


		#include "input-method-unstable-v1-protocol.h"

		static void noop() {}

swaywm / wlroots

[WIP] On-screen keyboard support #892

[WIP] On-screen keyboard support #892

Conversation

dcz-purism commented Apr 23, 2018

This comment has been minimized.

ddevault Apr 23, 2018 Member

agx commented Apr 23, 2018

dcz-purism commented Apr 23, 2018

agx commented Apr 23, 2018 • edited

ddevault left a comment

This comment has been minimized.

ddevault Apr 23, 2018 Member

This comment has been minimized.

ddevault Apr 23, 2018 Member

This comment has been minimized.

ddevault Apr 23, 2018 Member

This comment has been minimized.

ddevault Apr 23, 2018 Member

This comment has been minimized.

ddevault Apr 23, 2018 Member

This comment has been minimized.

ddevault Apr 23, 2018 Member

This comment has been minimized.

dcz-purism Apr 23, 2018 Author Contributor

This comment has been minimized.

ddevault Apr 23, 2018 Member

This comment has been minimized.

ddevault Apr 23, 2018 Member

This comment has been minimized.

ddevault Apr 23, 2018 Member

This comment has been minimized.

dcz-purism Apr 23, 2018 Author Contributor

This comment has been minimized.

ddevault Apr 23, 2018 Member

dcz-purism commented Apr 23, 2018

Shugyousha left a comment

This comment has been minimized.

Shugyousha Apr 23, 2018 Contributor

This comment has been minimized.

dcz-purism Apr 24, 2018 Author Contributor

This comment has been minimized.

emersion Apr 24, 2018 Member

This comment has been minimized.

Shugyousha Apr 23, 2018 Contributor

This comment has been minimized.

Shugyousha Apr 23, 2018 Contributor

This comment has been minimized.

Shugyousha Apr 23, 2018 Contributor

This comment has been minimized.

Shugyousha Apr 23, 2018 Contributor

This comment has been minimized.

Shugyousha Apr 23, 2018 Contributor

agx commented Apr 24, 2018

This comment has been minimized.

agx Apr 26, 2018 Contributor

This comment has been minimized.

dcz-purism Apr 26, 2018 Author Contributor

This comment has been minimized.

agx Apr 26, 2018 Contributor

This comment has been minimized.

dcz-purism Apr 26, 2018 • edited Author Contributor

agx commented Apr 27, 2018

This comment has been minimized.

agx Apr 27, 2018 Contributor

ddevault commented Jul 29, 2018

dcz-purism commented Jul 29, 2018

ddevault commented Jul 29, 2018

dcz-purism commented Aug 6, 2018

dcz-purism commented Aug 29, 2018

ddevault Apr 23, 2018
Member

agx commented Apr 23, 2018 •

edited

ddevault Apr 23, 2018
Member

ddevault Apr 23, 2018
Member

ddevault Apr 23, 2018
Member

ddevault Apr 23, 2018
Member

ddevault Apr 23, 2018
Member

ddevault Apr 23, 2018
Member

dcz-purism Apr 23, 2018
Author Contributor

ddevault Apr 23, 2018
Member

ddevault Apr 23, 2018
Member

ddevault Apr 23, 2018
Member

dcz-purism Apr 23, 2018
Author Contributor

ddevault Apr 23, 2018
Member

Shugyousha Apr 23, 2018
Contributor

dcz-purism Apr 24, 2018
Author Contributor

emersion Apr 24, 2018
Member

Shugyousha Apr 23, 2018
Contributor

Shugyousha Apr 23, 2018
Contributor

Shugyousha Apr 23, 2018
Contributor

Shugyousha Apr 23, 2018
Contributor

Shugyousha Apr 23, 2018
Contributor

agx Apr 26, 2018
Contributor

dcz-purism Apr 26, 2018
Author Contributor

agx Apr 26, 2018
Contributor

dcz-purism Apr 26, 2018 •

edited

Author Contributor

agx Apr 27, 2018
Contributor