Support virtual keyboard protocol #999

dcz-purism · 2018-05-24T18:21:09Z

This change is to support the recently proposed virtual-keyboard-v1 protocol.

Because virtual keyboards are privileged, the commits setting up an authorization system - meant to be general for all input methods - are separate.

The main commit provides support in wlroots as well as rootston. The keyboard does not attempt to synchronize its modifier status with the rest of the system, and code for having multiple instances running at the same time has not been extensively tested, however it's rather simple and wasn't failing either.

dcz-purism · 2018-05-24T18:22:59Z

The test client is at https://code.puri.sm/Librem5/weston/src/virtual_keyboard - please use the weston-keyboard program:

./configure --enable-clients
./weston-keyboard

agx · 2018-05-25T10:21:28Z

rootston/config.c

@@ -19,14 +19,15 @@

 static void usage(const char *name, int ret) {
 	fprintf(stderr,
-		"usage: %s [-C <FILE>] [-E <COMMAND>]\n"
+		"usage: %s [-C <FILE>] [-E <COMMAND>] [-E <EXEC>]\n"
 		"\n"


Duplicat -E, should be -K ?

agx · 2018-05-25T10:21:36Z

include/rootston/input_method.h

@@ -0,0 +1,11 @@
+//#include <wlr/types/wlr_input_method.h>


agx · 2018-05-25T10:32:06Z

rootston/input_method.c

+static void spawn_im_process() {
+	*im_pid = fork();
+	if (*im_pid == 0) {
+		sleep(1);


sleep for what? If a sleep is needed for any reason it shoud likely be polling for something to be not racy

Initially, I added the sleep there because when spawn_im_process is first called, the server is not completely initialized, causing an immediate crash. Later on, I realized this would be useful to rate-limit the spawns.

-1

Rate-limiting shouldn't be done via sleep, and rootston shouldn't rate-limit. Child processes should only be spawned when the server is completely initialized.

How to wait for full initialization?

When I say rate-limit, I mean the situation where a faulty process will crash very soon after spawning and get restarted. I think it the infinite spawning loop should be prevented from excessively consuming resources, although not necessarily using sleep.

You can either split your setup and call the spawn part directly before going into the main loop, or add an idle event at setup time to start the client from the mainloop.

agx · 2018-05-25T11:37:10Z

include/wlr/types/wlr_virtual_keyboard.h

+    void *credentials_data;
+    struct {
+        struct wl_signal new_virtual_keyboard; // struct wlr_virtual_keyboard*
+    };


Other places use

struct { } events;

Thanks, I knew something was off here.

agx · 2018-05-25T11:37:33Z

include/wlr/types/wlr_virtual_keyboard.h

+    struct wlr_seat *seat;
+    struct {
+		struct wl_signal destroy; // struct wlr_virtual_keyboard*
+    };


agx

See comments. Looks good to me otherwise.

agx · 2018-05-25T11:41:00Z

rootston/virtual_keyboard.c

+	roots_seat_add_device(seat, &keyboard->input_device);
+	wl_signal_add(&keyboard->destroy, &desktop->virtual_keyboard_destroy);
+	desktop->virtual_keyboard_destroy.notify = handle_virtual_keyboard_destroy;
+}


Why register it when handle_virtual_keyboard_destroy is a noop?

That's vestigial from an earlier version - removing.

agx · 2018-05-25T11:41:59Z

types/wlr_virtual_keyboard.c

+	return wl_resource_get_user_data(resource);
+}
+
+static void	virtual_keyboard_keymap(struct wl_client *client,


style nitpick: Extra space

This was actually a tab :)

ddevault · 2018-05-25T12:29:51Z

I don't like rootston having special support for spawing the IM and keeping up with its process. I also don't like that the wlroots implementation has a special function to check for authoriziation to the protocol. This isn't how we're going to secure protocols.

For now, just let any client use this protocol and yank out all of the -K and process handling stuff. If you want it to restart automatically just use a shell script that runs it in a loop.

ddevault

Overall this looks great, just gotta yank out the process management stuff

ddevault · 2018-05-25T12:30:08Z

rootston/input_method.c

+	gid_t gid;
+	wl_client_get_credentials(client, &pid, &uid, &gid);
+	return pid == allowed_pid;
+}


Basically this whole file should go away

ddevault · 2018-05-25T12:30:18Z

rootston/config.c

 		switch (c) {
 		case 'C':
 			config->config_path = strdup(optarg);
 			break;
 		case 'E':
 			config->startup_cmd = strdup(optarg);
 			break;
+		case 'K':
+			config->im_cmd = strdup(optarg);
+			break;


ddevault · 2018-05-25T12:30:39Z

include/wlr/types/wlr_virtual_keyboard.h

+struct wlr_virtual_keyboard_manager {
+	struct wl_global *global;
+	struct wl_resource *resource;
+	wlr_virtual_keyboard_check_credentials_func_t check_credentials;


agx · 2018-05-25T12:43:22Z

For now, just let any client use this protocol and yank out all of the -K and process handling stuff. If you
want it to restart automatically just use a shell script that runs it in a loop.

I think that's fine, we can carry this as a patch on top of rootston for the librem5 and rather work on the general logic to authorize clients since layer-shell, etc. are also affected.

Shugyousha · 2018-05-25T19:31:18Z

types/wlr_virtual_keyboard.c

+	xkb_context_unref(context);
+context_fail:
+	wl_client_post_no_memory(client);
+	return;


This return is not needed.

Shugyousha · 2018-05-25T19:37:22Z

types/wlr_virtual_keyboard.c

+	}
+
+	struct wlr_keyboard* keyboard = calloc(1, sizeof(struct wlr_keyboard));
+	if (keyboard == NULL) {


Should be

if (!keyboard) {

for consistency.

Nah, both are fine

The style is mixed in the same function though... your call.

dcz-purism · 2018-05-25T20:47:28Z

Yanked and fixed.

include/rootston/virtual_keyboard.h

@@ -0,0 +1,3 @@
+#include <wayland-server-core.h>


ddevault · 2018-05-27T13:31:14Z

This is looking really good. Just had one minor issue, and I still need to test it on my touchscreen device.

emersion · 2018-05-27T13:36:58Z

types/wlr_virtual_keyboard.c

+	}
+
+	wl_resource_set_implementation(resource, &manager_impl, manager, NULL);
+	manager->resource = resource;


If two clients bind to the global, manager->resource will be overwritten. You should use a list and wl_resource_get_link instead.

emersion · 2018-05-27T13:37:16Z

types/wlr_virtual_keyboard.c

+
+static void virtual_keyboard_manager_bind(struct wl_client *client, void *data,
+		uint32_t version, uint32_t id) {
+	assert(client);


This assert is unnecessary.

emersion · 2018-05-27T13:38:07Z

types/wlr_virtual_keyboard.c

+	virtual_keyboard->resource = keyboard_resource;
+	virtual_keyboard->seat = seat_client->seat;
+	wl_signal_init(&virtual_keyboard->events.destroy);
+	wlr_signal_emit_safe(&manager->events.new_virtual_keyboard, virtual_keyboard);


We should have a list of all virtual keyboards in the manager.

Let the users iterate on the list and properly cleanup when destroying the manager.

What's the advantage of letting manager do the cleanup over letting keyboards clean up themselves?
As I understand, there's not much utility in destroying the manager while objects are bound. In addition to that, the manager is acting as a factory rather than a collection, so there's no harm in leaving keyboards alive after destroying it.
In addition to that, having a list of active keyboards in the manager will need introducing another object to remember which keyboards are held by which user, making the whole thing a doubly-nested structure:

manager global

manager binds

keyboards

The way it is now seems just as functional and can stay simple.

Giving it a bit more thought, a binds list is not implied by a keyboard list, but I stand by my comment that it's not necessary.

No need for complicated list. The global, a list of resources and a list of keyboards in the manager is enough.

The list is useful to let compositors inspect it without having to maintain their own list.

emersion · 2018-05-27T13:38:36Z

types/wlr_virtual_keyboard.c

+
+	wlr_input_device_init(&virtual_keyboard->input_device,
+		WLR_INPUT_DEVICE_KEYBOARD, &input_device_impl, "virtual keyboard",
+		0xabcd, 0x1234);


Hmm, better to use 0 instead of these constants :)

emersion · 2018-05-27T13:39:56Z

types/wlr_virtual_keyboard.c

+
+static void virtual_keyboard_destroy(struct wl_client *client,
+		struct wl_resource *resource) {
+	virtual_keyboard_destroy_resource(resource);


This will leak the resource. Also, this allows the client to call destroy twice and make the compositor crash.

Should just call wl_resource_destroy(resource).

ddevault · 2018-05-27T13:40:05Z

How does this work relate to #892?

emersion · 2018-05-27T13:42:21Z

include/wlr/types/wlr_virtual_keyboard.h

+};
+
+struct wlr_virtual_keyboard_manager *wlr_virtual_keyboard_manager_create(
+	struct wl_display *display);


The compositor should have a way to destroy the manager here

emersion · 2018-05-27T13:42:53Z

include/wlr/types/wlr_virtual_keyboard.h

@@ -0,0 +1,28 @@
+#ifndef WLR_TYPES_WLR_VIRTUAL_KEYBOARD_H
+#define WLR_TYPES_WLR_VIRTUAL_KEYBOARD_H


All of the symbols should be include the unstable version too.

What do you mean?

This protocol is virtual_keyboard_unstable_v1. All symbols should be chosen so that there's no conflict when a new unstable version is published (e.g. virtual_keyboard_unstable_v2) or when the stable version is adopted (virtual_keyboard).

Does the file name need to change?

emersion

Looking good!

ddevault · 2018-05-27T13:52:45Z

Tested on touch screen 👍

dcz-purism · 2018-05-27T14:14:16Z

@SirCmpwn this provides some features which will be cut out from input-method protocol, so it's part of the work needed to get #892 done.

dcz-purism · 2018-05-27T19:55:16Z

Added list of manager resources and keyboards, renamed public interface to include _unstable_v1.

emersion · 2018-05-27T20:36:06Z

types/wlr_virtual_keyboard_unstable_v1.c

+	if (!context) {
+		goto context_fail;
+	}
+	void *data = mmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);


In case of success, when is this unmapped?

emersion · 2018-05-27T20:37:56Z

include/wlr/types/wlr_virtual_keyboard_unstable_v1.h

+};
+
+struct wlr_virtual_keyboard_manager_unstable_v1*
+	wlr_virtual_keyboard_manager_unstable_v1_create(


We currently name unstable protocols without "unstable" in the symbols, just with _v1.

We should probably change that to add a z prefix or something, but that's unrelated to this PR.

dcz-purism · 2018-05-29T09:29:34Z

Removed "unstable" and fixed unmapping.

emersion

LGTM

emersion · 2018-05-29T17:22:50Z

Thanks!

agx reviewed May 25, 2018

View reviewed changes

include/rootston/input_method.h Outdated

@@ -0,0 +1,11 @@

//#include <wlr/types/wlr_input_method.h>

Copy link

Contributor

agx May 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead code?

agx reviewed May 25, 2018

View reviewed changes

dcz-purism force-pushed the virtual-keyboard branch 2 times, most recently from 29ed980 to e9e537d Compare May 25, 2018 11:43

agx reviewed May 25, 2018

View reviewed changes

dcz-purism force-pushed the virtual-keyboard branch from e9e537d to 510a264 Compare May 25, 2018 11:52

ddevault suggested changes May 25, 2018

View reviewed changes

dcz-purism force-pushed the virtual-keyboard branch from 510a264 to a36c234 Compare May 25, 2018 13:35

Shugyousha reviewed May 25, 2018

View reviewed changes

dcz-purism force-pushed the virtual-keyboard branch from a36c234 to 8c64f16 Compare May 25, 2018 20:32

ddevault suggested changes May 27, 2018

View reviewed changes

include/rootston/virtual_keyboard.h

@@ -0,0 +1,3 @@

#include <wayland-server-core.h>

This comment was marked as resolved.

Sign in to view

emersion reviewed May 27, 2018

View reviewed changes

emersion suggested changes May 27, 2018

View reviewed changes

dcz-purism force-pushed the virtual-keyboard branch from 8c64f16 to 9196d22 Compare May 27, 2018 14:07

dcz-purism force-pushed the virtual-keyboard branch from 9196d22 to 220da12 Compare May 27, 2018 19:53

emersion reviewed May 27, 2018

View reviewed changes

virtual-keyboard: add support for the virtual-keyboard-v1 protocol

5334ee8

dcz-purism force-pushed the virtual-keyboard branch from 220da12 to 5334ee8 Compare May 28, 2018 08:06

ddevault approved these changes May 29, 2018

View reviewed changes

emersion approved these changes May 29, 2018

View reviewed changes

emersion merged commit e1f5653 into swaywm:master May 29, 2018

		@@ -0,0 +1,28 @@
		#ifndef WLR_TYPES_WLR_VIRTUAL_KEYBOARD_H
		#define WLR_TYPES_WLR_VIRTUAL_KEYBOARD_H

Support virtual keyboard protocol #999

Support virtual keyboard protocol #999

Conversation

dcz-purism commented May 24, 2018

dcz-purism commented May 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ddevault commented May 25, 2018

ddevault left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agx commented May 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dcz-purism commented May 25, 2018

This comment was marked as resolved.

ddevault commented May 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ddevault commented May 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emersion left a comment

Choose a reason for hiding this comment

ddevault commented May 27, 2018

dcz-purism commented May 27, 2018

dcz-purism commented May 27, 2018

Choose a reason for hiding this comment

emersion May 27, 2018 • edited Loading

Choose a reason for hiding this comment

dcz-purism commented May 29, 2018

emersion left a comment

Choose a reason for hiding this comment

emersion commented May 29, 2018

emersion May 27, 2018 •

edited

Loading