Can't interact with the application when using RDP on Weston

[topolittle]

When connecting to the device remotely using the remote desktop protocol, it's possible to launch the Flutter application using the embedder, but it's not possible to interact with it once it's launched.

Environment

Hardware: i.MX8M-Plus EVK (Eval board)
OS version: NXP-5.15.71_2.2.0 (Yocto/Kirkstone)
Window manager: Weston 10.0.1
Virtual keyboard: Maliit 2.3.0
RDP server: Built-in Weston using rdp-backend.so
RDP client: Remmina 1.4.30 on a Fedora 38 machine
Embedder version (flutter-client): 3dedfc0
Backend: Wayland

Step to reproduce:

1. Enable RDP in Weston, see weston-ini man page and weston-rdp man page
2. Restart the Weston service
3. Connect to the device over RDP
4. Launch a terminal (weston-terminal)
5. Launch the Flutter app using the embedder: /home/root/flutter-gallery-elinux/gallery --bundle=/home/root/flutter-gallery-elinux --onscreen-keyboard --window-decoration --width=450 --height=650

Expected result

The application is launched and we can interact with it

Actual result

The application is launched but it's not possible to interact with it, the app doesn't seems to accept any input from RDP. However, it works locally on the device using a keyboard, mouse or touch-screen, event when launched from RDP.

References

RDP section in weston.ini:

[screen-share]
command=/usr/bin/weston --backend=rdp-backend.so --shell=fullscreen-shell.so --no-clients-resize --rdp-tls-cert=/etc/freerdp/keys/server.crt --rdp-tls-key=/etc/freerdp/keys/server.key
start-on-startup=true

Note

The issue is not present when launching the same application compiled to target the Linux desktop platform (Google's embedder using GTK). It's is working fine locally as well as remotely using RDP in that case.

[HidenoriMatsubayashi]

I have never used RDP in Weston. How about other wayland apps? Do they work fine?

[topolittle]

I have never used RDP in Weston. How about other wayland apps? Do they work fine?

Yes, they do. I tried weston-editor, Gimp and Wireshark. Even a flutter Desktop app (gtk) works fine.

[HidenoriMatsubayashi]

Sorry for no updates, but I'll investigate this problem next week.

[topolittle]

Thank you. It's still an issue on my side.

[HidenoriMatsubayashi]

Sorry, I tried to run RDP on weston, but I couldn't setup/use this feature on Raspberry Pi4 in the first place.

RPI4:
weston version 9.0.0

$ weston --backend=rdp-backend.so --shell=fullscreen-shell.so --rdp-tls-cert=/etc/freerdp/keys/server.crt --rdp-tls-key=/etc/freerdp/keys/server.key

$ flutter-client -b bundle/
[ERROR][elinux_window_wayland.cc(1199)] Xdg-shell is invalid
Segmentation fault

Host:

$ wlfreerdp /v:192.168.10.22
[04:34:12:617] [637099:637099] [WARN][com.freerdp.crypto] - Certificate verification failure 'self-signed certificate (18)' at stack position 0
[04:34:12:617] [637099:637099] [WARN][com.freerdp.crypto] - CN = raspberrypi
[04:34:13:830] [637099:637099] [INFO][com.freerdp.gdi] - Local framebuffer format  PIXEL_FORMAT_BGRA32
[04:34:13:830] [637099:637099] [INFO][com.freerdp.gdi] - Remote framebuffer format PIXEL_FORMAT_BGRA32
[04:34:13:847] [637099:637099] [INFO][com.freerdp.channels.rdpsnd.client] - [static] Loaded fake backend for rdpsnd
[04:34:13:847] [637099:637099] [INFO][com.freerdp.channels.drdynvc.client] - Loading Dynamic Virtual Channel rdpgfx

I can only see the black screen in the RDP client.

So, can you please get trace level logs instead? Or can I try RDP on Ubuntu desktop PC?

$ cmake -DUSER_PROJECT_PATH=examples/flutter-wayland-client -DCMAKE_BUILD_TYPE=Debug ..
$ FLUTTER_LOG_LEVELS=TRACE  ./flutter-client -b <path_to_bundle>

[topolittle]

So, can you please get trace level logs instead? Or can I try RDP on Ubuntu desktop PC?

$ cmake -DUSER_PROJECT_PATH=examples/flutter-wayland-client -DCMAKE_BUILD_TYPE=Debug ..
$ FLUTTER_LOG_LEVELS=TRACE  ./flutter-client -b <path_to_bundle>

Sorry for my late reply, I was in vacation. I will try to get the traces soon and let you know.

[omarhogni]

Came here from #409 seeing the same issue. Here are TRACE logs of a release build when this occurs.

May 23 09:28:35 intel-corei7-64 systemd[1]: Started Start frontend on startup.
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(1351)] Created the Wayland surface: 1280x720
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(194)] wp_presentation_listener.clock_id
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(198)] clk_id = 1
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(287)] wl_seat_listener.capabilities
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(316)] wl_seat_listener.name
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(630)] wl_output_listener.geometry
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(688)] wl_output_listener.scale
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [INFO][elinux_window_wayland.cc(694)] Display scale for output(10): 1
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(638)] wl_output_listener.mode
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [INFO][elinux_window_wayland.cc(646)] Display output info: width = 1280, height = 800, refresh = 59810
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(685)] wl_output_listener.done
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(630)] wl_output_listener.geometry
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(688)] wl_output_listener.scale
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [INFO][elinux_window_wayland.cc(694)] Display scale for output(11): 1
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(638)] wl_output_listener.mode
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [INFO][elinux_window_wayland.cc(646)] Display output info: width = 1280, height = 800, refresh = 59810
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(685)] wl_output_listener.done
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(117)] xdg_toplevel_listener.configure: 1280, 800
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(78)] xdg_surface_listener.configure
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(616)] wl_keyboard_listener.repeat_info
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(558)] wl_keyboard_listener.keymap
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: flutter: Starting application
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(245)] wp_presentation_feedback_listener.discarded
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(156)] wl_surface_listener.enter
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(166)] window entered output 10
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(156)] wl_surface_listener.enter
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(166)] window entered output 11
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(571)] wl_keyboard_listener.enter
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(607)] wl_keyboard_listener.modifiers
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(940)] wl_data_device_listener.selection
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(117)] xdg_toplevel_listener.configure: 1280, 800
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(78)] xdg_surface_listener.configure
May 23 09:28:35 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(264)] wl_callback_listener.done
May 23 09:28:51 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(503)] wl_touch_listener.down
May 23 09:28:51 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(532)] wl_touch_listener.motion
May 23 09:28:51 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(518)] wl_touch_listener.up
May 23 09:29:01 intel-corei7-64 weston[23775]: [09:29:01.142] New VNC client connected 
May 23 09:29:01 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:29:01 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(53)] wl_registry_listener.global
May 23 09:29:01 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(70)] xdg_wm_base_listener.ping
May 23 09:29:01 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(287)] wl_seat_listener.capabilities
May 23 09:29:01 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(316)] wl_seat_listener.name
May 23 09:29:01 intel-corei7-64 flutter-client[24184]: [TRACE][elinux_window_wayland.cc(327)] wl_pointer_listener.enter

Looking at the source it looks like wl_touch is set to null for the second capabilities line. See

flutter-embedded-linux/src/flutter/shell/platform/linux_embedded/window/elinux_window_wayland.cc

Line 305 in 9743862

self->wl_touch_ = nullptr;

If you would like an example configuration for weston to get vnc up and running I can assist with that also.

[omarhogni]

I created a draft PR #417 for handling multiple seats.