XWayland Clipboard Proxy #454

ofuangka · 2019-01-25T20:08:03Z

The pull request #422 disabled Clipboard Sharing for Wayland in order to remove the GSConnect dependency on XWayland. This bug is intended to track a possible alternative implementation.

Steps to reproduce the behavior:

Log into a Wayland session
Open GSConnect Mobile Settings
Select a device
In the "Sharing" panel, Clipboard Sharing is not present

Expected behavior to be determined

System Details:

GSConnect version: >20
- Installed from: GitHub master branch
GNOME/Shell version: 3.30
Distro/Release: Arch

See the pull request linked above for additional details/discussion

getzze · 2019-02-20T18:18:18Z

It is not a perfect solution, but until clipboard is better handled in Wayland it would do: we could have a text area accessible from the gnome menu where we paste the text we want and it can be sent to the clipboard on the phone. At the same time, the phone clipboard is sent to desktop and the text area content is updated with it.
That would be a way to send text between devices.

andyholmes · 2019-02-22T16:07:53Z

I had two thoughts about how to do this. The first was just to run a mini XWayland process that used stdin/stdout pipes to communicate clipboard changes with the daemon. So in GSConnect there would just be a singleton with a signal and a few methods.

The second was something that occurred to me when I found out they built keypress simulation into gnome-shell's Clutter to make their new OSK work in Wayland. Since StClipboard also works in Wayland I thought it would be possible to export those both over DBus. That would kind of kill two birds with one stone, but the DBus interface would have to be always-on, and probably when Atspi works on Wayland we'd want to revert back anyways.

we could have a text area accessible from the gnome menu where we paste the text we want and it can be sent to the clipboard on the phone.

Probably this wouldn't be a bad idea anyways, since it would be a more accessible way to sync clipboard content on demand. I guess I just worry about UI clutter and we'd need a way to inject the StEntry since the device menu is built from a GMenu.

getzze · 2019-02-22T23:29:20Z

I like your second idea, although I don't know if dbus messaging will be fast enough for mouse and keyboard events.

andyholmes · 2019-02-25T17:52:23Z

I don't know if dbus messaging will be fast enough for mouse and keyboard events.

That's good point, although direct DBus connection is an option and still easier than writing some pipe protocol.

On the other hand, I just saw this MR although haven't had time to really look at it (https://gitlab.gnome.org/GNOME/mutter/merge_requests/320). I'm guessing because of how things like clipboards work in Wayland they don't have a choice but to build it into the compositor/mutter.

ArtyomZorin · 2019-03-12T00:46:58Z

I came across the new release of the Sway compositor for Wayland, and I saw that they included this library for interacting with the clipboard: https://github.com/bugaevc/wl-clipboard

Not sure if it'll work with Gnome, but it might be worth a look.

ferdnyc · 2019-03-12T11:42:59Z

@ArtyomZorin I looked at wl-clipboard a while ago. There are two main issues with using it in GSConnect:

It's not actually a library, it's a pair of command line utilities to get and set the clipboard contents. In theory GSConnect could use those utilities as helper programs, as it does several other tools, but...
It's entirely "pull"-based clipboard manipulation. IOW, it's meant as a way to manipulate the current clipboard, when the user wishes to do so. For clipboard updates coming from the Android side (when a selection is sent from the device by the KDE Connect app) that's fine, gsconnect could call wl-copy and pass it the clipboard data from KDE Connect. But GSConnect also needs to be able to automatically pick up any changes in the current clipboard and/or primary selection, so they can be mirrored to the device. wl-clipboard didn't appear to be useful for that side of the functionality, since there's no way to know when wl-paste should be called to collect the clipboard data.

andyholmes · 2019-03-14T13:23:20Z

Ugh, so I was looking into StClipboard since it seems to work in Wayland by way of some XWayland hackery, but unfortunately it doesn't have any signals like GtkClipboard. Dealing with this in GJS means polling in gnome-shell's main thread, which is what Clipboard Indicator does. Although a idle GSource with low priority probably shouldn't be intrusive, we then have to pass it over DBus as well.

Presumably GtkClipboard should actually work in Wayland (from what I can tell), however in Gtk/Gdk4 the clipboard seems to have been moved to Gdk. I might have a look into GtkClipboard, because I'd much rather this get fixed upstream before they stop doing releases of Gtk3, but I really have no idea what the plan is.

ferdnyc · 2019-03-18T17:31:25Z

I'm not sure if I mentioned this previously, apologies if I'm being redundant.

In my testing, CopyQ appeared to be fully-functional (or, at least, has the full functionality GSConnect would require in order to do clipboard sync, meaning selection/copy buffer capture and update) when it's running as an XWayland application. But...

It's a Qt application, so it's got a lot of heavy dependencies.
At least in my limited testing with the build in Fedora's repo, setting Qt to run in Wayland-native mode via the QT_QPA_PLATFORM=wayland environment variable breaks the necessary functionality (and, indeed, most functionality).

andyholmes · 2019-03-20T19:18:16Z

I had a look at the source of CopyQ, but it looks like it only really supports X11, and probably just works on XWayland because of GNOME using a little hidden GDK window.

diegovasconcelos · 2019-05-22T16:09:00Z

Look:
https://www.phoronix.com/scan.php?page=news_item&px=GNOME-3.34-Clipboard-Manager

rugk · 2019-06-01T12:40:54Z

This idea has not really been discussed yet:

we could have a text area accessible from the gnome menu where we paste the text we want and it can be sent to the clipboard on the phone.

IMHO, I very much like this as a workaround.
As it seems clipboard support is not coming soon, and I really, really miss it (it was one of the main reasons to use GSConnect), would not something like this be useful?

andyholmes · 2019-06-01T22:12:55Z

It's a less ideal solution, for the reason that it will require arbitrarily injecting a text entry into the menu. Currently, the whole device menu is exported over DBus using GMenu, which is very convenient and all done using pre-existing C code. We simply add and remove menu items and submenus in the service and they're picked up by the shell extension.

I have a branch somewhere with a partially working menu editor, which will be necessary if we ever want users to have the option of customizing the menu items. If we start injecting elements into the menu in this way, that will become a lot less feasible to do. It's also not very HIG, and complaints about that before was why I removed the circular battery graphic.

More generally, the problem with Wayland and KDE Connect is that it presumes access to desktop resources that often directly contradicts to the application isolation that Wayland presumes. I think the simplest way to fix it for now is just to run a small XWayland process and pipe the changes back and forth, I just haven't had time to do it.

Since GtkClipboard isn't working well under Wayland, we start a small subprocess and force the GdkX11 backend. We then pipe clipboard changes back and forth over stdin/stdout. closes #454

andyholmes · 2019-06-02T00:28:29Z

I threw together a quick fix sitting in the xwayland-clipboard branch. If something goes wrong, and it might, it will probably go very wrong and could spin up your CPU and use enough memory to start swapping out. Here's a build for ~~canaries~~ testers:

~~gsconnect@andyholmes.github.io.zip~~

ferdnyc · 2019-06-02T18:17:10Z

I'm still running X on my daily-driver (Nvidia card, binary kernel module), but I'll fire up my laptop w/ F30/Wayland later and give it a try. Been a while since I've visited a coal mine...

ferdnyc · 2019-06-03T08:34:56Z

Hmm. I installed the zipfile version you provided, @andyholmes, but something's being weird... I paired the phone to my laptop no problem, and everything looks normal (and enabled) from that end. But in the phone I'm only getting a tiny subset of plugins even AVAILABLE. And clipboard sync isn't one of them.

Everything is normal with the pairing to my desktop, still. Maybe I need to unpair/disconnect the other machine?

ferdnyc · 2019-06-03T10:28:40Z

Nope, same thing with the other computer unpaired. I've also restarted on both sides of the connection. I'll grab a debug log of the handshaking and pairing process when I get a chance.

Since GtkClipboard isn't working well under Wayland, we start a small subprocess and force the GdkX11 backend. We then pipe clipboard changes back and forth over stdin/stdout. closes #454

andyholmes · 2019-06-03T21:33:14Z

Oops, sorry. There was a logic inversion mistake in there. Here's an updated ZIP:

gsconnect@andyholmes.github.io.zip

ferdnyc · 2019-06-04T12:23:07Z

@andyholmes
So I gave the replacement zip a quick spin, between my laptop and my phone with Google Docs open.

Text typed into GEdit can be copied on the laptop, then immediately pasted on the phone.
Text copied in Google Docs on the phone can be pasted into LibreOffice Writer on the laptop, again no issues.
If the text has any formatting applied to it, that's discarded and the plain text comes through, as expected.
Copying non-text content, such as a selection area in the GIMP, results in an empty clipboard on the phone, also as expected.

The gjs process for xclipboard never even registered as far as CPU usage goes, and it takes up < 40MB of resident memory which is nothing significant.

All in all, looks good so far! Very cool.

rugk · 2019-06-04T13:44:22Z

it takes up < 40MB of resident memory

I'd consider 40MB for a simply clipboard proxy service pretty much… 🤔

andyholmes · 2019-06-04T19:04:45Z

The gjs process for xclipboard never even registered as far as CPU usage goes, and it takes up < 40MB of resident memory which is nothing significant.

For me it was only about 9MB RSS, which I think is ~5 for the gjs engine and ~4 for the libraries, was yours really about 40?

ferdnyc · 2019-06-04T19:49:07Z

For me it was only about 9MB RSS, which I think is ~5 for the gjs engine and ~4 for the libraries, was yours really about 40?

Yeah, well... that's what top showed in the "RES" column, who knows what gets included in there? If I just run gjs and leave it sitting at the command line the process is already 24MB, however it measures.

ETA: Looks like top's "RES" column is all RAM allocation including shared memory, which in that same sitting-at-the-command-line gjs is already 18MB, so that probably accounts for the difference.

Since GtkClipboard isn't working well under Wayland, we start a small subprocess and force the GdkX11 backend. We then pipe clipboard changes back and forth over stdin/stdout. closes #454

andyholmes · 2019-06-12T06:19:42Z

I've merged this in, since I haven't heard anyone complain. I'll spin up a RC sometime soon and see how that goes.

terencode · 2019-06-12T15:06:37Z

@andyholmes for GNOME 3.34 you would be able to use https://gitlab.gnome.org/GNOME/mutter/merge_requests/320

ferdnyc · 2019-06-13T09:31:38Z

@andyholmes for GNOME 3.34 you would be able to use https://gitlab.gnome.org/GNOME/mutter/merge_requests/320

Holy crap, that MR has a longer history than the city of Rome! 😆 All resolved in the end, I guess. (Though that note from the initial comment, "but it may work on both x11 and wayland" — emphasis mine — is vaguely concerning. Maybe I'm just reading it too literally.)

HEXcube · 2019-06-20T06:35:17Z

I've merged this in, since I haven't heard anyone complain. I'll spin up a RC sometime soon and see how that goes.

I tried the new v25-rc1 prerelease on my Ubuntu 18.04 LTS with GNOME 3.28 Wayland session. And guess what, clipboard still doesn't work! 😞 When copying from desktop, I get a {} on phone and nothing otherwise.

ferdnyc · 2019-06-20T09:12:30Z

@HEXcube Hmm. I tested on 3.32.2... possibly 3.28 is missing some functionality.

Can you do this test, for starters? Open a terminal, and inside it run:

$ cd ~/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/components/
$ ./xclipboard

It'll just sit there doing nothing (possibly after a version warning message or two), but then after that, anytime you copy text it should be instantly output to the terminal where xclipboard is running. Then just hit Ctrl+c in the terminal when you're done.

For example:

$ ~/.local/.../gsconnect@andyholmes.github.io/service/components/xclipboard
Gjs-Message: 05:06:49.193: JS WARNING: [./xclipboard 5]: Requiring Gdk but it ha
s 2 versions available; use imports.gi.versions to pick one
anytime you copy text&gave the replacement zip a quick spin, inside it run:
```sh#Text typed into GEdit can be copied^C

(...As I went around this page and copied some selected text a few times.)

Reopening for possible 3.28 compatibility issues.

ferdnyc · 2019-06-20T09:14:44Z

Oops. For the email readers, I messed up my commands in the previous message. The correct commands would be:

$ cd ~/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/components/
$ ./xclipboard

(Forgot the all-important ./!) I've edited my original comment accordingly.

* clarify process pipe names * watch stderr for errors * cleanup `xclipboard` and comments addresses #454

andyholmes · 2019-06-22T00:15:47Z

Here's an updated Zip with a little more corner-cases covered:

gsconnect@andyholmes.github.io.zip

HEXcube · 2019-06-25T10:58:34Z

Can you do this test, for starters? Open a terminal, and inside it run:
$ cd ~/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/components/
$ ./xclipboard
It'll just sit there doing nothing (possibly after a version warning message or two), but then after that, anytime you copy text it should be instantly output to the terminal where xclipboard is running. Then just hit Ctrl+c in the terminal when you're done.

This is what I got on v25-rc1:

./xclipboard
Gtk-Message: 16:17:48.052: Failed to load module "appmenu-gtk-module"
Gtk-Message: 16:17:48.054: Failed to load module "appmenu-gtk-module"
dointerminal�outputL(...As I went around this page and copied some selected text a few times.)

HEXcube · 2019-06-25T11:05:00Z

With the new v25-rc2:

~/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/components/xclipboard
Gtk-Message: 16:32:18.035: Failed to load module "appmenu-gtk-module"
Gtk-Message: 16:32:18.036: Failed to load module "appmenu-gtk-module"
email re� output to the terMxclipboard is running. Then just hit Ctrl+c in the terminal when you're done.Qsit there doing nothing (possibly after a version warning message or two), but th

HEXcube · 2019-06-25T12:37:45Z

Here's an updated Zip with a little more corner-cases covered:

gsconnect@andyholmes.github.io.zip

When I tried this zip, got this output:

~/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/components/xclipboard
Gtk-Message: 18:05:39.682: Failed to load module "appmenu-gtk-module"
Gtk-Message: 18:05:39.683: Failed to load module "appmenu-gtk-module"
 output to th� (possibly after a version warning message or two), but then after that, anytime you copy text it should be instantly output to the terminal where xclipboard is running. Then just hit Ctrl+c in the terminal when you're doneUt'll just sit there doing nothing (possibly after a version warning message or two),

andyholmes · 2019-06-25T19:44:00Z

The xclipboard script seems to be working correctly. Are you sure you have clipboard sync enabled in the settings?

HEXcube · 2019-06-26T09:04:53Z

The xclipboard script seems to be working correctly. Are you sure you have clipboard sync enabled in the settings?

Yes, it's enabled on both phone and GSconnect.

andyholmes · 2019-06-27T04:13:57Z

Does generating a support log provide any information as to what's happening?

HEXcube · 2019-06-28T13:08:37Z

Does generating a support log provide any information as to what's happening?

Got this:
gsconnect.9F643Z.log

Output of xclipboard:

~/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/components/xclipboard
Gtk-Message: 18:30:53.829: Failed to load module "appmenu-gtk-module"
Gtk-Message: 18:30:53.831: Failed to load module "appmenu-gtk-module"
`It'll just sit there doing nothing (possibly after a version warning message or two), but then a4 output to the terminal where xclipboard is running.9 Then just hit Ctrl+c in the terminal when you're done.

andyholmes · 2019-06-29T00:04:22Z

From your log it seems like the xclipboard script is failing to start for some reason. Are there any messages in journalctl or GNOME Logs about this happening? It should be right around the time this message is printed:

[session uid=1000 pid=23016] Successfully activated service 'org.gnome.Shell.Extensions.GSConnect'

HEXcube · 2019-06-30T07:23:14Z

I got this from GNOME Logs:

WARNING: XClipboard not running

It wasn't displayed immediately after GSconnect activation though. I couldn't find any other references to xclipboard, even after trying multiple copy paste attempts on my notebook and connected phone.

HEXcube · 2019-06-30T07:24:43Z

I don't know if it's related, but gnome-terminal-server outputs this message 😕 :

Display does not support owner-change; copy/paste will be broken!

andyholmes · 2019-06-30T21:08:59Z

Display does not support owner-change; copy/paste will be broken!

This is because in GDK's Wayland backend the function gdk_display_supports_selection_notification () always returns false. This shouldn't be related to the problem though.

I'm guessing the error is not showing up in the log because it's a debug message, which won't be logged until Generate Support Log is activated after GSConnect starts. Give this Zip a shot and see if there are any errors during around GSConnect startup:

gsconnect@andyholmes.github.io.zip

HEXcube · 2019-07-03T13:28:09Z

After testing this new zip, I got some error messages on log. These are some relevant parts of the log:

XClipboard log messages.log
GSconnect log messages.log

HEXcube · 2019-07-03T13:33:23Z

XClipboard log messages.log
GSconnect log messages.log

GitHub isn't showing preview for these files, so I thought I'll paste them here directly:

17:31:38 gjs: WARNING: XClipboard not running
17:31:38 gjs: WARNING: XClipboard not running
17:31:38 gjs: WARNING: XClipboard not running
17:31:13 gjs: WARNING: XClipboard Proxy: 
17:31:13 gjs: WARNING: XClipboard Proxy: @/home/user/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/components/xclipboard:90:1
17:31:13 gjs: WARNING: XClipboard Proxy: readContent@/home/user/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/components/xclipboard:76:9
17:31:13 gjs: WARNING: XClipboard Proxy: (gjs:3060): Gjs-WARNING **: 17:31:13.486: JS ERROR: Error: Expected type utf8 for Argument 'text' but got type 'object'
17:31:13 gjs: WARNING: XClipboard Proxy: 
17:05:02 gjs: WARNING: XClipboard Proxy: Gtk-Message: 17:05:02.519: Failed to load module "appmenu-gtk-module"
17:05:02 gjs: WARNING: XClipboard Proxy: Gtk-Message: 17:05:02.513: Failed to load module "appmenu-gtk-module"
11:25:34 gjs: WARNING: XClipboard not running

18:43:47 gnome-shell: [pixel-saver]: Finding XID for window gsconnect@andyholmes.github.io failed
17:31:56 dbus-daemon: ('Handling file: ', 'file:///home/user/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/stylesheet.css')
17:31:13 gjs: WARNING: XClipboard Proxy: @/home/user/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/service/components/xclipboard:90:1
17:26:06 dbus-daemon: ('Handling file: ', 'file:///home/user/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io')
17:05:02 gnome-shell: JS ERROR: /org/gnome/Shell/Extensions/GSConnect/Device/349dcdcd550cddea: TypeError: this.device.action_group is null
_init@/home/user/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/shell/device.js:51:9
Menu@/home/user/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/shell/device.js:187:25
_init@/home/user/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/shell/device.js:237:20
_Base.prototype._construct@resource:///org/gnome/gjs/modules/_legacy.js:18:5
Class.prototype._construct/newClass@resource:///org/gnome/gjs/modules/_legacy.js:114:32
_onDeviceAdded@/home/user/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/extension.js:235:29
_onInterfacesAdded@/home/user/.local/share/gnome-shell/extensions/gsconnect@andyholmes.github.io/shell/remote.js:201:13
17:05:02 dbus-daemon: [session uid=1000 pid=4081] Successfully activated service 'org.gnome.Shell.Extensions.GSConnect'

andyholmes · 2019-09-06T18:44:03Z

Hi, sorry there haven't been any updates on this issue - it's still targetted for v25.

@HEXcube So far I think you're the only tester having problems, and you seem to be the only one on 3.28. Could you please try this ZIP here:

gsconnect@andyholmes.github.io.zip

In particular these lines here:

https://github.com/andyholmes/gnome-shell-extension-gsconnect/blob/49364e555d001081e0cf10c6c635fd92351b0686/src/service/components/clipboard.js#L86-L90

https://github.com/andyholmes/gnome-shell-extension-gsconnect/blob/49364e555d001081e0cf10c6c635fd92351b0686/src/service/components/xclipboard#L69-L73

I believe these lines are the proper way to account for +/- 3.28 behaviour, but I haven't had a copy of 3.28 running in awhile. If the clipboard still doesn't work, think this may be the spot causing problems and might need your help with a little debugging.

andyholmes · 2019-09-10T07:39:39Z

I'm marking this as closed since the feature was released in v25. Please open a new issue if problems still occur with v25.

HEXcube · 2019-09-13T15:34:07Z

@andyholmes This bug is now fixed! 😃 I tested the latest update of GSconnect from GNOME Extensions site. I'm running GSConnect v26 now on Ubuntu 18.04 LTS with GNOME 3.28 Wayland session and clipboard sync works smoothly. Thanks a lot! 😘

andyholmes added enhancement A request for a feature or additional functionality wayland labels Jan 26, 2019

andyholmes mentioned this issue Jun 3, 2019

Clipboard: add XWayland subprocess for clipboard on Wayland #552

Merged

andyholmes closed this as completed in #552 Jun 12, 2019

ferdnyc reopened this Jun 20, 2019

andyholmes added a commit that referenced this issue Jun 22, 2019

Clipboard Component: improvements to XWayland clipboard proxy

b316750

* clarify process pipe names * watch stderr for errors * cleanup `xclipboard` and comments addresses #454

andyholmes added the needs testing A contribution that needs testing label Jun 22, 2019

andyholmes added this to the v25 milestone Aug 18, 2019

andyholmes closed this as completed Sep 10, 2019

andyholmes mentioned this issue Nov 10, 2019

It does not start on Wayland, it works only on X11. #684

Closed

XWayland Clipboard Proxy #454

XWayland Clipboard Proxy #454

Comments

ofuangka commented Jan 25, 2019

getzze commented Feb 20, 2019

andyholmes commented Feb 22, 2019

getzze commented Feb 22, 2019

andyholmes commented Feb 25, 2019

ArtyomZorin commented Mar 12, 2019

ferdnyc commented Mar 12, 2019 • edited Loading

andyholmes commented Mar 14, 2019

ferdnyc commented Mar 18, 2019

andyholmes commented Mar 20, 2019

diegovasconcelos commented May 22, 2019

rugk commented Jun 1, 2019

andyholmes commented Jun 1, 2019

andyholmes commented Jun 2, 2019 • edited Loading

ferdnyc commented Jun 2, 2019

ferdnyc commented Jun 3, 2019

ferdnyc commented Jun 3, 2019

andyholmes commented Jun 3, 2019

ferdnyc commented Jun 4, 2019

rugk commented Jun 4, 2019

andyholmes commented Jun 4, 2019

ferdnyc commented Jun 4, 2019 • edited Loading

andyholmes commented Jun 12, 2019

terencode commented Jun 12, 2019

ferdnyc commented Jun 13, 2019

HEXcube commented Jun 20, 2019

ferdnyc commented Jun 20, 2019 • edited Loading

ferdnyc commented Jun 20, 2019

andyholmes commented Jun 22, 2019

HEXcube commented Jun 25, 2019 • edited Loading

HEXcube commented Jun 25, 2019

HEXcube commented Jun 25, 2019

andyholmes commented Jun 25, 2019

HEXcube commented Jun 26, 2019

andyholmes commented Jun 27, 2019

HEXcube commented Jun 28, 2019

andyholmes commented Jun 29, 2019

HEXcube commented Jun 30, 2019

HEXcube commented Jun 30, 2019

andyholmes commented Jun 30, 2019

HEXcube commented Jul 3, 2019

HEXcube commented Jul 3, 2019

andyholmes commented Sep 6, 2019

andyholmes commented Sep 10, 2019

HEXcube commented Sep 13, 2019

ferdnyc commented Mar 12, 2019 •

edited

Loading

andyholmes commented Jun 2, 2019 •

edited

Loading

ferdnyc commented Jun 4, 2019 •

edited

Loading

ferdnyc commented Jun 20, 2019 •

edited

Loading

HEXcube commented Jun 25, 2019 •

edited

Loading