async clipboard api

[totaam]

Issue migrated from trac ticket # 1844

component: html5 | priority: critical | resolution: fixed

2018-05-16 06:42:13: antoine created the issue

---

See Unblocking Clipboard Access: It's a replacement for execCommand-based copy & paste that has a well-defined permissions model and doesn't block the page.

MDN: Navigator clipboard.

Original html5 clipboard tickets: Xpra-org/xpra#842, Xpra-org/xpra#1461

[totaam]

2019-04-10 08:29:10: antoine commented

---

See also related work in Xpra-org/xpra#812

[totaam]

2019-06-03 05:02:16: antoine changed status from new to assigned

[totaam]

2019-06-03 05:02:16: antoine commented

---

For images, see #71

[totaam]

2019-06-03 07:32:04: antoine changed priority from major to critical

[totaam]

2019-06-03 07:32:04: antoine commented

---

Blocker for #71

[totaam]

2019-06-03 11:50:06: antoine edited the issue description

[totaam]

2019-06-03 11:50:06: antoine commented

---

Updates:

• r22834 new async api calls
• r22837 answer clipboard requests
• r22838 workaround PRIMARY + CLIPBOARD overwriting values
• r22839 don't bother synchronizing PRIMARY if we have the new async API
• r22840 stricter test for navigator.clipboard API availability

Important: just found information on clipboardchange event in w3c: Async Clipboard API. This is not supported in any browser yet.
This means we would not necessarily have to claim the clipboard every time we get a token, we could rely on those events to notify the server.
For chrome: onClipboardDataChanged.

Maybe we should query the Permissions API rather than just checking if the method exists?
See Asynchronous Clipboard API Sample

Tested:

• Chrome 75.0.3770.51 (Official Build) beta (64-bit) Linux: OK (connecting to localhost using websocket)
• Chrome 74 MS Windows: OK via https, otherwise legacy mode
• Firefox 67 Linux: legacy mode
• Firefox 67 MS Windows: legacy mode

The Firefox results are surprising since they claim to support the async API since V63, but this matches the results from this test.
The spec says: Firefox only supports reading the clipboard in browser extensions, using the "clipboardRead" extension permission.

Then there are also issues with "secure contexts": works for localhost, otherwise https is required, etc..

[totaam]

2019-06-11 13:43:49: antoine commented

---

IE fixes in r22862. (no arrow functions)

The problem with the permissions API is that if the user denies the request, we cannot ask again and we're left with a non-functional API..
To restore things, one has to visit chrome://settings/content/siteDetails?site=https%3A%2F%2Fgooglechrome.github.io%2F (chrome), or follow steps similar to How do I make Chrome forget a “no” to geolocation on a site?

[totaam]

2019-06-11 17:22:50: antoine commented

---

Also fixes #70

[totaam]

2019-06-12 12:32:46: antoine changed status from assigned to new

[totaam]

2019-06-12 12:32:46: antoine changed owner from antoine to zaveri

[totaam]

2019-06-12 12:32:46: antoine commented

---

For testing: the html5 client will print a message to the console indicating if it's using the old or new (async) clipboard code. (message actually fixed in r22911)

Tweaks and better compatibility for legacy mode in r22918.
IE compatibility fixes in r22919 + r22920.

The new code saves having to synchronize the PRIMARY selection constantly, we do that with the old code just in case we get a clipboard copy request from the browser so that we can provide a response instantly. (this is the synchronous clipboard)
With the new async code, we populate the clipboard only when needed.

[totaam]

2019-08-25 04:27:12: antoine changed owner from zaveri to afarr

[totaam]

2019-08-25 04:27:12: antoine commented

---

@afarr: can you take over zaveri?
This ticket goes with #71.

[totaam]

2019-09-04 17:55:59: afarr commented

---

Well, as a start I did a test of the clipboard with the windows client (3.0r23707) against a 3.0-23689 server on Fedora 30 - and none of my old tricks was able to find a problem.
Will update with results from html5 client tests.

[totaam]

2019-09-05 21:55:09: afarr commented

---

Testing with the same 3.0-23689 server on Fedora 30 with Chrome html5 client, it looks like nothing copied from a local application can be pasted into the html5 client session with a right-click menu paste option (not into a browser input field, or a gedit field) - but the contents are being sync'd with the server in some way, because immediately following a right-click paste (which will paste the last contents copied/pasted server-side) failure an immediate use of keyboard shortcut (control-v in this case) will paste the contents last copied locally.

Repro (that got confusing just writing it).

• Copy something in a session.
• Paste inside the session (right-click menu or keyboard shortcut) just to be sure what's there (should succeed).
• Paste into a local application (should succeed, right-click of keyboard shortcut).
• Copy something else in local application (right-click or keyboard shortcut). (Optionally then paste it again locally just to be sure nothing super-weird is happening.)
• Return to application in the session (browser, gedit or other editor, not xterm - they behave uniquely) & use right-click to paste (should fail, pasting whatever was copied inside the session last, rather than what was copied from local application).
• Immediately use keyboard shortcut to paste same contents into session application (now it should work, pasting different contents than what was just pasted with the right-click).

As an added bonus, the issue of Xpra-org/xpra#2403 (more than 30 clipboard requests per second easily triggered with gedit) is equally easy to trigger with the Chrome html5 client (and equally easy to disable clipboard entirely, requiring a disconnect and reconnect of the session to re-enable the clipboard).

Additionally (and I'm not sure if this should be a separate ticket), after triggering the warning for about 544 ms I somehow triggered a keyboard layout error?

2019-09-05 13:35:36,696 client   2 broadway decoder initialized
2019-09-05 13:35:43,892 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:43,909 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:43,909 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:43,925 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:43,942 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:43,958 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:43,978 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:43,992 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,009 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,025 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,042 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,074 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,074 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,103 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,109 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,159 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,160 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,247 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,264 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,335 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,351 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,369 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,384 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,402 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:44,436 Warning: more than 30 clipboard requests per second!
2019-09-05 13:35:46,720 setting keyboard layout to 'us'
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Internal error:   Could not resolve keysym XF86MonBrightnessCycle
> Internal error:   Could not resolve keysym XF86RotationLockToggle
Errors from xkbcomp are not fatal to the X server

Will check with another html5 client when next I get a chance.

[totaam]

2019-09-06 14:13:02: antoine changed status from new to assigned

[totaam]

2019-09-06 14:13:02: antoine changed owner from afarr to antoine

[totaam]

2019-09-06 14:13:02: antoine commented

---

... looks like nothing copied from a local application can be pasted into the html5 client session with a right-click menu paste option ...
Here's what's happening here: when you copy something using a local application, we currently have no way of knowing that something has happened to the local clipboard and that some new data is available.. (unless we use some sort of polling)
So when you right click in the server side application and use the popup menu to do something with the clipboard, we're still using the old value.
Control-V on the keyboard is handled differently: we detect this specific key combination (and some others, Shift+Insert, macos ones) and proactively query the local clipboard to send the new clipboard contents before sending the key event, so the server will have the latest clipboard contents before handling the clipboard shortcut.
I don't think we can use a similar strategy for clicks as there is no way of knowing what the click is meant to do: the click could be on a "cut" or "copy" button or menu entry, just as well as a "paste" button. There's no way to tell.
And proactively copying the data might just override the application's intended action for this event.

I'm experimenting with a polling solution, which is tricky.

As an added bonus, the issue of Xpra-org/xpra#2403 (more than 30 clipboard requests per second easily triggered with gedit) is equally easy to trigger with the Chrome html5 client
It shouldn't be. We're not supposed to synchronize the PRIMARY selection at all when the async clipboard code is used. And the CLIPBOARD selection shouldn't be changing that often.
I'll look into it.
(and equally easy to disable clipboard entirely, requiring a disconnect and reconnect of the session to re-enable the clipboard).
That's no longer an issue: r23723, Xpra-org/xpra#2403#comment:1.

[totaam]

2019-09-06 14:54:16: antoine changed status from assigned to new

[totaam]

2019-09-06 14:54:16: antoine changed owner from antoine to alas

[totaam]

2019-09-06 14:54:16: antoine commented

---

Try r23724: whenever we get a click event, we now poll the local clipboard for changes.

Notes and caveats:

• may trigger clipboard warnings with some browser versions and security settings - so maybe this new feature should be made optional?
• tested on Chrome only for now - Firefox isn't working (will look into it - maybe not allowed from click events?)
• ideally the code should be modified to use an asynchronous call, to guarantee that the clipboard contents are sent before the click - meh: difficult to read, and may delay the clicks too much
• if this approach works well enough, maybe we should use it more so that we can catch weird keyboard shortcuts? (ie: fire the clipboard polling code on every key event? probably not with Firefox..)
• I still need to look into the PRIMARY 30 requests per second issue

[totaam]

2019-09-07 13:14:03: antoine commented

---

Updates:

• Firefox is unlikely to work: readText: Firefox only supports reading the clipboard in browser extensions, using the "clipboardRead" extension permission.
• MS Edge and IE11 don't have any support for the new clipboard API, but using the legacy API seems to work for polling (needed fixes from r23726 + r23727) - tested various combinations downloaded from microsoft.com : Download virtual machines
• I am unable to reproduce any problems with the PRIMARY selection: although I do see events firing on the server-side with -d clipboard, those events never get sent to the browser as it doesn't enable this selection:

$ xpra info | grep PRIMARY.enabled
clipboard.PRIMARY.enabled=False

[totaam]

2019-11-05 20:46:01: afarr changed owner from alas to antoine

[totaam]

2019-11-05 20:46:01: afarr commented

---

Testing again with a newly updated Fedora 30 3.0.2-24387, launching the server with my usual command:

... then connect with a Windows 7 Chrome 77 html5 client.

It looks like I'm still very easily able to trigger the more than 30 requests per second error when testing with gedit... but with the -d clipboard flag on, it looks like there are some tracebacks happening whenever I try to copy from a gedit window, regardless of whether I trigger the more than 30 requests! message.

Testing by very simply using one of the two xterms to launch gedit, typing 'test' into a local application (Word in this case), I see the following output when I use control-v to paste into the server-side gedit (which works as expected).

2019-11-05 11:56:47,890 process clipboard token selection=CLIPBOARD, local clipboard name=CLIPBOARD, proxy=X11ClipboardProxy(CLIPBOARD)
2019-11-05 11:56:47,890 wire selection to raw, encoding=bytes, type=UTF8_STRING, format=8, len(data)=4
2019-11-05 11:56:47,891 got token, selection=CLIPBOARD, targets=[b'UTF8_STRING', b'text/plain'], target data={'UTF8_STRING': ('UTF8_STRING', 8, b'test')}, claim=True, can-receive=True
2019-11-05 11:56:47,891 got_contents('TARGETS', 'ATOM', 32, '"b\'\\\\xe7\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\xaf\\\\x01\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\'"') pending=
2019-11-05 11:56:47,892 got_contents('UTF8_STRING', 'UTF8_STRING', 8, '"b\'test\'"') pending=
2019-11-05 11:56:47,892 claim_selection: set selection owner returned 1, owner=0x400008
2019-11-05 11:56:47,892 claim_selection: sending message to root window
2019-11-05 11:56:47,892 claim_selection: done, owned=True
2019-11-05 11:56:47,893 do_xpra_xfixes_selection_notify_event(<X11:XFSelectionNotify {'send_event': '0', 'serial': '0x1289', 'delivered_to': '0x400008', 'window': '0x400008', 'subtype': '0', 'owner': '0x400008', 'selection': 'CLIPBOARD', 'timestamp': '1503780738', 'selection_timestamp': '1503780738'}>)
2019-11-05 11:56:47,893 do_selection_notify_event(<X11:XFSelectionNotify {'send_event': '0', 'serial': '0x1289', 'delivered_to': '0x400008', 'window': '0x400008', 'subtype': '0', 'owner': '0x400008', 'selection': 'CLIPBOARD', 'timestamp': '1503780738', 'selection_timestamp': '1503780738'}>) owned=True, was True, enabled=True, can-send=True
2019-11-05 11:56:47,893 do_xpra_selection_request(<X11:SelectionRequest {'send_event': '0', 'serial': '0x128a', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'TARGETS', 'property': 'GDK_SELECTION', 'time': '1503780738'}>)
2019-11-05 11:56:47,893 do_selection_request_event(<X11:SelectionRequest {'send_event': '0', 'serial': '0x128a', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'TARGETS', 'property': 'GDK_SELECTION', 'time': '1503780738'}>)
2019-11-05 11:56:47,894 clipboard request for CLIPBOARD from window 0xa00124: 'gedit', target=TARGETS, prop=GDK_SELECTION
2019-11-05 11:56:47,894 using existing TARGETS value as response: ('UTF8_STRING', 'text/plain')
2019-11-05 11:56:47,894 set_selection_response(<GdkX11.X11Window object at 0x7f97ac529a00 (GdkX11Window at 0x55fa573d5450)>, TARGETS, GDK_SELECTION, ATOM, 32, "'ç\\x00\\x00\\x00\\x00\\x00\\x00\\x00¯\\x01\\x00\\x00\\x00\\x00\\x00\\x00'", 1503780738)
2019-11-05 11:56:47,896 do_xpra_selection_request(<X11:SelectionRequest {'send_event': '0', 'serial': '0x129b', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'GTK_TEXT_BUFFER_CONTENTS', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,896 do_selection_request_event(<X11:SelectionRequest {'send_event': '0', 'serial': '0x129b', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'GTK_TEXT_BUFFER_CONTENTS', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,897 clipboard request for CLIPBOARD from window 0xa00124: 'gedit', target=GTK_TEXT_BUFFER_CONTENTS, prop=GDK_SELECTION
2019-11-05 11:56:47,897 client is requesting an unknown target: 'GTK_TEXT_BUFFER_CONTENTS'
2019-11-05 11:56:47,897  valid targets: UTF8_STRING, text/plain
2019-11-05 11:56:47,897  dropping the request
2019-11-05 11:56:47,897 set_selection_response(<GdkX11.X11Window object at 0x7f97ac52ba00 (GdkX11Window at 0x55fa573d5450)>, GTK_TEXT_BUFFER_CONTENTS, GDK_SELECTION, STRING, 8, "''", 1503780742)
2019-11-05 11:56:47,898 do_xpra_selection_request(<X11:SelectionRequest {'send_event': '0', 'serial': '0x12a1', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'TARGETS', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,898 do_selection_request_event(<X11:SelectionRequest {'send_event': '0', 'serial': '0x12a1', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'TARGETS', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,898 clipboard request for CLIPBOARD from window 0xa00124: 'gedit', target=TARGETS, prop=GDK_SELECTION
2019-11-05 11:56:47,898 using existing TARGETS value as response: ('UTF8_STRING', 'text/plain')
2019-11-05 11:56:47,899 set_selection_response(<GdkX11.X11Window object at 0x7f97ac52ba00 (GdkX11Window at 0x55fa573d5450)>, TARGETS, GDK_SELECTION, ATOM, 32, "'ç\\x00\\x00\\x00\\x00\\x00\\x00\\x00¯\\x01\\x00\\x00\\x00\\x00\\x00\\x00'", 1503780742)
2019-11-05 11:56:47,899 do_xpra_selection_request(<X11:SelectionRequest {'send_event': '0', 'serial': '0x12a7', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'text/plain;charset=utf-8', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,899 do_selection_request_event(<X11:SelectionRequest {'send_event': '0', 'serial': '0x12a7', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'text/plain;charset=utf-8', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,900 clipboard request for CLIPBOARD from window 0xa00124: 'gedit', target=text/plain;charset=utf-8, prop=GDK_SELECTION
2019-11-05 11:56:47,900 client is requesting an unknown target: 'text/plain;charset=utf-8'
2019-11-05 11:56:47,900  valid targets: UTF8_STRING, text/plain
2019-11-05 11:56:47,900  dropping the request
2019-11-05 11:56:47,900 set_selection_response(<GdkX11.X11Window object at 0x7f97ac52ba00 (GdkX11Window at 0x55fa573d5450)>, text/plain;charset=utf-8, GDK_SELECTION, STRING, 8, "''", 1503780742)
2019-11-05 11:56:47,901 do_xpra_selection_request(<X11:SelectionRequest {'send_event': '0', 'serial': '0x12ad', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'UTF8_STRING', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,901 do_selection_request_event(<X11:SelectionRequest {'send_event': '0', 'serial': '0x12ad', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'UTF8_STRING', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,901 clipboard request for CLIPBOARD from window 0xa00124: 'gedit', target=UTF8_STRING, prop=GDK_SELECTION
2019-11-05 11:56:47,901 send_clipboard_request_handler(X11ClipboardProxy(CLIPBOARD), 'CLIPBOARD', 'UTF8_STRING')
2019-11-05 11:56:47,902 send_clipboard_request id=4
2019-11-05 11:56:47,912 process clipboard contents, selection=CLIPBOARD, type=UTF8_STRING, format=8
2019-11-05 11:56:47,912 wire selection to raw, encoding=bytes, type=UTF8_STRING, format=8, len(data)=4
2019-11-05 11:56:47,912 clipboard wire -> raw: ('UTF8_STRING', 8, 'bytes', '"b\'test\'"') -> "b'test'"
2019-11-05 11:56:47,913 clipboard got contents(4, 'UTF8_STRING', 8, '"b\'test\'"'): proxy=X11ClipboardProxy(CLIPBOARD) for selection=CLIPBOARD
2019-11-05 11:56:47,913 got_contents('UTF8_STRING', 'UTF8_STRING', 8, '"b\'test\'"') pending=(<GdkX11.X11Window object at 0x7f97ac52ba00 (GdkX11Window at 0x55fa573d5450)>, 'GDK_SELECTION', 1503780742)
2019-11-05 11:56:47,913 setting response 'test' to property GDK_SELECTION of window 'gedit' as UTF8_STRING
2019-11-05 11:56:47,913 set_selection_response(<GdkX11.X11Window object at 0x7f97ac52ba00 (GdkX11Window at 0x55fa573d5450)>, UTF8_STRING, GDK_SELECTION, UTF8_STRING, 8, "'test'", 1503780742)
2019-11-05 11:56:47,914 do_xpra_selection_request(<X11:SelectionRequest {'send_event': '0', 'serial': '0x12b4', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'TARGETS', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,914 do_selection_request_event(<X11:SelectionRequest {'send_event': '0', 'serial': '0x12b4', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'TARGETS', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,915 clipboard request for CLIPBOARD from window 0xa00124: 'gedit', target=TARGETS, prop=GDK_SELECTION
2019-11-05 11:56:47,915 using existing TARGETS value as response: ('UTF8_STRING', 'text/plain')
2019-11-05 11:56:47,916 set_selection_response(<GdkX11.X11Window object at 0x7f97ac52ba00 (GdkX11Window at 0x55fa573d5450)>, TARGETS, GDK_SELECTION, ATOM, 32, "'ç\\x00\\x00\\x00\\x00\\x00\\x00\\x00¯\\x01\\x00\\x00\\x00\\x00\\x00\\x00'", 1503780742)
2019-11-05 11:56:47,916 do_xpra_selection_request(<X11:SelectionRequest {'send_event': '0', 'serial': '0x12b5', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa0078a', 'selection': 'CLIPBOARD', 'target': 'TARGETS', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,916 do_selection_request_event(<X11:SelectionRequest {'send_event': '0', 'serial': '0x12b5', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa0078a', 'selection': 'CLIPBOARD', 'target': 'TARGETS', 'property': 'GDK_SELECTION', 'time': '1503780742'}>)
2019-11-05 11:56:47,917 clipboard request for CLIPBOARD from window 0xa0078a: 'gedit', target=TARGETS, prop=GDK_SELECTION
2019-11-05 11:56:47,917 using existing TARGETS value as response: ('UTF8_STRING', 'text/plain')
2019-11-05 11:56:47,917 set_selection_response(<GdkX11.X11Window object at 0x7f97ac52ba00 (GdkX11Window at 0x55fa573d5c70)>, TARGETS, GDK_SELECTION, ATOM, 32, "'ç\\x00\\x00\\x00\\x00\\x00\\x00\\x00¯\\x01\\x00\\x00\\x00\\x00\\x00\\x00'", 1503780742)

... and you can see that the text (test) is sync'd -
2019-11-05 11:56:47,913 clipboard got contents(4, 'UTF8_STRING', 8, '"b\'test\'"'): proxy=X11ClipboardProxy(CLIPBOARD) for selection=CLIPBOARD

When I then try to highlight and copy that text that was just pasted though, I get this... and from this point on the clipboard server side doesn't seem to be able to sync with anything.

2019-11-05 11:56:59,679 do_xpra_selection_clear(<X11:SelectionClear {'send_event': '0', 'serial': '0x1776', 'delivered_to': '0x400008', 'window': '0x400008', 'selection': 'PRIMARY', 'time': '1503792521'}>)
2019-11-05 11:56:59,679 do_xpra_selection_clear(<X11:SelectionClear {'send_event': '0', 'serial': '0x1776', 'delivered_to': '0x400008', 'window': '0x400008', 'selection': 'PRIMARY', 'time': '1503792521'}>) was owned=True
2019-11-05 11:56:59,679 do_owner_changed()
2019-11-05 11:56:59,679 do_xpra_xfixes_selection_notify_event(<X11:XFSelectionNotify {'send_event': '0', 'serial': '0x1776', 'delivered_to': '0x400008', 'window': '0x400008', 'subtype': '0', 'owner': '0xa00124', 'selection': 'PRIMARY', 'timestamp': '1503792525', 'selection_timestamp': '1503792521'}>)
2019-11-05 11:56:59,679 do_selection_notify_event(<X11:XFSelectionNotify {'send_event': '0', 'serial': '0x1776', 'delivered_to': '0x400008', 'window': '0x400008', 'subtype': '0', 'owner': '0xa00124', 'selection': 'PRIMARY', 'timestamp': '1503792525', 'selection_timestamp': '1503792521'}>) owned=False, was False, enabled=True, can-send=True
2019-11-05 11:56:59,679 get_contents(TARGETS, <function ClipboardProxy.schedule_emit_token.<locals>.got_targets at 0x7f97ac53a680>, 0) owned=False, have-token=False
2019-11-05 11:56:59,680 requesting local XConvertSelection from 'gedit' as 'TARGETS' into 'PRIMARY-TARGETS'
2019-11-05 11:56:59,680 do_xpra_selection_request(<X11:SelectionRequest {'send_event': '0', 'serial': '0x1779', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'TARGETS', 'property': 'GDK_SELECTION', 'time': '1503792521'}>)
2019-11-05 11:56:59,680 do_selection_request_event(<X11:SelectionRequest {'send_event': '0', 'serial': '0x1779', 'delivered_to': '0x400008', 'window': '0x400008', 'requestor': '0xa00124', 'selection': 'CLIPBOARD', 'target': 'TARGETS', 'property': 'GDK_SELECTION', 'time': '1503792521'}>)
2019-11-05 11:56:59,681 clipboard request for CLIPBOARD from window 0xa00124: 'gedit', target=TARGETS, prop=GDK_SELECTION
2019-11-05 11:56:59,681 using existing TARGETS value as response: ('UTF8_STRING', 'text/plain')
2019-11-05 11:56:59,681 set_selection_response(<GdkX11.X11Window object at 0x7f97ac539c80 (GdkX11Window at 0x55fa573d5450)>, TARGETS, GDK_SELECTION, ATOM, 32, "'ç\\x00\\x00\\x00\\x00\\x00\\x00\\x00¯\\x01\\x00\\x00\\x00\\x00\\x00\\x00'", 1503792521)
2019-11-05 11:56:59,682 do_xpra_property_notify_event(<X11:PropertyNotify {'send_event': '0', 'serial': '0x177e', 'delivered_to': '0x400008', 'window': '0x400008', 'atom': 'PRIMARY-TARGETS', 'time': '1503792527'}>)
2019-11-05 11:56:59,682 do_property_notify(<X11:PropertyNotify {'send_event': '0', 'serial': '0x177e', 'delivered_to': '0x400008', 'window': '0x400008', 'atom': 'PRIMARY-TARGETS', 'time': '1503792527'}>)
2019-11-05 11:56:59,683 PRIMARY-TARGETS='´\x01\x00\x00\x00\x00\x00\x00¬\x01\x00\x00\x00\x00\x00\x00µ\x01\x00\x00\x00\x00\x00\x00°\x01\x00\x00\x00\x00\x00\x00¶\x01\x00\x00\x00\x00\x00\x00ç\x00\x00\x00\x00\x00\x00\x00·\x01\x00\x00\x00\x00\x00\x00¸\x01\x00\x00\x00\x00\x00\x00\x1f\x00\x00\x00\x00\x00\x00\x00²\x01\x00\x00\x00\x00\x00\x00¯\x01\x00\x00\x00\x00\x00\x00' (ATOM : 32)
2019-11-05 11:56:59,683 filter_data(TARGETS, ATOM, 32, ..)
2019-11-05 11:56:59,683 got_local_contents: calling <function ClipboardProxy.schedule_emit_token.<locals>.got_targets at 0x7f97ac53a680>('ATOM', 32, '"b\'\\\\xb4\\\\x01\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\xac .. 0\\\\x00\\\\xaf\\\\x01\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\'"'), time=0
2019-11-05 11:56:59,683 got_targets: ('TIMESTAMP', 'TARGETS', 'MULTIPLE', 'GTK_TEXT_BUFFER_CONTENTS', 'application/x-gtk-text-buffer-rich-text', 'UTF8_STRING', 'COMPOUND_TEXT', 'TEXT', 'STRING', 'text/plain;charset=utf-8', 'text/plain')
2019-11-05 11:56:59,683 get_contents(UTF8_STRING, <function ClipboardProxy.schedule_emit_token.<locals>.with_targets.<locals>.got_text_target at 0x7f97ac53ab00>, 0) owned=False, have-token=False
2019-11-05 11:56:59,684 requesting local XConvertSelection from 'gedit' as 'UTF8_STRING' into 'PRIMARY-UTF8_STRING'
2019-11-05 11:56:59,684 do_xpra_property_notify_event(<X11:PropertyNotify {'send_event': '0', 'serial': '0x178a', 'delivered_to': '0x400008', 'window': '0x400008', 'atom': 'PRIMARY-TARGETS', 'time': '1503792529'}>)
2019-11-05 11:56:59,684 do_property_notify(<X11:PropertyNotify {'send_event': '0', 'serial': '0x178a', 'delivered_to': '0x400008', 'window': '0x400008', 'atom': 'PRIMARY-TARGETS', 'time': '1503792529'}>)
2019-11-05 11:56:59,684 do_property_notify() property 'PRIMARY-TARGETS' is gone?
Traceback (most recent call last):
  File "/usr/lib64/python3.7/site-packages/xpra/x11/gtk_x11/clipboard.py", line 578, in do_property_notify
    dtype, dformat = X11Window.GetWindowPropertyType(self.xid, event.atom, True)
  File "xpra/x11/bindings/window_bindings.pyx", line 1173, in xpra.x11.bindings.window_bindings.X11WindowBindingsInstance.GetWindowPropertyType
xpra.x11.bindings.window_bindings.BadPropertyType: None type

Trying to repeat the behavior though, I saw this long string of debug output, which seems to show multiple clipboard sync events as I try to highlight 'test3' ... with multiple events with just the 't' being sync'd, then multiple with 'te', etc... which seems to explain the more than 30 calls per second message - but oddly in this case the copy succeeded and I was able to paste it into a local application:
[/attachment/ticket/1844/multiple-clipboard-sync-events.txt]

I assume the use of the test 'UTF8' didn't break the sync of the text before and including that? I'm not sure I'm seeing any other clear pattern though.

[totaam]

2019-11-06 02:33:48: antoine changed status from new to assigned

[totaam]

2019-11-06 02:33:48: antoine commented

---

It looks like I'm still very easily able to trigger the more than 30 requests per second error when testing with gedit
As of v3, this is no longer an error. It is still logged as a warning but it will no longer disable the clipboard when it happens since the new code is immune to the crashes that the old code was susceptible to.

it looks like there are some tracebacks
xpra.x11.bindings.window_bindings.BadPropertyType: None type
Those are not actual problems, that's just the way we detect that the property has been deleted.

When I then try to highlight and copy that text that was just pasted though, I get this... and from this point on the clipboard server side doesn't seem to be able to sync with anything.
Worked for me, will try again.

but when I pasted it locally, I got (with the leading spaces) test1 test3 test5.
Did you select it from right to left?
Maybe it had the previous selection instead of the whole one?

[totaam]

2019-11-08 00:33:30: afarr commented

---

Ok, restested a bit - but I'm still using 3.0.2-24387.

Launched again with same command.

xpra start :13 --no-daemon --bind-tcp=0.0.0.0:1234 --bind-ws=0.0.0.0:1237 --start-child=xterm --start-child=xterm --exit-with-children -d clipboard

Some additional testing to see what happened to the rest of the copied string - I find that I'm able to reproduce it irregularly, but I think it is me. In some cases it seems like an attempted highlighting that keeps the left-click depressed as the mouse is dragged off the edge of the html5 client browser canvass doesn't register an un-click, which seems to cause some erratic behavior when using the control-c to copy near the edge of the browser canvas... intermittently. (No logs to reflect this, but I noticed that after one attempt to paste in a local application, when the mouse was hovered back over the session canvas it highlighted as I moved the mouse as if the left-click button were depressed, despite it's not being.)

I am able to consistently trigger a mis-sync when copying something in session, then copying something locally, then pasting those contents back to the session - whether I use the right-click menu or the control-c (again on server-side gedit... I think I see this same behavior only with right-click paste events in server-side chromium-browser or firefox tabs, like a translate page).

The logs for the copy and paste events server side become long, so I'll attach a txt file with the repro steps and the logs as they are gleaned - but I'll provide the repro steps here for greater ease.

• Launch a session.

• Launch gedit from an xterm (obviously could start with it as a start-child instead).
• Type something into the gedit application server-side ('test7' for example).
• Paste the copied text into local application (Word for example) - works as expected.
• Copy something else locally (logs from the xpra session, whatever).
• Paste this new content into the gedit application server-side - instead of the expected content, the content copied last server-side is pasted ('test7' for example).
• Paste again in the gedit application window without triggering any new clipboard events in the meantime (pretend you were probably just hallucinating what just happened) - this time the content copied locally pastes into the gedit application server-side.

I'm not sure if the initial copy server-side (to 'initialize' the server clipboard with some contents) is necessary or not to trigger this behavior (it wouldn't surprise me, but didn't get time to try that permutation).

I have no idea what I might have done previously to result in the server-side clipboard stopping all sync'ing with the local clipboard changes... so I'll assume I didn't and the above behaviors were just having a particularly good time at my expense.

[totaam]

2019-11-08 15:44:38: antoine changed status from assigned to new

[totaam]

2019-11-08 15:44:38: antoine changed owner from antoine to afarr

[totaam]

2019-11-08 15:44:38: antoine commented

---

As usual, great catch!

it seems like an attempted highlighting that keeps the left-click depressed as the mouse is dragged off the edge of the html5 client browser canvas doesn't register an un-click, which seems to cause some erratic behavior when using the control-c to copy near the edge of the browser canvas...
Yes, that will do it. Not sure there's much we can do about this one either.
This is similar to the problems we used to have with people Alt-Tabbing away, with the Alt key remaining pressed after they switched back. Except we can query the state of the modifier keys to get things back in sync, but we can't query the buttons state (we only get button events), especially not with an html5 client.

Paste this new content into the gedit application server-side - instead of the expected content, the content copied last server-side is pasted
That worked for me (as usual!), until I used a click on the browser's top bar or used Alt-Tab to switch to the browser window instead of clicking within the browser window.. The click was polling the clipboard and syncing it. FYI: I mostly use win7 through virtualbox, so unless I lock the VM with the "host key", Alt-Tab switches away from the VM, not to another window within that VM window.

And now the fixes:

• r24411 now polls the clipboard whenever we get a keyboard event - google chrome seems to allow that
• r24412 now also polls the clipboard whenever we get a right click

You can test this trivial fix by hand (just one line added, the other one is just a comment) or use the latest beta 4.0 builds I've just uploaded.
I will test some more with other browsers (IE, Firefox, etc) before backporting this to the v3.0.x branch.

[totaam]

2019-11-12 21:36:21: afarr changed owner from afarr to antoine

[totaam]

2019-11-12 21:36:21: afarr commented

---

Had to test with 4.0-24112 (couldn't find an html5/js/Client.js in the 3.0.2 for r24411 and the line numbers for the index.html in r24412 didn't match the 3.0.2 so I couldn't figure out where to move the code bit to...) - but the fix does the trick.

I'm still seeing the issue with the right-click pasting session-side not picking up the newly copied content locally (until after a keyboard shortcut paste, which sync's the new content)... but otherwise the clipboard stands up to all my tricks.

I'll assign this back to you to either close or re-assign back for a final re-testing when backported to the 3.x.

[totaam]

2019-11-13 08:34:40: antoine changed status from new to assigned

[totaam]

2019-11-13 08:34:40: antoine commented

---

Had to test with 4.0-24112
I am going to assume that this is a typo and that you mean r24412 instead, right? (24112 is old!)
That's fine for testing.

couldn't find an html5/js/Client.js in the 3.0.2 for r24411 and the line numbers for the index.html in r24412 didn't match the 3.0.2
I see, the patch context if really not helpful at all here.
FYI: not sure if that's what you did here, you can always update xpra-html5 from the beta repo (manually downloading the RPM or telling dnf to use the beta repo only for upgrading this particular package) and you can keep the rest of your system on the stable branch. (but if you do that, you have to remember that the package is newer because the 3.0.x stable updates won't upgrade it... or you can tell dnf to downgrade it when you're done testing)
Another option, which is not normally that difficult, is to browse the trunk source to figure out the patch context: [/browser/xpra/trunk/src/html5]. But in this particular case, the whole context had been moved too (converted to a new function), so not that easy.
That's all academic now really since I have merged the patches: 24419.
So now you can also just download the latest version of the files you need from the 3.0.x branch: [/browser/xpra/tags/v3.0.x/src/html5].

I'm still seeing the issue with the right-click pasting session-side not picking up the newly copied content locally
Tried with a Linux client and it worked every time... as usual.
But there is something wrong when using a win32 client, that's because the target is not supported by the html5 client:

clipboard request for CLIPBOARD from window 0x80012f: 'gedit', target=TARGETS, prop=GDK_SELECTION
using existing TARGETS value as response: ('UTF8_STRING', 'text/plain')
clipboard request for CLIPBOARD from window 0x80012f: 'gedit', target=GTK_TEXT_BUFFER_CONTENTS, prop=GDK_SELECTION
client is requesting an unknown target: 'GTK_TEXT_BUFFER_CONTENTS'
 valid targets: UTF8_STRING, text/plain
 dropping the request
client is requesting an unknown target: 'text/plain;charset=utf-8'
 valid targets: UTF8_STRING, text/plain
 dropping the request

The first one, we have every right to drop, GTK_TEXT_BUFFER_CONTENTS must be some kind of secret GTK magic handshake thing.
But text/plain;charset=utf-8, although it is also invalid since we never claimed to be able to handle it, we could return the UTF8_STRING value.
Will fix.

[totaam]

2019-11-13 11:40:08: antoine changed status from assigned to new

[totaam]

2019-11-13 11:40:08: antoine changed owner from antoine to afarr

[totaam]

2019-11-13 11:40:08: antoine commented

---

So, as of r24421 (server side fix) we now translate 'text/plain;charset=utf-8' to 'UTF8_STRING' or 'text/plain' automagically. More can be added easily.

As for the:

the right-click pasting session-side not picking up the newly copied content locally
are you certain that you are connecting to an https server? And that you accepted to grant clipboard access?
I am only seeing this problem when I connect over http, otherwise it works reliably, even on win32.
Over https, with "clipboard" debugging enabled in the html5 client, the server's debug log shows:

clipboard polling using function () { [native code] }
clipboard paste event, text= foo
clipboard clipboard contents have changed
clipboard sending clipboard token with data: foo
clipboard polling using function () { [native code] }
clipboard polling using function () { [native code] }

Whereas when connecting over http:

clipboard polling: no data available

(log message format has been improved in r24422)

Backports to v3: 24423 + 24424.
With these known caveats, I think we should close this ticket: it's as good as can be expected given the javascript / browser limitations we have to work with.

I will upload beta builds later.

See also related ticket: #12.

[totaam]

2019-11-13 23:44:56: afarr changed status from new to closed

[totaam]

2019-11-13 23:44:56: afarr set resolution to fixed

[totaam]

2019-11-13 23:44:56: afarr commented

---

Ahh... guilty as charged - I had forgotten and was indeed using an http connection.

Re-acquainting myself with the server flags, and process for generation of a non-localhost self-signed cert... I launched a new session of 4.0-r24425 with the new command of:

xpra start :13 --no-daemon --bind-tcp=0.0.0.0:1234 --bind-ws=0.0.0.0:1237 --bind-wss=0.0.0.0:1239 --ssl-cert=/etc/xpra/148-ssl.pem  --start-child=xterm --start-child=xterm --exit-with-children -d clipboard

Then imported the self-signed cert locally (and even set an \Windows\System32\drivers\etc\hosts entry with DNS)... and still had to click through the warning (Chrome and Firefox both).

Then I (finally) got the popup asking to give permission to access the clipboard.

Once that was done - sure enough even the right-click paste locally from a copy server-side is working consistently and on the first try.

All that said (in case I need to work through the process again for some reason) - I agree. Closing this. (I'll roll back to 3.0.2 to test backports, and re-open if I find something... though I don't expect to.)

[totaam]

2019-11-14 04:57:11: antoine uploaded file multiple-clipboard-sync-events.txt (150.5 KiB)

moving to an attachment: multiple clipboard sync events