osx shadow server improvements: mdns, keyboard support, etc #391

totaam · 2013-07-18T08:38:47Z

Issue migrated from trac ticket # 391

component: server | priority: minor | resolution: fixed | keywords: osx

2013-07-18 08:38:47: antoine created the issue

The OSX shadow server needs similar fixes to the win32 one (#389), namely:

keyboard

faster screenscraping

grab directly to gl? glgrab.c

damage notifications:

CGRegisterScreenRefreshCallback

register for WindowServer CGScreenRefreshCallback CGScreenUpdateMoveCallback in C++

maybe even export individual windows: Son of Grab / gdk_quartz_window_get_nswindow

totaam · 2013-07-18T08:50:35Z

2013-07-18 08:50:35: antoine edited the issue description

totaam · 2014-08-19T05:06:23Z

2014-08-19 05:06:23: totaam commented

Scheduling for 0.16

totaam · 2015-10-23T08:05:22Z

2015-10-23 08:05:22: antoine commented

Re-scheduling.

totaam · 2016-01-21T22:55:31Z

2016-01-21 22:55:31: antoine changed status from new to assigned

totaam · 2016-01-21T22:55:31Z

2016-01-21 22:55:31: antoine commented

Running from a non GUI terminal bails out with this ugly message, which should be turned into a more friendly init check:

xpra main error:
Traceback (most recent call last):
  File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/scripts/main.py", line 124, in main
    return run_mode(script_file, err, options, args, mode, defaults)
  File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/scripts/main.py", line 919, in run_mode
    return run_server(error_cb, options, mode, script_file, args)
  File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/scripts/server.py", line 927, in run_server
    app = ShadowServer()
  File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/shadow_server.py", line 48, in __init__
    raise Exception("cannot grab test screenshot - maybe you need to run this command whilst logged in via the UI")
Exception: cannot grab test screenshot - maybe you need to run this command whilst logged in via the UI

totaam · 2016-01-21T23:17:44Z

2016-01-21 23:17:44: antoine commented

Not sure if this is specific to OSX, but we should skip the stacktrace:

failed to create socket /var/tmp/502-Xpra/osx-2.local-0
Traceback (most recent call last):
  File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/scripts/server.py", line 357, in setup_local_sockets
    sock, cleanup_socket = create_unix_domain_socket(sockpath, mmap_group, socket_permissions)
  File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/scripts/server.py", line 231, in create_unix_domain_socket
    listener.bind(sockpath)
  File "socket.pyc", line 228, in meth
error: [Errno 13] Permission denied
xpra initialization error:
 failed to create socket /var/tmp/502-Xpra/osx-2.local-0
closing tcp socket 0.0.0.0:10000

totaam · 2016-01-21T23:53:07Z

2016-01-21 23:53:07: antoine commented

Some fixes:

as of r11712 + r11714, the error from comment:4 looks like this:
cannot grab pixels from the screen, make sure this command is launched from a GUI session
r11714 also fixes sound forwarding from osx and win32 shadow servers (and therefore microphone forwarding too)

r11711 removes the full stacktrace from the error in comment:5

r11713 fixes the underlying error: the default xpra config was including a path specific to the machine it is built on! oops (part of the changes for multiple bind options for local sockets #963)

New TODO: we should either ship the pybonjour mdns wrapper (as done with winswitch), or not bother trying avahi to avoid this message:
Warning: failed to load the mdns avahi publisher: No module named avahi
 either fix your installation or use the 'mdns=no' option

totaam · 2016-03-28T03:25:29Z

2016-03-28 03:25:29: antoine commented

r12246: the osx shadow server also has a systray

r12231: publishes the shadow server via mdns using pybonjour

totaam · 2016-03-28T04:15:16Z

2016-03-28 04:15:16: antoine edited the issue description

totaam · 2016-03-28T04:15:16Z

2016-03-28 04:15:16: antoine commented

Important fix to backport (keypresses cause a crash!) in r.

The CGRegisterScreenRefreshCallback is deprecated in 10.8, instead we're supposed to use the Streaming the Contents of a Display API, which can send us frames in BGRA, YUV420, ..

Using Quartz API, CGRegisterScreenRefreshCallback should be able to replace the timer, and look like this (adapted from the pyobjc tests):
from Quartz.CoreGraphics import CGRegisterScreenRefreshCallback, CGRect
myInfo = object()
callcount = [0]
def callbackRefresh(count, rects, info):
    assert info is myInfo
    assert isinstance(rects, tuple)
    assert isinstance(count, (int, long))
    for i in rects:
        assert isinstance(i, CGRect)
        callcount[0] += 1
err = CGRegisterScreenRefreshCallback(callbackRefresh, myInfo)
assert err==0
...
CGUnregisterScreenRefreshCallback(callbackRefresh, myInfo)
Also useful:

CGCursorIsVisible()

CGCursorIsDrawnInFramebuffer

CGWaitForScreenRefreshRects - crashes apparently

pykeycode for converting keycodes to strings

the toKeyCode function from this answer to "Generate keyboard events"

How can I call CGEventKeyboardSetUnicodeString from python? for passing specific characters as key events?

The keymapping is not implemented yet..

totaam · 2016-03-28T05:54:17Z

2016-03-28 05:54:17: antoine commented

osx shadow server fixes: r12250 (crash!), r12254 (fix control-c hang)

damage notifications implemented in r12252!

The printing stuff is blocked by #641: we need the installer to add the cups backend script to the system.

totaam · 2016-04-07T12:32:45Z

2016-04-07 12:32:45: antoine commented

Note about selecting the audio device on OSX: osxaudiosrc/osaudiosink uses non-unique identifiers as device ID

totaam · 2016-04-12T13:20:38Z

2016-04-12 13:20:38: antoine commented

A lot of progress in #641 / r12376:

we have a PKG

man page is installed

cups backend (needs testing)

/usr/bin wrappers

In order to support launching a remote shadow server via ssh (ie: xpra shadow ssh:OSXIP), we would need to run as an agent:

Creating Launch Daemons and Agents

Technical Note TN2083 - Daemons and Agents: Agents

Maybe the agent can just sit there, listening for events and we just tell it what to run (ie: xpra shadow "$@"?) - which will then run from the correct context and be able to access the display server (avoiding the dreaded _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.): see #1169.

totaam · 2016-04-16T08:55:51Z

2016-04-16 08:55:51: antoine changed status from assigned to new

totaam · 2016-04-16T08:55:51Z

2016-04-16 08:55:51: antoine changed owner from antoine to afarr

totaam · 2016-04-16T08:55:51Z

2016-04-16 08:55:51: antoine commented

r12390 switches to gstreamer 1.6 (see switch to gstreamer 1.6.x on osx #1069)

r12394 will make us use the osxaudiosrc ahead of autoaudiosrc

r12395 + r12393 + r12396 + r12398: add very basic shadow keymap support (hardcoded) - will follow up in osx shadow keyboard layout support #1171

r12397 disables printer forwarding with shadow servers (fails silently - why was I expecting anything more helpful from apple?) - will follow up in fix printer sharing with osx shadow servers #1170

r12391 fixes a hard crash in netifaces.gateways()

r12392 should workaround NSCFData being passed to the network layer (but apparently does not.. will fix once I manage to reproduce reliably)

the ability to start a shadow via ssh is in xpra agent for osx - ssh start support #1169

keyboard improvements is now in untranslated keyboard #1172

sound in fix sound forwarding for osx and win32 shadow servers #1205

@afarr: this is a FYI, feel free to close.

totaam · 2016-04-16T08:57:32Z

2016-04-16 08:57:32: antoine changed title from osx shadow server improvements to osx shadow server improvements: mdns, keyboard support, etc

totaam · 2016-05-20T00:06:32Z

2016-05-20 00:06:32: afarr changed status from new to closed

totaam · 2016-05-20T00:06:32Z

2016-05-20 00:06:32: afarr set resolution to fixed

totaam · 2016-05-20T00:06:32Z

2016-05-20 00:06:32: afarr commented

Ok, thanks for info.

I actually gave it a shot, some luck with tcp, not so much with ssh... details of that failure in #1169.

This one looks like it's time it got closed.

totaam · 2016-05-20T07:04:33Z

2016-05-20 07:04:33: antoine commented

You must be doing something wrong.
I've just re-tested against 10.6.x, 10.9.x and 10.10.x and it shadowed the existing display in all cases.

totaam · 2016-09-22T11:21:39Z

2016-09-22 11:21:39: antoine commented

Follow up: #1321, #1322

totaam closed this as completed May 20, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

osx shadow server improvements: mdns, keyboard support, etc #391

osx shadow server improvements: mdns, keyboard support, etc #391

totaam commented Jul 18, 2013

totaam commented Jul 18, 2013

totaam commented Aug 19, 2014

totaam commented Oct 23, 2015

totaam commented Jan 21, 2016

totaam commented Jan 21, 2016

totaam commented Jan 21, 2016

totaam commented Jan 21, 2016

totaam commented Mar 28, 2016

totaam commented Mar 28, 2016

totaam commented Mar 28, 2016

totaam commented Mar 28, 2016

totaam commented Apr 7, 2016

totaam commented Apr 12, 2016

totaam commented Apr 16, 2016

totaam commented Apr 16, 2016

totaam commented Apr 16, 2016

totaam commented Apr 16, 2016

totaam commented May 20, 2016

totaam commented May 20, 2016

totaam commented May 20, 2016

totaam commented May 20, 2016

totaam commented Sep 22, 2016

osx shadow server improvements: mdns, keyboard support, etc #391

osx shadow server improvements: mdns, keyboard support, etc #391

Comments

totaam commented Jul 18, 2013

2013-07-18 08:38:47: antoine created the issue

totaam commented Jul 18, 2013

2013-07-18 08:50:35: antoine edited the issue description

totaam commented Aug 19, 2014

2014-08-19 05:06:23: totaam commented

totaam commented Oct 23, 2015

2015-10-23 08:05:22: antoine commented

totaam commented Jan 21, 2016

2016-01-21 22:55:31: antoine changed status from new to assigned

totaam commented Jan 21, 2016

2016-01-21 22:55:31: antoine commented

totaam commented Jan 21, 2016

2016-01-21 23:17:44: antoine commented

totaam commented Jan 21, 2016

2016-01-21 23:53:07: antoine commented

totaam commented Mar 28, 2016

2016-03-28 03:25:29: antoine commented

totaam commented Mar 28, 2016

2016-03-28 04:15:16: antoine edited the issue description

totaam commented Mar 28, 2016

2016-03-28 04:15:16: antoine commented

totaam commented Mar 28, 2016

2016-03-28 05:54:17: antoine commented

totaam commented Apr 7, 2016

2016-04-07 12:32:45: antoine commented

totaam commented Apr 12, 2016

2016-04-12 13:20:38: antoine commented

totaam commented Apr 16, 2016

2016-04-16 08:55:51: antoine changed status from assigned to new

totaam commented Apr 16, 2016

2016-04-16 08:55:51: antoine changed owner from antoine to afarr

totaam commented Apr 16, 2016

2016-04-16 08:55:51: antoine commented

totaam commented Apr 16, 2016

2016-04-16 08:57:32: antoine changed title from osx shadow server improvements to osx shadow server improvements: mdns, keyboard support, etc

totaam commented May 20, 2016

2016-05-20 00:06:32: afarr changed status from new to closed

totaam commented May 20, 2016

2016-05-20 00:06:32: afarr set resolution to fixed

totaam commented May 20, 2016

2016-05-20 00:06:32: afarr commented

totaam commented May 20, 2016

2016-05-20 07:04:33: antoine commented

totaam commented Sep 22, 2016

2016-09-22 11:21:39: antoine commented