Running with --hostdisplay on TurboVNC with full Xfce desktop environment

[casper]

This may be a stupid question since it's the first time I'm using docker and x11docker, but is it possible for the docker image to run Xfce on the host display?

I'm using Ubuntu 14.04, and I'm running a TurboVNC session on this host.

I want to get x11docker/xfce-wine-playonlinux to run fully on this VNC display, including the Xfce window manager and desktop environment.

So I run it like this:
x11docker --desktop --hostdisplay x11docker/xfce-wine-playonlinux

This works, but I get no window manager. I can't drag and resize any windows, because there are no title bars or resize handles on them.

When I start x11docker it says:

x11docker note: Can not avoid to use host window manager
  along with option --hostdisplay. 
  You may get strange interferences with your host desktop.
  Can be interesting though, having two overlapping desktops.

I'm not running any wm on the TurboVNC display myself. It seems to imply it should work, but it doesn't seem to :-/

Is there any way to get this combo to run? My goal is to try and get 3D acceleration for Wine through the TurboVNC VirtualGL layer in the end. However first I need to get the desktop working somehow.

Thanks.

[mviereck]

An interesting question.

I'm not running any wm on the TurboVNC display myself. It seems to imply it should work, but it doesn't seem to :-/

I am a bit surprised that you get no window manager. If there is none on the TurboVNC display, xfwm4 from the xfce image should do the job.

Is there any way to get this combo to run?

It should work. You could run x11docker with option --verbose and copy x11docker.log to https://pastebin.com/ so I can have a look for failure messages. Or you look yourself for error messages from xfwm4. As a workaround, you can specify a host window manager with option --wm=auto.

My goal is to try and get 3D acceleration for Wine through the TurboVNC VirtualGL layer in the end.

x11docker has an option --gpu for hardware acceleration. You will need that anyway, with or without TurboVNC. For local access only, you can drop TurboVNC at all, it won't give additional help for acceleration. Or do you use TurboVNC for real remote sessions?

[casper]

Ok I ran it with the --verbose switch. There seems to be a lot of errors, but not sure which ones are the relevant ones:

http://paste.ubuntu.com/26154553/

I also see this error on stdout right at the beginning. Does not seem to appear in the log file:

xauth: (argv):1:  unable to read any entries from file "(stdin)"

x11docker has an option --gpu for hardware acceleration. You will need that anyway, with or without TurboVNC. For local access only, you can drop TurboVNC at all, it won't give additional help for acceleration. Or do you use TurboVNC for real remote sessions?

I use it for a real remote session. The host is a headless server, so I want everything running in the TurboVNC view.

However, now as I am investigating GPU acceleration in docker a bit further, perhaps I need to use a base image that is Ubuntu 14.04 also, in order to have matching GPU libs in the docker image?

Or maybe it's possible to get the matching libs into the x11docker playonlinux image too? Have to investigate this further. The host runs an i915 Intel GMA-based GPU. I have VirtualGL acceleration working fine with the TurboVNC display on the host right now at least.

[mviereck]

It seems the TurboVNC display misses a lot of important X extensions. For GPU acceleration, you need at least XRender and XComposite:

(xfwm4:40): xfwm4-WARNING **: The display does not support the XShape extension.
(xfwm4:40): xfwm4-WARNING **: The display does not support the XSync extension.
(xfwm4:40): xfwm4-WARNING **: The display does not support the XRender extension.
(xfwm4:40): xfwm4-WARNING **: The display does not support the XRandr extension.
(xfwm4:40): xfwm4-WARNING **: The display does not support the XComposite extension.
(xfwm4:40): xfwm4-WARNING **: The display does not support the XDamage extension.
(xfwm4:40): xfwm4-WARNING **: The display does not support the XFixes extension.
(xfwm4:40): xfwm4-WARNING **: Compositing manager disabled.
(xfwm4:40): xfwm4-WARNING **: Could not find a screen to manage, exiting

Is TurboVNC able to access an already running display? Than the solution would be to create a proper X display with x11docker that TurboVNC can send remotly.

In the readme I show a possible VNC setup. Though, I am not familar with VNC in general:

Sample setup for VNC access:

read Xenv < <(x11docker --xdummy  x11docker/lxde pcmanfm)
echo $Xenv && export $Xenv
x11vnc -localhost -noshm

In another terminal, start VNC viewer with vncviewer localhost:0.
See man x11vnc for many details and further infos.
Option -noshm disables shared memory (MIT-SHM). To allow shared memory, remove -noshm and use isolation breaking x11docker option --ipc.

The first step would be to get a working sample without GPU acceleration. If that works, we can have a deeper look at how to get the GPU running.

However, now as I am investigating GPU acceleration in docker a bit further, perhaps I need to use a base image that is Ubuntu 14.04 also, in order to have matching GPU libs in the docker image?

If you have open source drivers on your server, most probably it will work fine with the debian image.

Or maybe it's possible to get the matching libs into the x11docker playonlinux image too?

If the image misses some driver libs, you can add them with a custom Dockerfile:

FROM x11docker/xfce-wine-playonlinux
RUN apt-get update && apt-get upgrade -y
RUN apt-get install -y drivername

I have VirtualGL acceleration working fine with the TurboVNC display on the host right now at least.

Early versions of x11docker used VirtualGL, too, to allow GPU acceleration with Xephyr. But I found that immediate GPU access was quite more performant with later added options --weston-xwayland, --xdummy-xwayland and --xorg.

I will have a look at the xauth error; maybe TurboVNC does not provide an authentication cookie. x11docker should catch xauth errors and handle them.

[casper]

It should be possible to support GPU acceleration for 3D apps without those extensions though(?), as I've seen videos with this setup running 3D apps just fine.

However for 3D we only require OpenGL. I guess the desktop environment requires more advanced X extensions. I don't really have a clue about those. I'm really surprised though, as I would have assumed someone must have tested TurboVNC on a modern desktop environment before. I've run the Ubuntu desktop on VNC before without problems (natively without docker).

I found TigerVNC which seems to be able to attach to a native X display, and has a few more extensions, but with TigerVNC x11docker doesn't start at all:

http://paste.ubuntu.com/26157476/

[mviereck]

In the logfile I see some xauth/cookie errors. It seems I have to look deeper in this, maybe TigerVNC has no cookies, but x11docker does not recognize that and handles this wrong. You can try to use option --no-auth do disable cookie usage.

You still try to run x11docker on the display provided by TigerVNC with option --hostdisplay. I suggest the other way around: run x11docker with option --xdummy or --xvfb and let TigerVNC access the display created by x11docker. (Compare my x11vnc example above).

Side note: In your first logfile with TurboVNC there is another error message:

X Error of failed request:  BadAccess (attempt to access private resource denied)

This indicates that the TurboVNC display has extension MIT-SHM enabled; short said, it enables some shared memory that docker containers normally cannot access. Avoid this error with option --ipc.

[casper]

Ok thanks. I'll take a look at your suggestions. In the meantime I managed to get the native X running (X.Org X Server 1.15.1), and attached a TigerVNC viewer to that. However the result was the same as with TurboVNC (it works, but no window manager):

http://paste.ubuntu.com/26157861/

[casper]

Progress! Got the window manager working with TurboVNC and your suggestions:

x11docker --no-auth --ipc --verbose --desktop --hostdisplay x11docker/xfce-wine-playonlinux

The deciding switch was --no-auth. However everything is extremely sluggish. Much slower than without --no-auth and running a native wm instead. Seems everything is fully repainted, instead of optimized painting as before launching the docker wm. This results in 10x more network traffic, even from just moving the mouse around. But in any case the full desktop environment is up with this switch.

Still need to test out your other suggestions.

[mviereck]

Progress!

:-)

I just got a working local setup with TigerVNC:

read xenv < <(x11docker --xdummy  --desktop  x11docker/xfce)
env $xenv x0tigervncserver -SecurityTypes None
vncviewer localhost:0

Additionally using --ipc may speed this up.

Much slower than without --no-auth and running a native wm instead.

Do you mean, with a host window manager using --wm?

[casper]

Do you mean, with a host window manager using --wm?

No I wasn't using that option. When I was running without the --no-auth I just ran twm manually on the host to get a simple WM.

But I now figured out what was making the desktop slow. In the Xfce settings I disabled desktop composition, and that significantly sped up everything. I think VNC does not like compositing. It probably forces many more updates to be drawn, making everything sluggish. Disabling composition almost restores it to twm-like speed.

This is pretty cool now. Seems almost usable. Probably needs a bit more tweaking and tuning, but the important part is that it works :)

Now I need to figure out what your --xdummy version is doing. Is that a better (faster?) way to run everything?

[mviereck]

But I now figured out what was making the desktop slow. In the Xfce settings I disabled desktop composition, and that significantly sped up everything. I think VNC does not like compositing. It probably forces many more updates to be drawn, making everything sluggish. Disabling composition almost restores it to twm-like speed.

Good to know, thanks for pointing that out!

Now I need to figure out what your --xdummy version is doing. Is that a better (faster?) way to run everything?

It is not better or faster, it is just a simple invisible/headless X server like Xvfb, but using Xorg itself. It is not capable of GPU acceleration.

The next step is to get GPU to work. Do I understand right that you can use your graphics card without a monitor being plugged in? Most older cards can't.

If you can run a regular Xorg session (like with startx), the most performant setup should be

x11docker --xorg --gpu --ipc --desktop x11docker/xfce-wine-playonlinux

[casper]

Do I understand right that you can use your graphics card without a monitor being plugged in?

Yep. This is the trick with VirtualGL. The GPU requests are forwarded from whatever display you're using (for example a VNC session on a virtual display), to the :0 display and then sent back to the virtual display for drawing. In a sense you're "borrowing" the native display for the GPU acceleration part.

So yes, I am able to run a native desktop without any physical display being connected.

If you can run a regular Xorg session (like with startx), the most performant setup should be

x11docker --xorg --gpu --ipc --desktop x11docker/xfce-wine-playonlinux

What does this do? I assume it runs Xorg on the host, and then forwards everything from the docker container to it? Or is it running Xorg in the container somehow instead?

[mviereck]

I am a bit confused.
You are using VirtualGL to access a GPU that is not on your headless server, but on your VNC client?

I thought, you would use the GPU of the headless server itself. Some new graphics cards allow that, while older ones need a real monitor to work. (Older ones can be fooled with a fake monitor, a cheap VGA plug with a few chips simulating a real monitor.)

What does this do? I assume it runs Xorg on the host, and then forwards everything from the docker container to it?

--xorg (and all other x11docker X options) runs the X server on the host, in your case on the headless server. x11docker shares the X unix socket to allow docker applications immediate access to the X server.

If your headless server has a GPU and it works without a real monitor, a setup with --xorg --gpu would provide much more performance than a VirtualGL setup.

[casper]

I thought, you would use the GPU of the headless server itself. Some new graphics cards allow that, while older ones need a real monitor to work. (Older ones can be tricked out with a fake monitor, a cheap VGA plug with a few chips simulating a real monitor.)

Yeah sorry I wasn't clear. This is the way I was planning to use it. Headless using the host GPU. I just wasn't aware of TigerVNC and x0vncserver before, so I was using TurboVNC with VirtualGL instead (and TurboVNC/VirtualGL was then accessing Xorg on the host for GPU operations, but not for anything else).

Now since TigerVNC allows direct attachment to the native Xorg through x0vncserver, I think your --xorg suggestion will be the one I will attempt to get working.

Only problem is I just stumbled on a bug in my current kernel that crashes the Intel GPU drivers when running any 3D apps...doh! So I need to update the kernel and reboot before I can proceed further. But this is a server with other people on it, so I'll have to do that later.

In any case thanks a lot for your help! Once I get the updated kernel running I'll let you know how it went.

[mviereck]

Only problem is I just stumbled on a bug in my current kernel that crashes the Intel GPU drivers when running any 3D apps.

Does the setup work with --xorg but without option --gpu? Then it will most probably work with --gpu after you upgraded the kernel.

Once I get the updated kernel running I'll let you know how it went.

Yeah, I am quite interested!

Maybe TurboVNC allows attaching to existing X servers, too, I just did not find the possibility on the first glance. It seems to me that TurboVNC is better maintained than TigerVNC. The developer of TurboVNC is also the developer of VirtualGL. He was very kind when I asked some questions years ago.

Oh, and an important note: You need option --showenv to get the X environment variables for the VNC server. With this option, x11docker prints them on stdout. (Per default, x11docker does this only with --xvfb and --xdummy).

read xenv < <(x11docker --xorg --gpu --ipc --desktop --showenv x11docker/xfce-wine-playonlinux)

[casper]

Does the setup work with --xorg but without option --gpu?

Nope :-/ I'm stuck again.

x11docker ERROR: Startup of X server --xorg failed.
  Last lines of xinit logfile:
(EE) Cannot open log file "/var/log/Xorg.8.log"

I think I need to run X as root. Should I run x11docker with sudo? That seems kind of risky.

[mviereck]

Cannot open log file "/var/log/Xorg.8.log"

Strange ... never had that error message before.

Should I run x11docker with sudo? That seems kind of risky.

That is not risky, but would not help either :-D. x11docker runs X as non-privileged user even if it was started as root.

From Xorg man page:

-logfile filename
Use the file called filename as the Xorg server log file. The default log file when running as root is /var/log/Xorg.n.log and for non root it is $XDG_DATA_HOME/xorg/Xorg.n.log where n is the
display number of the Xorg server. The default may be in a different directory on some platforms. This option is only available when the server is run as root (i.e, with real-uid 0).

It may help to set XDG_DATA_HOME. Try export XDG_DATA_HOME=$HOME/.cache. Otherwise, I could add a -logfile option for X in x11docker. It may work although the man page says it would work for root only.

[casper]

Nope. Same error. Is there any way I could pre-run X as root myself, and let x11docker use that display instead of x11docker starting X itself?

As I google about running X as non-root, I get pages upon pages of discussions, but could not find any real solution for it. It seems it is not a solved problem, as most people run X on the native display from the console, and it doesn't seem that it likes being run from a remote SSH session unless root.

[casper]

Easiest solution seems to let x11docker just sudo X instead, and always run it as root, or add an option to force it to sudo X?

[mviereck]

Before some recent changes Xorg was always started as root and had the setuid bit set. Ubuntu 16.04 and debian 9 can restore this behaviour with package xserver-xorg-legacy. I thought, it should not be needed (and not available) on Ubuntu 14.04.

Do you have a file /etc/X11/Xwrapper.config? If it has an entry allowed_users=console and you change it to allowed_users=anybody, it may work. Maybe it also needs line needs_root_rights=yes.

force it to sudo X?

That is already possible. You can run sudo x11docker --hostuser=root --user=$USER [...]. But I would prefer better solutions.

[casper]

The Xwrapper.config file was present, but changing it didn't help. I'm able to run X as non-root manually though, so it's kind of strange x11docker isn't able to do it.

[casper]

I also get these warnings. Especially the first one is not correct. It thinks it's starting within X, but it's started from within an SSH session.

x11docker WARNING: Environment variable DISPLAY is empty, but
  it seems x11docker was started within X, not from console.
  Please set DISPLAY and XAUTHORITY.
  If you have started x11docker with su or sudo, su/sudo may be configured to
  unset X environment variables. It may work if you run x11docker with
    sudo -E x11docker [...]
  If your system does not support 'sudo -E', you can try
    sudo env DISPLAY=$DISPLAY XAUTHORITY=$XAUTHORITY x11docker [...]
  Otherwise, you can use tools like gksu/gksudo/kdesu/kdesudo/lxsu/lxsudo.

x11docker note: You can switch between X servers and console terminals
  with [CTRL][ALT][F1]...[F12].

x11docker WARNING: On debian 9, switching often between multiple X servers can
  cause a crash of one X server. This bug may be debian specific and is
  probably some sort of race condition. If you know more about this or it
  occurs on other systems, too, please report.

x11docker WARNING: Security risk:
  Option --ipc breaks down container isolation!
  Drawback: IPC namespace remapping is disabled.
  Advantage: X extension MIT-SHM is possible.

x11docker WARNING: Although x11docker starts Xorg as unprivileged user,
  most system setups wrap Xorg to give it root permissions (setuid).
  Evil containers may try to abuse this.
  Other x11docker X server options like --xephyr are more secure at this point.

[mviereck]

I'm able to run X as non-root manually though, so it's kind of strange x11docker isn't able to do it.

Do you run X with X or with Xorg? There may be a difference, though it should not. Please show me the output of

ls -la /usr/bin/X
ls -la /usr/bin/Xorg
command -v Xorg
tty

x11docker checks the output of tty to see if it runs on console or not. I don't know what tty gives in a ssh session, your last post shows that x11docker misinterprets the output of tty.

[casper]

I run it with X.
From the output it seems X it's a setuid wrapper for Xorg.

-rwsr-sr-x 1 root root 10192 Jan 29  2014 /usr/bin/X
-rwxr-xr-x 1 root root 2331776 Jul 30  2014 /usr/bin/Xorg
/usr/bin/Xorg
/dev/pts/20

[mviereck]

I have uploaded a slightly changed x11docker here, please try out: https://raw.githubusercontent.com/mviereck/x11docker/experimental/x11docker
It runs your setuid /usr/bin/X instead of /usr/bin/Xorg.

If that works, you can try to undo your changes in Xwrapper.config and try again.

Thanks for the tty-output ... now I don't know how to divide between an X and an SSH session. :rolleyes:

[casper]

Progress again :) It works! However we still have some issues:

1. My Xorg does not support +glx, so I have to comment it out manually in the x11docker code.
2. I still have to use --no-auth, because otherwise when starting x0vncserver I get this:

Invalid MIT-MAGIC-COOKIE-1 key/opt/tigervnc/usr/bin/x0vncserver: unable to open display ":8"

3. The output of x11docker doesn't give any clue which display number it chose. I have to use ps to check where it went (:8 in this case), and then attach x0vncserver there.

4. The resulting X display is 320x200. I went back to running X on a mobile screen from the year 2003 :-/ Need some way to specify the size. I wish TigerVNC could resize it like it can with its own X server, but it doesn't seem to be able to do it while piggy backed to Xorg like this. Maybe there's some x*** command to resize a running X display?

[mviereck]

Progress again :) It works!

:-)

My Xorg does not support +glx

ok, I'll remove that option from the code. I assume, you mean +iglx?

The output of x11docker doesn't give any clue which display number it chose.

You need option --showenv. That gives you DISPLAY and XAUTHORITY. With XAUTHORITY provided to x0vncserver, you can drop --no-auth.

The resulting X display is 320x200. I went back to running X on a mobile screen from the year 2003

LOL. Try if xrandr shows you some useable resolutions. You can specify a resolution with x11docker option --size=1280x800.

Maybe there's some x*** command to resize a running X display?

xrandr allows that. You can also change the resolution with the xfce display settings from inside the docker container.

[casper]

ok, I'll remove that option from the code. I assume, you mean +iglx?

Yes.

--showenv seems to work. --size does not. Perhaps this update x11docker script does not propagate size to the sudo X command?

Spent 30 minutes fiddling with xrandr, manually adding a modeline. It crashed the X server in the end. No go. xrandr by itself lists no available resolutions.

[mviereck]

Perhaps this update x11docker script does not propagate size to the sudo X command?

x11docker tries to set the --size resolution or to do some workarounds to get it. But it currently does not create modelines. It only uses resolutions already shown by xrandr.

Can you show me the output of xrandr ?

A workaround with weston and Xwayland is possible, but that adds some overhead I would like to avoid. Plain X should work. However, I am not familar with headless graphics cards.

A possible technical solution, a fake monitor with a set of resolutions: http://www.ebay.de/itm/Lindy-EDID-DDC-Emulator-Adapter-for-VGA-Displays-EDID-Leser-32101/282698297700?epid=1004725383&hash=item41d221b164:g:tWQAAOSwhvFZJm8z

A workaround:

• Install weston and Xwayland
• Run X :1
• Run

DISPLAY=:1 x11docker --weston-xwayland --size 1280x800 --desktop --showenv x11docker/xfce-wine-playonlinux

[casper]

>  xrandr
Screen 0: minimum 320 x 200, current 320 x 200, maximum 32767 x 32767
VGA1 disconnected (normal left inverted right x axis y axis)
VIRTUAL1 disconnected (normal left inverted right x axis y axis)

I'm running out of steam a bit for today. One question though, what's the difference between --xorg and manually running X on :0 + --hostdisplay?

I think it's better I test the sizes just by manually running X. Don't need the x11docker complexity added for figuring out this resolution problem. I can't imagine it's not possible to get this setup to work without any other workarounds. If it can display 320x200 it must be able to display other resolutions too.

Even xrandr says the maximum is as big as a planet. It's just that we're missing some tweak here that we didn't figure out yet I think.

[casper]

See, I run X :0 and then run xrandr, it already shows it's possible to get a working resolution out of it:

> xrandr -display :0
Screen 0: minimum 320 x 200, current 1024 x 768, maximum 32767 x 32767
VGA1 disconnected (normal left inverted right x axis y axis)
VIRTUAL1 disconnected (normal left inverted right x axis y axis)

[mviereck]

I'm running out of steam a bit for today.

me too :)

One question though, what's the difference between --xorg and manually running X on :0 + --hostdisplay?

It is rather small. x11docker sets some options to have a good match of settings and expectations, but you can set those X options manually. For example, X option +extension GLX allows OpenGL, -extension MIT-SHM avoids RAM access failures without --ipc, +extension SECURITY allows untrusted cookies. As seen with TurboVNC display, a mismatching X setup can lead to failures.

I think it's better I test the sizes just by manually running X.

Yes.

See, I run X :0 and then run xrandr, it already shows it's possible to get a working resolution out of it:

That looks good! xrandr shows VIRTUAL1 disconnected. I suspect this is the one we need to connect somehow. I wonder why x11docker gets 320x200 and X :0 gives 1024x768.

[casper]

I wonder why x11docker gets 320x200 and X :0 gives 1024x768.

Yeah, that is a good question. I even tried it with X :8 like x11docker seems to run it, and it still gave me 1024x768.

But I think what we see here also is one of the reasons you may want to use VirtualGL after all; there's no boxing match with the headless Xorg needed.

I think I may have to edit the dreaded xorg.conf file, and manually add something to it. But still it's a bit of a mystery why x11docker gives different results than manual startup.

[mviereck]

I got this to work with disconnected real VGA port on my Laptop:

Modeline=$(echo $(cvt 870 660 | tail -n1 | cut -d' ' -f2- | tr '"' ' '))
xrandr --newmode $Modeline
xrandr --addmode VGA1 $(echo $Modeline | cut -d' ' -f1)
xrandr --output VGA1 --mode $(echo $Modeline | cut -d' ' -f1)

I can place this virtual display beside my real display and move my mouse in this invisible space, and the resolution appears in xrandr output.

Looks promising. :)

[casper]

Yeah those do not work on my machine. VGA1 and VIRTUAL1 are both disconnected. It refuses to accept any new modelines.

I think this is the place where we hit the wall. It seems for this trick I might need a VGA dongle after all. The GPU is running and accessible through VirtualGL, but I cannot figure out how to set any other resolutions.

[casper]

Ok. More progress. This is how to change the resolution directly:

xrandr --fb 1280x800

That will change it to whatever I want instantly. However I have to restart x0vncserver each time, but I guess it's not such a big deal. More important is I was able to set the resolution to something sensible. This works on x11docker too.

But, TigerVNC is not as fast as TurboVNC. Much more lag. I think it's because it's mirroring the real X display through x0vncserver. I also don't have a mouse cursor nor clipboard integration.

Anyway, now the intel drivers are missing in the docker image. I run glxinfo and it says failed to load driver i965, and failed to open drm device: No such file or directory. I probably need at least --gpu. But this is enough for today. Need some sleep.

[mviereck]

Ok. More progress. This is how to change the resolution directly:
xrandr --fb 1280x800
That will change it to whatever I want instantly.

Good hit!

It refuses to accept any new modelines.

Maybe xrandr will accept modelines within your --fb framebuffer size.

The GPU is running

The GPU of the headless server? Did you make the kernel upgrade?

I probably need at least --gpu

Of course, yes. Without this option the docker container cannot access the GPU hardware.

But, TigerVNC is not as fast as TurboVNC. Much more lag. I think it's because it's mirroring the real X display through x0vncserver. I also don't have a mouse cursor nor clipboard integration.

Maybe TurboVNC will offer an option to access existing displays like TigerVNC does.

[casper]

I was thinking about this project this morning, and maybe the most VNC-friendly solution would be to run a VNC server inside the docker container. Not sure if that's in the spirit of x11docker, but I think that's the most appropriate and self-contained solution.

Then there's no need to mess with xauth cookies or any other security implications. The only thing that is needed is for VirtualGL in the container to have access to the host GPU. So basically this would be your standard VNC + VirtualGL setup, except the server is running in the container. A setup like this was the reason VirtualGL was created in the first place.

Running x11docker on X :0 is much less flexible. Whereas if the container would run its own VNC server, then you could also run multiple x11docker continers on the same machine, and they would all be nicely isolated.

But I think perhaps you didn't build x11docker for this purpose in mind, so don't want to force any new ideas on your project. I might have to figure out how to build my own container from scratch, in order to understand all the moving parts properly and get it set up as I want. I'm putting this project on hold for now. My "quick and easy" idea is turning into a full-time project otherwise..hehe. Just wanted some easy way to get Wine + VirtualGL working in a self-contained package, but turns out the "easy way" is learning three new technologies (Wine + VirtualGL + Docker) at the same time. Need more than a weekend for this.

[mviereck]

I was thinking about this project this morning, and maybe the most VNC-friendly solution would be to run a VNC server inside the docker container. Not sure if that's in the spirit of x11docker, but I think that's the most appropriate and self-contained solution.

That is possible, of course. Example: https://stackoverflow.com/a/16311264
The spirit of x11docker? Use if it is useful for you :-D. If not, just drop it.

The only thing that is needed is for VirtualGL in the container to have access to the host GPU.

With 'host' you mean the VNC server or VNC client?

Running x11docker on X :0 is much less flexible.

It would provide the best GPU performance. To run multiple container on display :0 and keep them isolated, option --weston-xwayland would allow that. With this option, you have no trouble with xrandr, it works immediatly with --size. Though, it does not allow changing screen size in a running container due to an Xwayland restriction.

But I think perhaps you didn't build x11docker for this purpose in mind

Ok, the main purpose was with local access in mind. I am not experienced in network setups.

I'm putting this project on hold for now.

You have everything running now what you need ... if the xrandr part is to annoying, just use --weston-xwayland.

One note, a cool alternative to VNC is xpra. It also allows single applications instead of full desktop transfer and has good netwerk transfer rates.

[casper]

That is possible, of course. Example: https://stackoverflow.com/a/16311264

Thanks. Yeah this looks along the lines of what I was thinking.

The spirit of x11docker? Use if it is useful for you :-D. If not, just drop it.

Hehe..yes it was useful. At least I learned some new tricks :)

The only thing that is needed is for VirtualGL in the container to have access to the host GPU.

With 'host' you mean the VNC server or VNC client?

No I mean the docker container needs some way to access the GPU on the docker host, so that VirtualGL is able to run. At least I think this is the way it's supposed to work.

... --weston-xwayland would allow that.

Seems this has dependencies on the host though(?). Which was something I wanted to minimize. I don't have that stuff on Ubuntu 14.04, as far as I know. Also it means yet another component to research and configure. The problem is the physical display is missing, so it makes setting up stuff like that harder. Most headless solutions all focus on VNC, so there's little support or docs for setting up anything else.

Also the overwhelming performance hit will come from VNC itself and how it operates with the remote display, which I already noticed with x0vncserver vs vncserver, so it's probably better to focus on optimizing that part. VirtualGL vs native GPU through X :0 I think won't make such a big difference compared to the hit taken by VNC in the end. For example, I doubt x0vncserver will ever be able to match a pure vncserver + VirtualGL combo in terms of remotely usable performance.

One note, a cool alternative to VNC is xpra. It also allows single applications instead of full desktop transfer and has good netwerk transfer rates.

Yeah xpra is pretty cool, but it's not so good with 3D stuff I think. TurboVNC on the other hand really seems focused on bringing decent remote 3D performance. I would test all these options out individually though, if I had some time to do it. Would be interesting to see and document the differences of all the solutions.

[mviereck]

No I mean the docker container needs some way to access the GPU on the docker host, so that VirtualGL is able to run. At least I think this is the way it's supposed to work.

if I remember right, VirtualGLis thought to allow GPU acceleration on computers without a GPU, for example using the GPU of client computers while the application is running on a headless server without GPU. Of course, using VirtualGL to forward your server GPU in docker container / VNC display is possible, too.

I've used VirtualGL for docker containers years ago. If I remember right, my setup was:

• share /dev/dri with docker option --device /dev/dri:/dev/dri:rw
• set all environment variables of vglrun inside of container.
• either install VirtualGL in image or just share the VirtualGL libraries (2 lib*xy files, if I remember correctly)

Seems this has dependencies on the host though(?).

Yes, it needs weston and Xwayland. Afaik, Xwayland is part of xorg package in Ubuntu 14.04, weston is available in repository.

Also it means yet another component to research and configure.

x11docker would do the setup, it just needs this as dependencies.

Most headless solutions all focus on VNC, so there's little support or docs for setting up anything else.

Maybe the best current docu is this thread ;-).

Yeah xpra is pretty cool, but it's not so good with 3D stuff I think.

You can run x1docker --weston-xwayland --gpu' and attach xpra to it. Than you have 3D accelerated xpra :-p .

[casper]

if I remember right, VirtualGLis thought to allow GPU acceleration on computers without a GPU, for example using the GPU of client computers while the application is running on a headless server without GPU. Of course, using VirtualGL to forward your server GPU in docker container / VNC display is possible, too.

Yes it can be used in several setups. Both remote GPU and local GPUs are supported. With VNC its primary purpose is to provide GPU acceleration to virtual displays that would otherwise not support it. In most cases you will run vncserver on the same host as the GPU, but it's possible that everything is on separate computers (vncserver, GPU, client).

I've used VirtualGL for docker containers years ago. If I remember right, my setup was:

Ok, this is exactly what I need for running vncserver inside the docker container. I think this would give the simplest setup, only requiring docker on the headless host, and a VNC client on the remote computer. No other components are really required.

Yes, it needs weston and Xwayland. Afaik, Xwayland is part of xorg package in Ubuntu 14.04, weston is available in repository.

Ok can you explain what the advantage of this setup is? So Xwayland is sort of like a new version of Xorg? So this is like running X :0 on the docker host, but instead it runs Wayland, and then the docker container utilizes the Wayland display somehow, and why is Weston needed? So many questions...:-/

[casper]

... then you have 3D accelerated xpra

Except try setting up xpra on Ubuntu 14.04. I tried it once and it was a complete nightmare. So I'm a bit reluctant to burn another 12 hours of my life up in smoke. That's why every time you bring up xpra I try to change the subject. Not a fan of touching that Python Spaghetti bolognese mix on 14.04 again.

[mviereck]

I think this would give the simplest setup I think this would give the simplest setup

Yes, you may be right. x11docker would give more GPU performance; I found that VirtualGL is not as effective as direct GPU access, even on a local setup. But in a VNC setup, the network transfer lag is more important than GPU performance.

Ok can you explain what the advantage of this setup is? So Xwayland is sort of like a new version of Xorg?

Wayland is the successor of Xorg and incompatible to Xorg's X11 protocol. Xwayland is an X server that can run in Wayland to support X11 further on. Weston is a reference implementation of the new Wayland protocol.

x11docker can run the combination weston + Xwayland on Xorg. This causes some overhead (two layers above Xorg), but has some advantages:

• it supports GPU acceleration
• it isolates from X display :0
• it allows arbitrary display sizes regardless of the size of X display :0

Except try setting up xpra on Ubuntu 14.04. I tried it once and it was a complete nightmare.

Oh, ok. I only use xpra for local setups, don't know how hard it is to set up real network connections. But I want to note that xpra in Ubuntu trusty is very outdated and has been developed quite further. www.xpra.org has an Ubuntu repository.

I've used VirtualGL for docker containers years ago. If I remember right, my setup was:
share /dev/dri with docker option --device /dev/dri:/dev/dri:rw
set all environment variables of vglrun inside of container.
either install VirtualGL in image or just share the VirtualGL libraries (2 lib*xy files, if I remember correctly)

I remember furthermore (and breaking isolation):

• share X socket of display :0 -v /tmp/.X11-unix:/tmp/.X11-unix
• share and prepare X cookie of display :0

[casper]

x11docker can run the combination weston + Xwayland on Xorg. This causes some overhead (two layers above Xorg), but has some advantages:

Ok. And can I attach a remote xpra client to the x11docker container somehow, without needing xpra support on the docker host?

Ubuntu trusty is very outdated and has been developed quite further.

Yes exactly. This was the conclusion I came to also, and that's where the nightmare started when I decided to upgrade to the latest version. Trusty is too old for the latest xpra, and you need to update a lot of components while trying to isolate the upgrades from breaking everything else on Trusty (mostly Python and all its dependencies, pip libraries + their dependencies..it all ends up in DLL hell, and you curling up in a corner begging for the torture to end).

Oh, ok. I only use xpra for local setups, don't know how hard it is to set up real network connections.

Not hard at all. Getting an xpra client running on Windows (which happens to be my remote client in this case) is actually quite easy. The problem is I still have my doubts about remote 3D performance, but perhaps it's worth a shot if there are no docker host xpra requirements.

I remember furthermore (and breaking isolation):

Ok thanks.

[mviereck]

And can I attach a remote xpra client to the x11docker container somehow, without needing xpra support on the docker host?

You need xpra either on docker host or in docker image. To avoid DLL hell on Ubuntu trusty, you can install xpra in the docker image.

A Dockerfile could look like:

FROM x11docker/xfce-wine-playonlinux

RUN apt-get update && apt-get upgrade -y && apt-get install -y curl
RUN curl https://winswitch.org/gpg.asc | apt-key add -
RUN echo "deb http://winswitch.org/ stretch main" > /etc/apt/sources.list.d/winswitch.list;
RUN apt-get update && apt-get install -y xpra

# with x11docker providing the display:
CMD xpra start-desktop $DISPLAY --use-display --exit-with-children --start-child startxfce4

# or with xpra creating the display:
#CMD xpra start-desktop :100 --exit-with-children --start-child startxfce4
# or a single application:
#CMD xpra start :100 --exit-with-children --start-child playonlinux

Build with docker build -t wine < Dockerfile
I am not sure how to access the xpra server as the docker container has another ip adress than the docker host. If you add --net to x11docker (or --net=host to docker run), I assume the container has the same IP as the docker host. Maybe it is easy: xpra attach ssh:HOSTNAME:100

To avoid --net=host, it may help to use docker run option --hostname=winecontainer to have a host name for ssh access.

Possible setup:

X :0 & sleep 3
DISPLAY=:0  x11docker --weston-xwayland --desktop --gpu --display 100 --size 1024x768 -- --hostname=winecontainer wine

On client: xpra attach ssh:winecontainer:100

---

Maybe, for first tries, don't use x11docker, instead enable the second or third CMD in image, build again und run docker on server with docker run --hostname=winecontainer wine and xpra on client with xpra attach ssh:winecontainer:100

---

If it does not work with hostname winecontainer, try docker run --net=host wine and xpra on client with xpra attach ssh:YOUR_SERVER_IP:100.

[mviereck]

I had "trusty" instead of "stretch" in Dockerfile sources, it is corrected now (confused host system vs. image system). If you are already trying out, please regard this change.

[mviereck]

Oh, and by the way: you can create a dummy VGA plug yourself:


http://www.geeks3d.com/20091230/vga-hack-how-to-make-a-vga-dummy-plug/

[mviereck]

Not sure if you are still reading here ... just want to tell you that in x11docker version 3.9.1.4 your solution with xrandr --fb XxY is included. If x11docker cannot detect a connected monitor, it sets the framebuffer size instead. Default is 1024x768 and can be changed with --size 1920x1080.

To avoid messing around with DISPLAY and XAUTHORITY, you can disable XAUTHORITY cookies with --no-auth and set desired DISPLAY with --display :1. You should not need a VGA plug. ;-)

[casper]

I'm still here :) Thanks for your help and fixes. I stopped participating and gave up on the whole Wine setup in the end, because I discovered cloud computing has evolved to enable high-FPS remote displays.

It was easier to rent a Windows cloud computer than mess with Wine in the end :-/ Everything works, and the latency is not too bad. My purpose was to do some light gaming, and this setup works pretty well now, And I got an 8 core 16G machine for only a few cents per hour. Not too bad. Beats my dusty Ubuntu server any day.

[mviereck]

And I got an 8 core 16G machine for only a few cents per hour.

ok, that is hard to beat ... :-D

Thanks for your discovery of xrandr --fb, its inclusion in x11docker may be useful for someone in future!