Xwayland runs but apps can't connect to it

[lukefromdc]

Describe the bug
A clear and concise description of what the bug is.
When wayfire built with xayland support (system OR local wlroots) is started, xwayland starts and shows up in mate-system-monitor(which works fine on wayland). Trying to run an app that does not support wayland such as GIMP (a gtk2 app) or any QT5 app on my setup (where for some reason QT5 on wayland fails) gets the following errors:

With xterm:

luke@ubuntu:~$ xterm
xterm: Xt error: Can't open display: wayland-1
luke@ubuntu:~$ 

with Kdenlive (QT5):
(kdenlive:35812): Gtk-WARNING **: 01:54:37.385: cannot open display: wayland-1

With GIMP (GTK-2)
Cannot open display:

Running any of these apps functions as expected in weston, showing that my Xwayland install is not the
problem. Even kdenlive (a complex QT5 video editor) works the same as it does in x11 in Weston. Therefore
something in wayfire isn't letting apps connect to Xwayland, since we can see it running in Top or mate-system-monitor

To Reproduce
Steps to reproduce the behavior:
Install any recent version of wayfire build with Xwayland support
try to run any app that does not support wayland

Expected behavior
Non-wayland supporting apps should open in Xwayland in wayfire the same as they do in Weston, which for
all it's limitations does have a working Xwayland system

Screenshots or stacktrace
None, the text from terminal tells the story, and since Xwayland keeps running it doesn't appear to be crashing though a silent crash and restart with no text in terminal cannot be ruled out.

Wayfire version
3 versions tested:
0.7.4 from Debian
A local build of 0.7.4 with builtin wlroots
A local build of 0.8.0 from git master as of 7-30-2022 buildable against Debian's builtin wlroots (this to avoid a build failure on Debian with current git master and system wlroots)

The latter two are confirmed to behave identically. I didn't check Debian's build for running wayfire before replacing it

[lukefromdc]

Retested with wayfire from current git master, got a successful build with system wlroots libwlroots11_0.16.2-1 from Debian experimental. Same behavior as before.

[lukefromdc]

Also same exact problem in Sway with same error messages, suggesting wlroots is the issue. Building wayfire with internal wlroots from the last version before the soname bump to 0.12 (last usable version right now) didn't help. Note that there have been some xwayland changes to wlroots since then, but wayfire needs some porting to build against them due to some removed and replaced functions

[Sunderland93]

I have the exact same error when running xterm in a terminal in a LabWC built with wlroots 0.16.2. However, it starts up fine and works if you run it through the application menu (mate-menu in my case). I don't think that is Wlroots issue but rather some environment variable is missing.

[lukefromdc]

I just confirmed that opening GTK2 and QT apps from the panel or by clicking on associated files in Caja works just fine, and that there is a fix for the problem in mate-terminal. No bug here so closing

[marcusbritanicus]

Most likely, your apps in the terminal are not able to read the requisite env vars. You should add something like

0_environment = dbus-update-activation-environment --systemd WAYLAND_DISPLAY DISPLAY XAUTHOR

at the top of the [autostart] section.

You'll find a few tips and tricks here: https://github.com/WayfireWM/wayfire/wiki/Tips-&-Tricks