Unable to start sway on Arch Linux running in VMWare Fusion #5834

gpanders · 2020-11-23T23:43:41Z

Please fill out the following:

Sway Version: 1.5.1
Debug Log: https://paste.sr.ht/~gpanders/64de4204fcd197701c1750ffcd02cf2397b37e37
Configuration File: Default sway config file
Description: This is a fresh install of Arch Linux on VMWare Fusion. I'm running sway from the command line and then the screen goes black and unresponsive. The only way for me to kill it is to switch to another tty using Ctrl+Alt+F3 and running killall sway.

These are the errors I see in the log file:

00:00:00.178 [ERROR] [backend/drm/atomic.c:36] Virtual-1: Atomic commit failed (pageflip): Invalid argument
00:00:00.190 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:00.190 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:00.190 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:00.190 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:00.192 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:00.192 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:00.207 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:00.207 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:01.196 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:01.196 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:02.200 [ERROR] [backend/drm/renderer.c:381] Tried to acquire an FB with a NULL BO
00:00:02.200 [ERROR] [backend/drm/atomic.c:164] Failed to set plane 35 properties
00:00:12.225 [ERROR] [backend/drm/atomic.c:36] Virtual-1: Atomic commit failed (modeset): Permission denied

I've tried with and without VMWare's "Accelerate 3D Graphics" option. It doesn't seem to make a difference.

The text was updated successfully, but these errors were encountered:

emersion · 2020-11-24T07:56:14Z

Can you obtain DRM debug logs? See https://github.com/swaywm/wlroots/wiki/DRM-Debugging

gpanders · 2020-11-24T15:31:45Z

dmesg.log

emersion · 2020-11-24T15:36:22Z

[ 2965.440924] [drm:vmw_du_cursor_plane_atomic_check [vmwgfx]] *ERROR* surface not suitable for cursor
[ 2965.440934] [drm:drm_atomic_helper_check_planes [drm_kms_helper]] [PLANE:35:plane-1] atomic driver check failed
[ 2965.440952] [drm:drm_atomic_check_only [drm]] atomic driver check for 000000002c4d38b2 failed: -22

gpanders · 2020-11-24T15:59:31Z

$ lsmod | grep drm_kms_helper
drm_kms_helper        266240  1 vmwgfx
cec                    73728  1 drm_kms_helper
syscopyarea            16384  1 drm_kms_helper
sysfillrect            16384  1 drm_kms_helper
sysimgblt              16384  1 drm_kms_helper
fb_sys_fops            16384  1 drm_kms_helper
drm                   585728  5 vmwgfx,drm_kms_helper,ttm

I've scoured the package manager for every permutation of mesa, drm, dri, and vmware I can think of and it appears I have all appropriate packages installed.

Also, I don't know if this is relevant/helpful, but I can start sway from an already running Gnome session (i.e. running sway from Gnome Terminal). It opens sway in a separate window but everything seems to be working as expected. It's only when started from the tty that it fails to start. That makes me think it's not a driver problem (though I could be wrong).

hedgepigdaniel · 2020-12-01T10:00:59Z

WLR_NO_HARDWARE_CURSORS=1 sway is a workaround for me.

emersion · 2020-12-01T10:06:31Z

It sounds like this check fails in the driver:

	if (surface && !surface->snooper.image) {
		DRM_ERROR("surface not suitable for cursor\n");
		ret = -EINVAL;
	}

The snooper image is created in these conditions:

	if (metadata->scanout &&
	    metadata->num_sizes == 1 &&
	    metadata->sizes[0].width == 64 &&
	    metadata->sizes[0].height == 64 &&
	    metadata->format == SVGA3D_A8R8G8B8) {

		srf->snooper.image = kzalloc(64 * 64 * 4, GFP_KERNEL);
		if (!srf->snooper.image) {
			DRM_ERROR("Failed to allocate cursor_image\n");
			ret = -ENOMEM;
			goto out_no_copy;
		}
	} else {
		srf->snooper.image = NULL;
	}

We need to figure out which one of these conditions fail.

Can you provide a drm_info log?

hedgepigdaniel · 2020-12-01T10:36:03Z

drm_info, in windowed mode: https://gist.github.com/hedgepigdaniel/63c37724779f70a5415efe8237f61bca

hedgepigdaniel · 2020-12-01T12:36:31Z

The "preferred" mode changes in drm_info when resizing the host window, or when switching to and from full screen modes, or enabling/disabling the use of multiple screens in full screen mode.

However, the modes seen by sway according to swaymsg -t get_outputs don't seem to update consistently. When new displays are added or removed they appear (I think usually with the correct resolution), but resizing an existing host window (guest screen) does not seem to cause sway to update the available modes.

Also when guest use of multiple screens is enabled, the cursor is sometimes not able to travel across the entire area of all the screens.

LMK if any of these deserve separate issues - just posting here in case it's related.

hedgepigdaniel · 2020-12-01T13:02:24Z

The symptom is different when VMware is set to full screen mode - sway starts and works, but the cursor is not visible as in #3814

drm_info: https://gist.github.com/hedgepigdaniel/bf4a121cace8d9eb0e0e83393dd71a82

hedgepigdaniel · 2020-12-01T13:38:59Z

I see many messages like this in my kernel log:

[drm:vmw_du_cursor_plane_atomic_check [vmwgfx]] *ERROR* surface not suitable for cursor

gpanders · 2020-12-01T15:39:30Z

Thanks @hedgepigdaniel, setting WLR_NO_HARDWARE_CURSORS=1 works for me as well. I had seen that recommended in other issues (such as #3814) but I didn't think to try it since the problems in those issues were different than mine.

hedgepigdaniel · 2020-12-16T22:18:09Z

The snooper image is created in these conditions:

	if (metadata->scanout &&
	    metadata->num_sizes == 1 &&
	    metadata->sizes[0].width == 64 &&
	    metadata->sizes[0].height == 64 &&
	    metadata->format == SVGA3D_A8R8G8B8) {

		srf->snooper.image = kzalloc(64 * 64 * 4, GFP_KERNEL);
		if (!srf->snooper.image) {
			DRM_ERROR("Failed to allocate cursor_image\n");
			ret = -ENOMEM;
			goto out_no_copy;
		}
	} else {
		srf->snooper.image = NULL;
	}

We need to figure out which one of these conditions fail.

Preliminary findings to be confirmed: None of this code runs at all. I put a DRM_ERROR in the else block and didn't see any log messages.

Next step is to put more of them at all the early returns earlier in that function and see of one of them is the culprit.

hedgepigdaniel · 2020-12-17T07:26:57Z

To save me a bunch of confusion - what code in wlroots would you expect to be calling vmw_surface_define_ioctl?

emersion · 2020-12-17T11:46:41Z

what code in wlroots would you expect to be calling vmw_surface_define_ioctl?

This function is hooked up to the ioctl VMW_CREATE_SURFACE, which I expect would be called when creating a GBM BO (so as a result of a gbm_bo_create call, which should call the driver-specific function to allocate a buffer in Mesa).

emersion · 2020-12-17T11:48:54Z

Mesa call, if that helps: https://gitlab.freedesktop.org/mesa/mesa/-/blob/dc3505f87ed69ac843cd4ce7d269b2ab01a32ff7/src/gallium/winsys/svga/drm/vmw_screen_ioctl.c#L187

hedgepigdaniel · 2020-12-21T09:50:02Z

Well, it turns out that mesa doesn't call vmw_surface_define_ioctl - instead it calls vmw_gb_surface_define_ext_ioctl, and that does not set snooper.image.

The choice between the two ioctls is made here on the basis that have_gb_objects is true.

emersion · 2020-12-21T10:00:54Z

OK. Can you report a Mesa bug about this?

hedgepigdaniel · 2020-12-21T11:00:49Z

Sure, https://gitlab.freedesktop.org/mesa/mesa/-/issues/4003

IvoMajic · 2020-12-24T17:36:26Z

WLR_NO_HARDWARE_CURSORS=1 doesn't help in VMWare Fusion 12.1.0 (17195230) (macOS Big Sur host, fresh Arch Linux guest). I get the following error when starting from tty1:

00:00:00.004 [sway/main.c:349] Unable to retrieve socket path

Same issue when starting from GDM. If started from gnome-terminal sway successfully runs in a separate window.

emersion · 2020-12-24T22:42:24Z

@IvoMajic Please redirect the output of Sway to a file to get full logs: sway -d >sway.log 2>&1. The console is messed up when Sway switches it to graphics mode.

IvoMajic · 2020-12-24T23:03:37Z

@emersion full log:

https://gist.github.com/IvoMajic/e09a49d12510e1efa61d1c67dfa3c90b

The result is just a black screen, which can only be exited by switching to other tty or shutdown.

emersion · 2020-12-24T23:36:15Z

You're also getting this:

00:00:00.221 [ERROR] [backend/drm/atomic.c:36] Virtual-1: Atomic commit failed (pageflip): Invalid argument

lvets · 2021-03-03T01:26:59Z

Is there anything I can do to help? I'm successfully running Sway 1.5.1 on a Macbook Pro 2019 with Big Sur 11.2.2 in VMware Fusion Pro 12.1.0 in a window (not fullscreen). I am setting WLR_NO_HARDWARE_CURSORS=1.

Debug log from my VM: https://gist.github.com/lvets/944d9d351f814567d472000583822b40

Qix- · 2021-05-03T18:09:58Z

Having the same issue. My dmesg log shows no errors, and the redirected output from sway is an empty file.

This is using VMWare Workstation Player 16 and a fresh install of Arch.

Using WLR_NO_HARDWARE_CURSORS=1 changed nothing.

Running sway immediately freezes the VM and no additional input works. A forced reboot is required.

emersion · 2021-05-03T18:11:19Z

the redirected output from sway is an empty file.

You need to redirect stderr.

Qix- · 2021-05-03T18:26:56Z

@emersion I did. I ran sway -d >sway.log 2>&1 exactly as prescribed in #5834 (comment).

peterlewis · 2021-06-24T09:11:06Z

Just popping in with a +1 - I am experiencing the same issue when running sway (albeit in a Void Linux VMware VM).

WLR_NO_HARDWARE_CURSORS=1 did not resolve.

00:00:00.225 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:00.225 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:00.230 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:00.230 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:00.285 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:00.285 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:01.236 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:01.236 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:02.241 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:02.241 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:03.248 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:03.248 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:04.254 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:04.254 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:05.261 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:05.261 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:06.266 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:06.266 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:07.273 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:07.273 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:08.279 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:08.279 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:09.284 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:09.284 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties
00:00:10.291 [ERROR] [wlr] [backend/drm/atomic.c:140] Failed to acquire FB
00:00:10.291 [ERROR] [wlr] [backend/drm/atomic.c:162] Failed to set plane 35 properties

asininemonkey · 2021-09-13T13:02:56Z

Sway works perfectly for me when setting WLR_NO_HARDWARE_CURSORS=1 and ensuring Accelerate 3D Graphics in the guest VM's display preferences is enabled.

If the former is not set you will get a black screen. If the latter is not set you will get a Could not connect to socket error.

galaxy4public · 2022-09-02T00:52:32Z

Something fishy is definitely going on with wlroots and virtualised drm drivers (such as virtio-gpu, vmwgfx, etc.). On all virtual machines (tried Parallels Desktop, VMware Fusion, but above there are reports re: Virtualbox too) any wlroots-based compositor can't produce a hardware cursor (either it is not visible or results in a console lockup). At the same time, Weston happily works with all these drm drivers and enables hardware cursor. JFYI.

ClaCodes · 2022-11-14T17:57:06Z

I am having the same issue on a fresh install of ubuntu in vmware. Same symptoms (black screen, starting from already running gnome session opens sway correctly in window).

WLR_NO_HARDWARE_CURSORS=1 sway is a workaround for me.

Works for me too. I have to disable the graphical login manager with sudo systemctl set-default multi-user and start it with the workaround from the terminal.

fislysandi · 2022-12-01T20:54:56Z

hey bros doesnt anybody know how to make WLR_NO_HARDWARE_CURSORS=1 sway this work with sddm? because if i try to input it into the .desktop file it doenst work

pls halp.

blackmirroxx · 2023-05-13T02:41:15Z

I am having the same issue with arch as a guest in qemu, arch kernel vesions:
--6.3.1

6.1-19
6.3.2
no luck on either kernel version
but it does work on version 6.1.24, if i build the kernel myself with {WLR_NO_HARDWARE_CURSORS=1}

JanJenicek · 2023-10-25T14:24:21Z

+1
not much to add
host: Win10 Ent; VMware 16 PRO
guest: Debian12 headless; sway 1.7 from scratch + swaybg and all recommended
WLR_NO_HARDWARE_CURSORS=1

black screen after sway launched from terminal

after switching between TTYs, graphical output is updated, but again hangs until another TTY switch back and forward.

all env.var. SWAYSOCK etc. appear to be empty by default. Manually setting SWAYSOCK has no effect.

Anything to provide or help?

gpanders added the bug Not working as intended label Nov 23, 2020

hedgepigdaniel mentioned this issue Dec 21, 2020

"VGA-1: Atomic commit failed (pageflip): Cannot allocate memory" with vboxvideo swaywm/wlroots#2439

Open

boredland mentioned this issue Aug 21, 2021

Doesn't start in VMware - Asked to enter username on Live CD manjaro-sway/manjaro-sway#139

Closed

SuperSamus mentioned this issue Nov 26, 2021

Can't launch sway in a vmware virtual machine. NixOS/nixpkgs#147392

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to start sway on Arch Linux running in VMWare Fusion #5834

Unable to start sway on Arch Linux running in VMWare Fusion #5834

gpanders commented Nov 23, 2020

emersion commented Nov 24, 2020

gpanders commented Nov 24, 2020 •

edited

Loading

emersion commented Nov 24, 2020

gpanders commented Nov 24, 2020

hedgepigdaniel commented Dec 1, 2020

emersion commented Dec 1, 2020

hedgepigdaniel commented Dec 1, 2020 •

edited

Loading

hedgepigdaniel commented Dec 1, 2020

hedgepigdaniel commented Dec 1, 2020 •

edited

Loading

hedgepigdaniel commented Dec 1, 2020

gpanders commented Dec 1, 2020 •

edited

Loading

hedgepigdaniel commented Dec 16, 2020

hedgepigdaniel commented Dec 17, 2020

emersion commented Dec 17, 2020

emersion commented Dec 17, 2020

hedgepigdaniel commented Dec 21, 2020

emersion commented Dec 21, 2020

hedgepigdaniel commented Dec 21, 2020

IvoMajic commented Dec 24, 2020 •

edited

Loading

emersion commented Dec 24, 2020

IvoMajic commented Dec 24, 2020 •

edited

Loading

emersion commented Dec 24, 2020

lvets commented Mar 3, 2021 •

edited

Loading

Qix- commented May 3, 2021

emersion commented May 3, 2021

Qix- commented May 3, 2021 •

edited

Loading

peterlewis commented Jun 24, 2021 •

edited

Loading

asininemonkey commented Sep 13, 2021 •

edited

Loading

galaxy4public commented Sep 2, 2022 •

edited

Loading

ClaCodes commented Nov 14, 2022 •

edited

Loading

fislysandi commented Dec 1, 2022

blackmirroxx commented May 13, 2023 •

edited

Loading

JanJenicek commented Oct 25, 2023

Unable to start sway on Arch Linux running in VMWare Fusion #5834

Unable to start sway on Arch Linux running in VMWare Fusion #5834

Comments

gpanders commented Nov 23, 2020

Please fill out the following:

emersion commented Nov 24, 2020

gpanders commented Nov 24, 2020 • edited Loading

emersion commented Nov 24, 2020

gpanders commented Nov 24, 2020

hedgepigdaniel commented Dec 1, 2020

emersion commented Dec 1, 2020

hedgepigdaniel commented Dec 1, 2020 • edited Loading

hedgepigdaniel commented Dec 1, 2020

hedgepigdaniel commented Dec 1, 2020 • edited Loading

hedgepigdaniel commented Dec 1, 2020

gpanders commented Dec 1, 2020 • edited Loading

hedgepigdaniel commented Dec 16, 2020

hedgepigdaniel commented Dec 17, 2020

emersion commented Dec 17, 2020

emersion commented Dec 17, 2020

hedgepigdaniel commented Dec 21, 2020

emersion commented Dec 21, 2020

hedgepigdaniel commented Dec 21, 2020

IvoMajic commented Dec 24, 2020 • edited Loading

emersion commented Dec 24, 2020

IvoMajic commented Dec 24, 2020 • edited Loading

emersion commented Dec 24, 2020

lvets commented Mar 3, 2021 • edited Loading

Qix- commented May 3, 2021

emersion commented May 3, 2021

Qix- commented May 3, 2021 • edited Loading

peterlewis commented Jun 24, 2021 • edited Loading

asininemonkey commented Sep 13, 2021 • edited Loading

galaxy4public commented Sep 2, 2022 • edited Loading

ClaCodes commented Nov 14, 2022 • edited Loading

fislysandi commented Dec 1, 2022

blackmirroxx commented May 13, 2023 • edited Loading

JanJenicek commented Oct 25, 2023

gpanders commented Nov 24, 2020 •

edited

Loading

hedgepigdaniel commented Dec 1, 2020 •

edited

Loading

hedgepigdaniel commented Dec 1, 2020 •

edited

Loading

gpanders commented Dec 1, 2020 •

edited

Loading

IvoMajic commented Dec 24, 2020 •

edited

Loading

IvoMajic commented Dec 24, 2020 •

edited

Loading

lvets commented Mar 3, 2021 •

edited

Loading

Qix- commented May 3, 2021 •

edited

Loading

peterlewis commented Jun 24, 2021 •

edited

Loading

asininemonkey commented Sep 13, 2021 •

edited

Loading

galaxy4public commented Sep 2, 2022 •

edited

Loading

ClaCodes commented Nov 14, 2022 •

edited

Loading

blackmirroxx commented May 13, 2023 •

edited

Loading