VM root window resize failure

[rbdavis]

I am unable to satisfactorily resize the root window of a newly-installed Fedora Workstation 33 VM running on my macOS Catalina 10.15.6 host under VMWare Fusion 11.5.5. Running xdpyinfo inside the VM immediately after bootup always shows 800x600 resolution. When I manually enlarge the VM window from macOS by dragging on a window corner, the window borders stretch as expected but the VM's internal pixel resolution remains 800x600 and the visual effect is merely to proportionally enlarge the existing window contents rather than to expose new usable VM pixels. The Fusion menubar visible at the top of my macOS display shows the 'View -> Resize Virtual Machine to Fit' and 'View -> Resize Virtual Machine' options as grayed out, i.e., they are inoperable.

I'm not sure if this is the appropriate forum to seek advice, apologies if not and many thanks if anyone can help! This VM is absolutely unusable to me if constrained to 800x600 resolution.

The Fedora default install procedure loaded the following RPMs of interest:

open-vm-tools-11.2.0-1.fc33.x86_64
open-vm-tools-desktop-11.2.0-1.fc33.x86_64
xorg-x11-drv-vmware-13.2.1-13.fc33.x86_64
xorg-x11-server-Xorg-1.20.10-1.fc33.x86_64
xorg-x11-server-Xwayland-1.20.10-1.fc33.x86_64
gnome-session-wayland-session-3.38.0-1.fc33.x86_64

The VM's process table includes the following processes of interest:

[irq/16-vmwgfx]
fusermount -o rw,allow_other,auto_unmount,subtype=vmhgfs-fuse -- /mnt/hgfs
vmhgfs-fuse .host:/Users /mnt/hgfs -o rw,auto_unmount,allow_other,dev,suid
/usr/bin/vmtoolsd
/usr/bin/vmtoolsd -n vmusr --uinputFd 3
/usr/bin/Xwayland :0 -rootless -noreset -accessx -core -auth /run/user/1330/.mutter-Xwaylandauth.TEBZW0 -listen 4 -listen 5 -displayfd 6 -listen 7
/usr/libexec/gdm-wayland-session /usr/bin/gnome-session

I have done nothing inside the Fedora VM to customize anything related to the display config (e.g., X11, Wayland, etc.), nor have I made any changes to the VM's internal open-vm-tools configuration as installed by Fedora other than to add an fstab entry to mount one of the macOS hosts's HGFS filesystems. (Thanks to others on this forum and elsewhere for providing tips on how to do that!)

I found one reference to this problem hinting that it's triggered by Wayland use, which seems to be in effect here (see above process list). The suggested fix requires installation of an xf86-video-vmware package which does not seem to be available for Fedora. (I located a SUSE version of the package which will not install due to dependencies which seem incompatible with Fedora.) Also, I wonder if the Fedora xorg-x11-drv-vmware-13.2.1-13 package (see above package list) is functionally equivalent?

I am considering two workarounds for this issue but would appreciate advice before proceeding with either:

(1) The Fusion menubar's 'Virtual Machine -> Reinstall VMWare Tools' option is grayed out and inoperable. Presumably this is because the VM is running open-vm-tools rather than the older VMWareTools, but I am wondering if I can fix things by uninstalling open-vm-tools and then installing VMWareTools? I have a CentOS 7 VM also installed on this host which is using VMWareTools-10.2.5-8068393 and working perfectly in this regard. Can I just grab the VMWareTools tar.gz file from that VM and install it to control my Fedora 33 VM, or is it incompatible with the latter bleeding-edgier distro?

(2) I can live with a hack that permanently sets the display size of my VM to something usable, i.e., anything 1920x1200 or larger. Apologies for being Fedora-/xorg-stupid, but is there an easy way to do this inside the Fedora VM's X11 (or Wayland) config?

Thanks!

[rbdavis]

Problem found and fixed thanks to a tip in the Reddit Fedora forum.

/etc/vmware-tools/tools.conf does not exist after installation. Create one by copying tools.conf.example, then go to that file's resolutionKMS section and uncomment the 'enable' line, e.g.:

[resolutionKMS]

#Default is true if tools finds an xf86-video-vmware driver with
#version >= 13.2.0. If you don't have X installed, set this to true manually.
#This only affects tools for Linux.
enable=true

According to the file's comments 'true' should be the default if a 13.2 driver is found. Either this is not true or the system could not locate my driver even though xorg-x11-drv-vmware-13.2.1-13.fc33.x86_64 is clearly installed.

Guess I'll have to look this file over and see what else the Fedora install left undone.

[lousybrit]

Hi there,
I got some information from our graphics guys on this.

=====================================
There’s a few things that could go wrong:

1. resolutionKMS requires the following libraries to be available at runtime:
   libudev.so.1 or libudev.so.0
   libdrm.so.2

2. It requires vmwgfx kernel module to be loaded.

3. The xorg driver is searched for only under:
   /usr/lib64/xorg/modules/drivers/vmware_drv.so
   /usr/lib/xorg/modules/drivers/vmware_drv.so

If any of the above doesn’t exist (and in case of the vmwgfx kernel driver it "isn’t loaded”) then resolutionKMS won’t work.
It’d be great to see some logs from the tools.

Please ask the user for /var/log/vmware-vmsvc-root.log which should shed some light on what’s happening (and ideally a version of that log file from a system without the "enable=true” line under [resolutionKMS] to make the autodetection code run)

=====================================
So providing the log file mentioned above, if you can reproduce the issue, would be great.
Thanks

[rbdavis]

Thanks for replying, lousy(buthelpful?)brit!

The udev and drm libraries are present as you specified, as well as /usr/lib64/xorg/.../vmware_drv.so, and lsmod says that vmwgfx is loaded.

I have attached two versions of /var/log/vmware-vmsvc-root.log, each containing the results of a single reboot, attempted VM root window resize and login. (I deleted the old log file just prior to reboot in each case.) The *log-notoolsconf.txt file is the log created by this test when /etc/vmware/tools.conf does not exist at boot time. The *log-toolsconf.txt file is the log created when that tools.conf file DOES exist and the resolutionKMS 'enable=true' line has been uncommented. For each test, I (i) let the Fedora 33 VM fully reboot, (ii) manually resized the VM root window via mouse grab-corner-and-stretch in the macOS GUI, and then (iii) logged into the VM. As noted in my original post, the VM root window does not properly resize to new X11/Wayland server display dimensions when tools.conf does not exist and vmtoolsd is presumably using whatever autodetect defaults it comes up with on its own.

Obviously the workaround here is trivial (i.e., make a tools.conf file with resKM enabled). It did take me some effort to research this solution, however, and it would undoubtedly be much appreciated by others to not have to repeat the effort.

Log files follow, I hope, please let me know if you need additional info on my system and thanks again for your help!!

vmsvc-root-log-notoolsconf.txt
vmsvc-root-log-toolsconf.txt

[lousybrit]

Hello there,

From the graphics developer:
Got it. OK, this is our bug. It’s affecting systems which are running wayland (which is the default on e.g. current Fedora). The check for Xorg driver version fails when running on top of Wayland.

As the user mentioned, the workaround is to "cp /etc/vmware-tools/tools.conf.example /etc/vmware-tools/tools.conf” and uncomment the “#enable=true” line inside the “[resolutionKMS]” section.

I’m going to fix the autodetection so it properly works by default.

Thank you for reporting this.

[rbdavis]

Thanks a bunch, much appreciated!

If nothing else I've learned many useful things about open-vm-tools and was finally able to fix a long-broken VoidLinux VM where VMWareTools failed to install when I created the VM and installed Void. In that case I was able to work around the resize problem by using xrandr to hand-config Xorg to a useful resolution. I was not able to get that workaround to fly with this Fedora VM, possibly again because of Wayland interlopery.

[uber6]

Hello. I think this is still an issue. I recently installed Fedora 33, and there was no /etc/vmware-tools/tools.conf

I copied the example file and uncommented #enable=true under “[resolutionKMS]”, and now it's working.

I am using Workstation 16 Pro

[ItsJamie9494]

This is still an issue, even in Fedora 34

[lousybrit]

This is still an issue, even in Fedora 34

From our graphics engineer:
"The change is in the devel branch in the github repo of open-vm-tools. I doubt any distro builds their open-vm-tools package out of the devel branch. Unfortunately, I’m not sure when the open-vm-tools team will merge the devel branch into stable."

So the fix is in progress.
Thanks.

[johnwvmw]

The fix for the KMS autodetection is in the "devel" branch: ff5eb5f

[scaronni]

This is still an issue, even in Fedora 34

Can someone with the issue try this build with no workaround applied?

https://koji.fedoraproject.org/koji/taskinfo?taskID=69080524

If it works fine, I can build an updated Fedora package with the upstream patch for Fedora 33+.

[alexhaydock]

This is still an issue, even in Fedora 34

Can someone with the issue try this build with no workaround applied?

https://koji.fedoraproject.org/koji/taskinfo?taskID=69080524

If it works fine, I can build an updated Fedora package with the upstream patch for Fedora 33+.

I can confirm that installing the open-vm-tools and open-vm-tools-desktop RPMs from this Koji build does fix the issue on a fresh Fedora 34 install inside VMware Workstation 16 on a RHEL 8 host which hadn't had any workaround applied and which did show the issue before installing the packages. Screen resize is now working fine in the default Wayland session, including at the GDM login screen.

Thanks! 👍

Edit: I also notice you're building for aarch64 here too and you've addressed the other issue I filed in anticipation of ARM-native Fusion for M1, so thanks for that too! 😄

[tiadobatima]

Can someone with the issue try this build with no workaround applied?

https://koji.fedoraproject.org/koji/taskinfo?taskID=69080524

I tried this fix on fedora34 and it fixed the window resizing problem, But the copy/paste problem referenced by #510 is still borked when running under both i3 (X11) and sway (Wayland), but in my short tests copy/paste did work under Gnome (both X11 and Wayland) though.

[hermidalc]

There's multiple issues related to this so not sure where to post to help close out fixed issues, but I just installed a fresh Fedora 36 guest and resize is working without needing to enable resolutionKMS or do anything else.

[johnwvmw]

@hermidalc Thank you for the confirmation.

The change for "resolution: Fix kms autodetection" was available in open-vm-tools 11.3.0 and later releases.

Closing this issue as fixed. If additional auto-resolution problems are suspected, please open an new issue, indicating the open-vm-tools version and the Linux release involved.

[johnwvmw]

Fixed