Spine PlayStation 4 emulator does not work in WSLg with GPU enabled

[sjkorvin]

Environment

Windows build number: 22000.184
Your Distribution version: Ubuntu 20.04
Your WSLg version: 1.0.26
Nvidia GeForce Drivers: 510.06
SR-IOV: Enabled in motherboard BIOS

Steps to reproduce

I got Spine working in my Ubuntu 21.04 virtual machine in VMWare Workstation 16. Unfortunately, I cannot get it to work in WSLg. Spine requires PlayStation 4 firmware files that I have placed in the emulator directory's firmware directory. Please note that in Ubuntu, Spine requires libsdl2-dev to be installed. I have installed libsdl2-dev in Ubuntu in WSL through sudo apt install libsdl2-dev.

1.) Copy Spine emulator directory (spine-20210901) to /home/steven/
2.) Run chmod a+x on the spine binary file and on the contents of the lib directory in the emulator directory
3.) Run cd /home/steven/spine-20210901 to cd to the emulator directory
4.) Run ./spine /home/steven/Sonic_Mania/eboot.bin to launch Sonic Mania.
5.) I am using a 144 Hz display. I have added the following lines to c:\ProgramData\Microsoft\WSL\.wslgconfig; however, this issue has occurred with and without these lines added to my .wslgconfig file:

[system-distro-env]
WESTON_RDP_MONITOR_REFRESH_RATE=144

Below are my file permissions to verify that they are correctly set:

steven@My-Desktop:~/spine-20210901$ ls -l spine
-rwxr-xr-x 1 root root 3221616 Sep  1 05:58 spine
steven@My-Desktop:~/spine-20210901$ ls -l ./lib
total 6512
-rwxr-xr-x 1 root root 1983096 Jun 29 05:36 libavcodec.so.58
-rwxr-xr-x 1 root root 1983096 Jun 29 05:36 libavcodec.so.58.134.100
-rwxr-xr-x 1 root root  663112 Jun 29 05:36 libavutil.so.56
-rwxr-xr-x 1 root root  663112 Jun 29 05:36 libavutil.so.56.70.100
-rwxr-xr-x 1 root root  117056 Jun 29 05:36 libswresample.so.3
-rwxr-xr-x 1 root root  117056 Jun 29 05:36 libswresample.so.3.9.100
-rwxr-xr-x 1 root root  563512 Jun 29 05:36 libswscale.so.5
-rwxr-xr-x 1 root root  563512 Jun 29 05:36 libswscale.so.5.9.100

Below is my dmesg output:

[   24.215967] process 'home/steven/spine-20210901/spine' started with executable stack
[   24.775775] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   24.775776] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   24.775777] dxgk:err: adapter_by_handle failed 40000080
[   24.776619] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   24.776619] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   24.776620] dxgk:err: adapter_by_handle failed 40000080
[   24.778768] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   24.778768] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   24.778769] dxgk:err: adapter_by_handle failed 40000080
[   24.779602] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   24.779603] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   24.779603] dxgk:err: adapter_by_handle failed 40000080
[   25.683854] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   25.683855] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   25.683856] dxgk:err: adapter_by_handle failed 40000080
[   25.684766] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   25.684767] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   25.684768] dxgk:err: adapter_by_handle failed 40000080
[   25.687062] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   25.687063] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   25.687063] dxgk:err: adapter_by_handle failed 40000080
[   25.687949] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   25.687950] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   25.687950] dxgk:err: adapter_by_handle failed 40000080

Below is my glxinfo output:

steven@My-Desktop:~/spine-20210901$ glxinfo -B
name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Microsoft Corporation (0xffffffff)
    Device: D3D12 (NVIDIA GeForce RTX 2080 SUPER) (0xffffffff)
    Version: 21.0.3
    Accelerated: yes
    Video memory: 40735MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 3.3
    Max compat profile version: 3.1
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.0
OpenGL vendor string: Microsoft Corporation
OpenGL renderer string: D3D12 (NVIDIA GeForce RTX 2080 SUPER)
OpenGL core profile version string: 3.3 (Core Profile) Mesa 21.0.3
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 3.1 Mesa 21.0.3
OpenGL shading language version string: 1.40
OpenGL context flags: (none)

OpenGL ES profile version string: OpenGL ES 3.0 Mesa 21.0.3
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.00

WSL logs:

Unfortunately, running wpr -start wsl.wprp -filemode from an administrative command prompt gave me the output:

        Error code: 0xc00ce584
        DTD is prohibited.

        Line Number: 7
        Column Number: 11

Please find attached directly below my WSLg logs.
pulseaudio.log
weston.log

Attempting to open versions.txt in Notepad++ had given me a message that versions.txt did not exist. I could not copy versions.txt elsewhere.

Edit: Running cat on versions.txt from within WSL gave me the following output:

WSLg ( x86_64 ): 1.0.26+Branch.main.Sha.26ce2c09b86442f3c7f4f6462f770ed2afa76a25
Mariner: VERSION="1.0.20210224"
FreeRDP: b05321cd4e6a862aef76163a69db4e1910245736
weston: 46756d0e77e5c01b5995fbbee6f3ab0db9b30612
pulseaudio: 2f0f0b8c3872780f15e275fc12899f4564f01bd5
mesa:

Expected behavior

I expected Sonic Mania to start with the Spine emulator through WSLg, just as it did for me in my Ubuntu 21.04 virtual machine.

Please find below a screenshot of Sonic Mania running in Spine in my Ubuntu 21.04 virtual machine.

Actual behavior

Sonic Mania is not running in Spine in WSLg.

[hideyukn88]

@sjkorvin, thanks for reporting the issue, have you tried this with GPU acceleration disabled? While I can't be sure if GPU is the sole root cause, but the "dxgk:err:" in dmesg indicates WSLg's GL stack (NVIDIA driver or D3D12 mapping layer) is calling dxgk module in Linux kernel with invalid handle, thus all GL rendering is most likely failing. If this works or does not work using software rendering, it could give us some clue on what we should look for next. You can force software rendering by adding below in .wslgconfig, and also "export LIBGL_ALWAYS_SOFTWARE=1" prior to run emulator in your user distro, GL has to be disabled on both system and user distro. Thanks!

[system-distro-env]
LIBGL_ALWAYS_SOFTWARE=1

[sjkorvin]

@hideyukn88 Thank you very much for your response. I have added that to .wslgconfig. The contents of my .wslgconfig file are currently:

[system-distro-env]
WESTON_RDP_MONITOR_REFRESH_RATE=144

[system-distro-env]
LIBGL_ALWAYS_SOFTWARE=1

After shutting down WSL with wsl --shutdown in PowerShell, I opened Ubuntu in WSL again.

My steps taken are as follow:

1.) Run export LIBGL_ALWAYS_SOFTWARE=1
2.) Run cd /home/steven/spine-20210901
3.) Run ./spine /home/steven/Sonic_Mania/eboot.bin

The emulator still does not open.

My dmesg output appears to be the same as before GPU acceleration was disabled:

[   26.206146] process 'home/steven/spine-20210901/spine' started with executable stack
[   26.801874] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   26.801877] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   26.801877] dxgk:err: adapter_by_handle failed 40000080
[   26.802933] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   26.802934] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   26.802935] dxgk:err: adapter_by_handle failed 40000080
[   26.805320] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   26.805321] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   26.805322] dxgk:err: adapter_by_handle failed 40000080
[   26.806256] dxgk:err: is_handle_valid Entry is freed 40000080 2
[   26.806256] dxgk:err: hmgrtable_get_object_by_type invalid handle 40000080
[   26.806257] dxgk:err: adapter_by_handle failed 40000080
[   48.977442] hv_balloon: Max. dynamic memory size: 32724 MB

My glxinfo -b output is as follows:

41

Thank you again for your assistance in this matter.

[sjkorvin]

Firstly, I now realize that the proper command was glxinfo -B.

Secondly, I thought that I would try running this emulator in an Ubuntu 20.04 virtual machine as opposed to an Ubuntu 21.04 virtual machine. My WSL distribution was specifically Ubuntu 20.04.3. The Spine emulator did not work in my Ubuntu 20.04.3 virtual machine in VMWare Workstation.

Upon experiencing this, I had installed the Ubuntu on Windows Community Preview WSL distribution. It's currently based on Ubuntu 21.04.

Without disabling GPU acceleration, I had received the same dmesg output as when attempting to run the emulator in my Ubuntu 20.04 WSL distribution.

When attempting to run the emulator with GPU acceleration disabled on both system and user distro, it ran!

It takes up a fourth of the screen on my 4K display and performance isn't close to where it was in my Ubuntu 21.04 virtual machine. Nonetheless, this is amazing progress!

While some things need work, would you wish to close this issue? I have no further inquiries because the issue that I had raised has been resolved. Thank you so much for your assistance in solving this.

[sjkorvin]

Before this issue is closed, would you have any idea what could be causing this emulator to not run with hardware acceleration enabled? Performance is not at all where it should be because the emulator is running with software acceleration. Thank you once again.

[Masamune3210]

Not sure if default installed kernel is built with nested virt enabled, afaik support is still pretty new unless I'm mistaken

[hideyukn88]

@sjkorvin, no need to close the issue, we will use this to track the GPU acceleration issue. Unfortunately, I don't have access to PlayStation firmware thus not able to repro, but we will see what we can do to investigate this, thanks!

[sjkorvin]

@hideyukn88 Thank you so much! Someone on the emulation subreddit told me that I need two GPUs to get GPU acceleration working in WSL. Is that correct? I only have my 2080 Super. I'm planning on purchasing a 1050 Ti to add to my PC for another purpose than this issue.

[sundhaug92]

@hideyukn88 Thank you so much! Someone on the emulation subreddit told me that I need two GPUs to get GPU acceleration working in WSL. Is that correct? I only have my 2080 Super. I'm planning on purchasing a 1050 Ti to add to my PC for another purpose than this issue.

WSLg shares GPU resources with Windows, so it doesn't need you to have multiple GPUs

[Masamune3210]

They probably thought you were talking about traditional emulation which does require separate gpu's. Thankfully wslg doesn't since it's integrated closely with Windows

[dnullptr]

Can confirm the same with wsl2 running under Win 11 and WSL graphics driver 510.06
GPU acceleration off and based on 21.04 - works.
(Bad performance ofc)
GPU acceleration off and based on 20.04 - not working, but won't work under a VM too.
GPU acceleration ON on either distro - not working