Cannot run Godot on virtual X desktop with xvfb-run and lavapipe

[qarmin]

Godot version:
Godot 4.0(Vulkan) - 4.0.dev.custom_build. f5cd33f
it works fine with Godot 3.2 GLES 2, 3

OS/device including version:
Ubuntu 20.04

Issue description:

When I try to open master Godot 4.0 branch in virtual X desktop, then I got a crash:

WARNING: XOpenIM failed
     at: DisplayServerX11 (platform/linuxbsd/display_server_x11.cpp:3475)
ERROR: Cant find layer: VK_LAYER_KHRONOS_validation
   at: _check_layers (drivers/vulkan/vulkan_context.cpp:158)
ERROR: Cant find layer: VK_LAYER_LUNARG_standard_validation
   at: _check_layers (drivers/vulkan/vulkan_context.cpp:158)
ERROR: Cant find layer: VK_LAYER_GOOGLE_threading
   at: _check_layers (drivers/vulkan/vulkan_context.cpp:158)
ERROR: Condition "err" is true. returned: ERR_CANT_CREATE
   at: _create_physical_device (drivers/vulkan/vulkan_context.cpp:353)
thirdparty/vulkan/loader/loader.h:396:83: runtime error: load of misaligned address 0xbebebebebebebebe for type 'struct VkLayerDispatchTable *', which requires 8 byte alignment
0xbebebebebebebebe: note: pointer points here
<memory cannot be printed>
handle_crash: Program crashed with signal 11
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[1] godots() [0x17c49da] (/mnt/Miecz/godot/platform/linuxbsd/crash_handler_linuxbsd.cpp:54)
[2] /lib/x86_64-linux-gnu/libc.so.6(+0x46210) [0x7f7a9de14210] (??:0)
[3] godots() [0x64bccf2] (/mnt/Miecz/godot/thirdparty/vulkan/loader/loader.h:396)
[4] godots(vkDestroyFence+0x24) [0x64c91ed] (/mnt/Miecz/godot/thirdparty/vulkan/loader/trampoline.c:1030)
[5] VulkanContext::~VulkanContext() (/mnt/Miecz/godot/drivers/vulkan/vulkan_context.cpp:1596)
[6] VulkanContextX11::~VulkanContextX11() (/mnt/Miecz/godot/platform/linuxbsd/vulkan_context_x11.cpp:56)
[7] void memdelete<VulkanContextX11>(VulkanContextX11*) (/mnt/Miecz/godot/./core/os/memory.h:119)
[8] DisplayServerX11::DisplayServerX11(String const&, DisplayServer::WindowMode, unsigned int, Vector2i const&, Error&) (/mnt/Miecz/godot/platform/linuxbsd/display_server_x11.cpp:3538)
[9] DisplayServerX11::create_func(String const&, DisplayServer::WindowMode, unsigned int, Vector2i const&, Error&) (/mnt/Miecz/godot/platform/linuxbsd/display_server_x11.cpp:3141)
[10] DisplayServer::create(int, String const&, DisplayServer::WindowMode, unsigned int, Vector2i const&, Error&) (/mnt/Miecz/godot/servers/display_server.cpp:563)
[11] Main::setup2(unsigned long) (/mnt/Miecz/godot/main/main.cpp:1261)
[12] Main::setup(char const*, int, char**, bool) (/mnt/Miecz/godot/main/main.cpp:1188)
[13] godots(main+0x22b) [0x17c37a1] (/mnt/Miecz/godot/platform/linuxbsd/godot_linuxbsd.cpp:49)
[14] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f7a9ddf50b3] (??:0)
[15] godots(_start+0x2e) [0x17c34be] (??:?)

Fixing this bug will help a lot with testing Godot with CI

Steps to reproduce:

apt install -y xvfb
xvfb-run godot

[Calinou]

Unfortunately, I don't think this is feasible yet. There is no stable software implementation of Vulkan yet, which is required to render Godot without having a GPU available. There are some efforts ongoing like Kazan, SwiftShader and Mesa's Vallium renderer, but it seems none of them have reached a state where they can be used as a daily driver yet.

Here's hoping we'll be able to do this before Godot 4.0 is released 🙂

[qarmin]

Closing, since #43444 is more descriptive.

[akien-mga]

For the reference, #43444 is now fixed, but it's still true that Godot can't be run with xvfb-run without using a software renderer. When using my normal Intel or Radeon Vulkan driver on my normal Linux desktop, I get this crash:

xvfb-run godot-git
Godot Engine v4.0.dev.custom_build.f73374629 - https://godotengine.org
vulkan: No DRI3 support detected - required for presentation
Note: you can probably enable DRI3 in your Xorg config
vulkan: No DRI3 support detected - required for presentation
Note: you can probably enable DRI3 in your Xorg config
ERROR: Could not find both graphics and present queues

   at: _initialize_queues (drivers/vulkan/vulkan_context.cpp:590)
handle_crash: Program crashed with signal 11
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[1] /lib64/libc.so.6(+0x3a3a0) [0x7fbab52143a0] (??:0)
[2] VulkanContext::_update_swap_chain(VulkanContext::Window*) (/home/akien/Projects/godot/godot.git/drivers/vulkan/vulkan_context.cpp:945)
[3] VulkanContext::_window_create(int, VkSurfaceKHR_T*, int, int) (/home/akien/Projects/godot/godot.git/drivers/vulkan/vulkan_context.cpp:717)
[4] VulkanContextX11::window_create(int, unsigned long, _XDisplay*, int, int) (/home/akien/Projects/godot/godot.git/platform/linuxbsd/vulkan_context_x11.cpp:49)
[5] DisplayServerX11::_create_window(DisplayServer::WindowMode, unsigned int, Rect2i const&) (/home/akien/Projects/godot/godot.git/platform/linuxbsd/display_server_x11.cpp:3580)
[6] DisplayServerX11::DisplayServerX11(String const&, DisplayServer::WindowMode, unsigned int, Vector2i const&, Error&) (/home/akien/Projects/godot/godot.git/platform/linuxbsd/display_server_x11.cpp:3854 (discriminator 1))
[7] DisplayServerX11::create_func(String const&, DisplayServer::WindowMode, unsigned int, Vector2i const&, Error&) (/home/akien/Projects/godot/godot.git/platform/linuxbsd/display_server_x11.cpp:3416)
[8] DisplayServer::create(int, String const&, DisplayServer::WindowMode, unsigned int, Vector2i const&, Error&) (/home/akien/Projects/godot/godot.git/servers/display_server.cpp:608)
[9] Main::setup2(unsigned long) (/home/akien/Projects/godot/godot.git/main/main.cpp:1462 (discriminator 2))
[10] Main::setup(char const*, int, char**, bool) (/home/akien/Projects/godot/godot.git/main/main.cpp:1384)
[11] godot-git(main+0x101) [0x1d14773] (/home/akien/Projects/godot/godot.git/platform/linuxbsd/godot_linuxbsd.cpp:51)
[12] /lib64/libc.so.6(__libc_start_main+0xea) [0x7fbab5200d1a] (??:0)
[13] godot-git(_start+0x2a) [0x1d145ca] (??:?)
-- END OF BACKTRACE --
/usr/bin/xvfb-run: line 181: 1312824 Aborted                 (core dumped) DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE "$@" 2>&1

[qarmin]

Same crash is now shown with Lavapipe

On Nvidia graphics running xvfb-run(without LavaPipe or similar) just shows

ERROR: Condition "err" is true. Returning: ERR_CANT_CREATE
   at: _create_physical_device (drivers/vulkan/vulkan_context.cpp:351)
ERROR: Could not initialize Vulkan
   at: DisplayServerX11 (platform/linuxbsd/display_server_x11.cpp:3784)

[c0d1f1ed]

Unfortunately, I don't think this is feasible yet. There is no stable software implementation of Vulkan yet, which is required to render Godot without having a GPU available. There are some efforts ongoing like Kazan, SwiftShader and Mesa's Vallium renderer, but it seems none of them have reached a state where they can be used as a daily driver yet.

Here's hoping we'll be able to do this before Godot 4.0 is released

SwiftShader has been a conformant Vulkan 1.1 driver implementation since 2019-09-16. Since then, more features have been added, and games based on Unity's Universal Render Pipeline are supported. Please give it another try for running Godot, and if any issues are encountered feel free to file a bug at https://g.co/swiftshaderbugs.

[qarmin]

For now I don't think that SwiftShader is suitable for Godot, because I can't find easy way to install it on Ubuntu inside Github CI(no PPA or deb package for Ubuntu 20.04, compilation will take too much time).

[c0d1f1ed]

compilation will take too much time

Thanks for the feedback. Note that the compilation time can be reduced drastically by using the Subzero JIT instead of the LLVM JIT. Specify -DREACTOR_BACKEND=Subzero to cmake.

I hope that helps. We'll look into creating a package, but note that SwiftShader evolves fast, and customizing a Debug build can provide a lot more useful information when issues are detected.

[pdfrod]

I tried running the editor with Xvfb and SwiftShader. It kind of works - it runs but there are some glitches and it's quite slow:

[qarmin]

Looks that SwiftShader support even Vulkan 1.2, but the main task of CPU based implementations of Vulkan in Godot is ability to run them inside Ci, but still I cannot find any method how install it in ~5 minutes(which is I think even a little too high value), so for now I'm looking at Lavapipe which for now crashes when opening editor and support only Vulkan 1.1, but probably it will be available out of box without needing to install any software.

[Calinou]

but the main task of CPU based implementations of Vulkan in Godot is ability to run them inside Ci, but still I cannot find any method how install it in ~5 minutes(which is I think even a little too high value)

This could be solved by providing precompiled builds of SwiftShader. These will have to be compiled on an old enough Linux distribution to be compatible with as many distributions as possible (Ubuntu 18.04 should be an OK baseline for this nowadays).

Maybe someone could fork the SwiftShader repository, add a GitHub Actions setup that uploads compiled artifacts, then manually download one of its artifacts and upload it to GitHub Releases for easy downloading from another CI system.

[qarmin]

I compiled SwiftShader with this instructions in Ubuntu 20.04 Docker:

apt update
apt install -y git build-essential python3 python wget zlib1g-dev libx11-dev xvfb libxext-dev

wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-x86_64.sh
chmod +x cmake-3.20.0-linux-x86_64.sh 
./cmake-3.20.0-linux-x86_64.sh --prefix=/usr/local --exclude-subdir


git clone https://swiftshader.googlesource.com/SwiftShader
cd SwiftShader/build
cmake ..
cmake --build . -j6 # --parallel option takes more than 20GB Ram on my machine

Next I took on my computer(also Ubuntu 20.04) libvk_swiftshader.so, libvulkan.so.1files and I create also vk_swiftshader_icd.json file with this content:

{
    "file_format_version" : "1.0.0",
    "ICD": {
        "library_path": "/home/rafal/Downloads/AA/libvk_swiftshader.so",
        "api_version" : "1.0.0"
    }
}

And executing VK_ICD_FILENAMES=./vk_swiftshader_icd.json godot4 -e shows same error like Lavapipe

ERROR: No platform surface extension found, is a driver installed?
   at: _initialize_extensions (drivers/vulkan/vulkan_context.cpp:287)
ERROR: Could not initialize Vulkan
   at: DisplayServerX11 (platform/linuxbsd/display_server_x11.cpp:4013)
ERROR: Unable to create DisplayServer, all display drivers failed.
   at: setup2 (main/main.cpp:1541)

Compiled libvk_swiftshader.so file - Swift.zip

[pdfrod]

@qarmin I tested with what compiled library and I'm getting the same error about not finding drivers. It's weird though, because if I follow your exact build exacts steps, the resulting library works fine for me.

Here's a working verion: libvk_swiftshader.so.tar.gz

And a Docker file that works for Ubuntu 18.04:

FROM ubuntu:18.04 AS builder

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get install -y --no-install-recommends \
      ca-certificates git libx11-dev libxext-dev make python3 \
      software-properties-common wget zlib1g-dev
RUN add-apt-repository ppa:ubuntu-toolchain-r/test
RUN apt-get install -y --no-install-recommends gcc-9 g++-9
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 1
RUN update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-9 1
RUN update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 1
RUN update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 1
RUN wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-x86_64.sh
RUN sh cmake-3.20.0-linux-x86_64.sh --prefix=/usr/local --exclude-subdir

ENV COMMIT=fb53aa2bb7dad9de683d963b5e5d30c40bbf16e1

RUN git clone https://github.com/google/swiftshader.git
WORKDIR /swiftshader
RUN git checkout $COMMIT
WORKDIR /swiftshader/build
RUN cmake ..
RUN cmake --build . -j8


FROM busybox:1.32.1
COPY --from=builder /swiftshader/build/Linux/* /build/

I also had to comment a validation in drivers/vulkan/rendering_device_vulkan.cpp about Tessellation.