Headless Rendering with latest Unity 2019.3 version

[JohnBergago]

Hello,
with Unity 2019.3 being in Beta, I'm wondering if there is any possibility to run a standalone without any display and xserver.

From my own tests I can tell, that running a Unity standalone with vulkan renderer in headless mode with

$ mystandalone -headless

works. It opens a window, which stays black, but the visual observations are rendered correctly. I have to correct: It doesn't work. Passing the headless option still results in no rendered images. However, as soon as I try the same on a headless machine with no display nor xserver it won't start.

The Player.log says:

Mono path[0] = '/home/user/path/to/mystandalone_Data/Managed'
Mono config path = '/home/user/path/to/mystandalone_Data/MonoBleedingEdge/etc'
Preloaded 'libgrpc_csharp_ext.x64.so'
Preloaded 'ScreenSelector.so'
Desktop is 0 x 0 @ 0 Hz
[Vulkan init] extensions: count=15
[Vulkan init] extensions: name=VK_KHR_device_group_creation, enabled=0
[Vulkan init] extensions: name=VK_KHR_display, enabled=1
[Vulkan init] extensions: name=VK_KHR_external_fence_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_physical_device_properties2, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_surface_capabilities2, enabled=0
[Vulkan init] extensions: name=VK_KHR_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_xcb_surface, enabled=0
[Vulkan init] extensions: name=VK_KHR_xlib_surface, enabled=1
[Vulkan init] extensions: name=VK_EXT_acquire_xlib_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_debug_report, enabled=0
[Vulkan init] extensions: name=VK_EXT_debug_utils, enabled=0
[Vulkan init] extensions: name=VK_EXT_direct_mode_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_display_surface_counter, enabled=0
Vulkan detection: 2
Initialize engine version: 2019.3.0f5 (44796c9d3c2c)
[Subsystems] Discovering subsystems at path /home/user/path/to/mystandalone_Data/UnitySubsystems
GfxDevice: creating device client; threaded=1
[Vulkan init] extensions: count=15
[Vulkan init] extensions: name=VK_KHR_device_group_creation, enabled=0
[Vulkan init] extensions: name=VK_KHR_display, enabled=1
[Vulkan init] extensions: name=VK_KHR_external_fence_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_physical_device_properties2, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_surface_capabilities2, enabled=0
[Vulkan init] extensions: name=VK_KHR_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_xcb_surface, enabled=0
[Vulkan init] extensions: name=VK_KHR_xlib_surface, enabled=1
[Vulkan init] extensions: name=VK_EXT_acquire_xlib_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_debug_report, enabled=0
[Vulkan init] extensions: name=VK_EXT_debug_utils, enabled=0
[Vulkan init] extensions: name=VK_EXT_direct_mode_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_display_surface_counter, enabled=0
[Vulkan init] extensions: count=81
[Vulkan init] extensions: name=VK_KHR_8bit_storage, enabled=0
[Vulkan init] extensions: name=VK_KHR_16bit_storage, enabled=0
[Vulkan init] extensions: name=VK_KHR_bind_memory2, enabled=0
[Vulkan init] extensions: name=VK_KHR_create_renderpass2, enabled=0
[Vulkan init] extensions: name=VK_KHR_dedicated_allocation, enabled=1
[Vulkan init] extensions: name=VK_KHR_depth_stencil_resolve, enabled=0
[Vulkan init] extensions: name=VK_KHR_descriptor_update_template, enabled=1
[Vulkan init] extensions: name=VK_KHR_device_group, enabled=0
[Vulkan init] extensions: name=VK_KHR_draw_indirect_count, enabled=0
[Vulkan init] extensions: name=VK_KHR_driver_properties, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_fence, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_fence_fd, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory_fd, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore_fd, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_memory_requirements2, enabled=1
[Vulkan init] extensions: name=VK_KHR_image_format_list, enabled=1
[Vulkan init] extensions: name=VK_KHR_maintenance1, enabled=1
[Vulkan init] extensions: name=VK_KHR_maintenance2, enabled=0
[Vulkan init] extensions: name=VK_KHR_maintenance3, enabled=0
[Vulkan init] extensions: name=VK_KHR_multiview, enabled=0
[Vulkan init] extensions: name=VK_KHR_push_descriptor, enabled=0
[Vulkan init] extensions: name=VK_KHR_relaxed_block_layout, enabled=0
[Vulkan init] extensions: name=VK_KHR_sampler_mirror_clamp_to_edge, enabled=1
[Vulkan init] extensions: name=VK_KHR_sampler_ycbcr_conversion, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_atomic_int64, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_draw_parameters, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_float16_int8, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_float_controls, enabled=0
[Vulkan init] extensions: name=VK_KHR_storage_buffer_storage_class, enabled=0
[Vulkan init] extensions: name=VK_KHR_swapchain, enabled=1
[Vulkan init] extensions: name=VK_KHR_swapchain_mutable_format, enabled=0
[Vulkan init] extensions: name=VK_KHR_variable_pointers, enabled=0
[Vulkan init] extensions: name=VK_KHR_vulkan_memory_model, enabled=0
[Vulkan init] extensions: name=VK_EXT_blend_operation_advanced, enabled=0
[Vulkan init] extensions: name=VK_EXT_buffer_device_address, enabled=0
[Vulkan init] extensions: name=VK_EXT_conditional_rendering, enabled=0
[Vulkan init] extensions: name=VK_EXT_conservative_rasterization, enabled=0
[Vulkan init] extensions: name=VK_EXT_depth_clip_enable, enabled=0
[Vulkan init] extensions: name=VK_EXT_depth_range_unrestricted, enabled=0
[Vulkan init] extensions: name=VK_EXT_descriptor_indexing, enabled=0
[Vulkan init] extensions: name=VK_EXT_discard_rectangles, enabled=0
[Vulkan init] extensions: name=VK_EXT_display_control, enabled=0
[Vulkan init] extensions: name=VK_EXT_global_priority, enabled=0
[Vulkan init] extensions: name=VK_EXT_host_query_reset, enabled=0
[Vulkan init] extensions: name=VK_EXT_inline_uniform_block, enabled=0
[Vulkan init] extensions: name=VK_EXT_memory_budget, enabled=0
[Vulkan init] extensions: name=VK_EXT_pci_bus_info, enabled=0
[Vulkan init] extensions: name=VK_EXT_post_depth_coverage, enabled=0
[Vulkan init] extensions: name=VK_EXT_sample_locations, enabled=0
[Vulkan init] extensions: name=VK_EXT_sampler_filter_minmax, enabled=0
[Vulkan init] extensions: name=VK_EXT_scalar_block_layout, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_subgroup_ballot, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_subgroup_vote, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_viewport_index_layer, enabled=0
[Vulkan init] extensions: name=VK_EXT_transform_feedback, enabled=0
[Vulkan init] extensions: name=VK_EXT_vertex_attribute_divisor, enabled=0
[Vulkan init] extensions: name=VK_NV_clip_space_w_scaling, enabled=0
[Vulkan init] extensions: name=VK_NV_compute_shader_derivatives, enabled=0
[Vulkan init] extensions: name=VK_NV_cooperative_matrix, enabled=0
[Vulkan init] extensions: name=VK_NV_corner_sampled_image, enabled=0
[Vulkan init] extensions: name=VK_NV_dedicated_allocation, enabled=0
[Vulkan init] extensions: name=VK_NV_dedicated_allocation_image_aliasing, enabled=0
[Vulkan init] extensions: name=VK_NV_device_diagnostic_checkpoints, enabled=0
[Vulkan init] extensions: name=VK_NV_fill_rectangle, enabled=0
[Vulkan init] extensions: name=VK_NV_fragment_coverage_to_color, enabled=0
[Vulkan init] extensions: name=VK_NV_fragment_shader_barycentric, enabled=0
[Vulkan init] extensions: name=VK_NV_framebuffer_mixed_samples, enabled=0
[Vulkan init] extensions: name=VK_NV_geometry_shader_passthrough, enabled=0
[Vulkan init] extensions: name=VK_NV_mesh_shader, enabled=0
[Vulkan init] extensions: name=VK_NV_sample_mask_override_coverage, enabled=0
[Vulkan init] extensions: name=VK_NV_representative_fragment_test, enabled=0
[Vulkan init] extensions: name=VK_NV_scissor_exclusive, enabled=0
[Vulkan init] extensions: name=VK_NV_shader_image_footprint, enabled=0
[Vulkan init] extensions: name=VK_NV_shader_subgroup_partitioned, enabled=0
[Vulkan init] extensions: name=VK_NV_shading_rate_image, enabled=0
[Vulkan init] extensions: name=VK_NV_viewport_array2, enabled=0
[Vulkan init] extensions: name=VK_NV_viewport_swizzle, enabled=0
[Vulkan init] extensions: name=VK_NVX_device_generated_commands, enabled=0
[Vulkan init] extensions: name=VK_NVX_multiview_per_view_attributes, enabled=0
Vulkan API version 1.1.0 (1.1.99 supported by driver)
Vulkan vendor=[NVIDIA] id=[10de]
Vulkan renderer=[GeForce RTX 2080 Ti] id=[1e04]
Vulkan device type 2
Vulkan driverversion=[0x6b8c8000] uint=[1804369920]
Vulkan PSO: cache data not found [path=/tmp/DefaultCompany/mystandalone_project/vulkan_pso_cache.bin]
Caught fatal signal - signo:11 code:1 errno:0 addr:(nil)
Obtained 11 stack frames.
#0  0x007f0933c473f9 in DisplayManagerVK::Initialize()
#1  0x007f0933c4f8b7 in CreateVKGfxDevice()
#2  0x007f093404691f in CreateRealGfxDevice(GfxDeviceRenderer)
#3  0x007f0933bc2aee in CreateClientGfxDevice(GfxDeviceRenderer, GfxCreateDeviceFlags)
#4  0x007f0934047217 in CreateGfxDevice(GfxDeviceRenderer, GfxCreateDeviceFlags)
#5  0x007f09340475a1 in InitializeGfxDevice()
#6  0x007f0933f35456 in InitializeEngineGraphics(bool)
#7  0x007f0933f45380 in PlayerInitEngineGraphics(bool)
#8  0x007f09340d69a7 in PlayerMain(int, char**)
#9  0x007f09329a7b97 in __libc_start_main
#10 0x00000000400569 in _start

So it seems like it initializes everything correctly until it crashes. I guess its because it is unable to open a new window? As this machine is used by multiple users via ssh who don't have root permissions, the virtual x server solution is not suitable.

Do you have any idea how the headless rendering could work anyway? It feels really close to the solution already.

[andrewcoh]

Currently, we still only support running headlessly with x server. Also, I see you mentioned visual observations, which cannot be collected by an agent when running in headless mode. See here

[MarcoMeter]

@andrewcoh
Do you know or could you check if EGL is a suitable alternative to X?

[surfnerd]

Hi @MarcoMeter,
EGL is being explored but we don't have a timeline on it yet. I'll create an internal ticket for it and we'll update this thread when we have more information. Thanks.

[surfnerd]

Hi @JohnBergago and @MarcoMeter,
I've logged this feature request as MLA-557 and will update this ticket as changes happen. Thanks.

[stale]

This issue has been automatically marked as stale because it has not had activity in the last 14 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it has not had activity in the last 28 days. If this issue is still valid, please ping a maintainer. Thank you for your contributions.

[tsunamaru]

@surfnerd kindly ping. Any update?

[MarcoMeter]

The last state (July) that I know of is that the engine team is still working on CPU rendering. It is not clear when this will be available.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.