Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Direct display feature for the Vulkan backend #3759

Merged
merged 53 commits into from Jun 1, 2021

Conversation

Uniformbuffer3
Copy link
Contributor

@Uniformbuffer3 Uniformbuffer3 commented May 12, 2021

Hi, i have implemented the ability to draw directly on screen for the Vulkan backend using the VK_KHR_display extension.
Also the VK_EXT_display_control extension has been implemented, so that it is possible to get notifications about disconnecting monitors and it gives the ability to control the power state of the monitor (on, off, suspended).

The "quad" example has been modified so that, if DIRECT_DISPLAY env variable is setted, it will try to draw on the first monitor returned from the system. Since "quad" is an example, i have clearly separated the two paths with a big if statement, so that users do not get confused about this new feature.
The display power control has been tested with a separate test crate, but i can of course provide the code if you want it.

Also to be pointed out, Vulkan will try to enumerate all possible monitors that are not controlled by other systems. If all the monitors are already controlled by some programs (like an already running compositor/desktop environment), the enumerate function will return an empty list.

This is mainly a Linux feature, but i have discovered that it is possible to get the control of a monitor even on Windows if that monitor is not controlled by the Windows compositor (but unfortunately i can make tests only on Linux).

If you are interested in such feature, i would like to know your opinion about this and get your feedback so that i can improve it .

Thanks for the dedicated time,
Have a good day

Tests:
To test the direct display surface creation, a special path has been implemented into the quad example. To use it the DIRECT_DISPLAY env variable need to be present before launching the example. Please keep in mind that if the example (using this mode) is launched on an already running desktop environment, there is an high chance that it will fail beause there is no available monitor the application can take the control (because already controlled by the running compositor). To be sure it works launch it directly on a tty. The example will run for 5 seconds and terminate. On Linux and based on the driver implementation, the tty could not return automatically in text mode. Switching from and back to the tty will bring back the text mode.
It is also possible to use this feature on Windows by telling the Windows compositor to do not use one of the monitors.
I'm not too familiar with Windows, but i have found a git repository that explain how to this: https://github.com/nvpro-samples/gl_render_vk_ddisplay
That monitor will be then available for the display enumeration of this feature and the Gfx logo should appear it (still, i don't have a Windows machine, so this has not been tested on Windows).

Also functions from VK_EXT_display_control has been implemented and tested using this quick and dirty test program: https://github.com/Uniformbuffer3/gfx_direct_display_test .
On the main function are present the functions that will test these features (remove the comment syntax from the desired function).

Tests result on my system (Intel Broadwell + Nvidia Gtx 850M Optimus system):

  • Display power state control ✔️
  • Display hotplug events ⏩ (seems to be unsupported on my system)
  • Display first pixel out event ⏩ (seems to be unsupported on my system, but i really don't know what's wrong here)

I suppose some of these tests fails due to the fact that mine is an Optimus system; they are famous to have weird/unresolved bugs related to display syncronization and comunication due to the way Optimus systems works. If someone want to test this on a non-Optimus system, it would be very appreciated.

Fabio Sgamma and others added 30 commits April 7, 2021 02:46
…a display surface, making possible to render directly to the display without windowing system.

Currently this is achieved using Vulkan extensions, so this is a Vulkan only feature.
Removed "unsafe" from functions since there is no chance of undefined behaviour
Added missing `image_extent` parameter to `create_display_surface`, even if from preliminary tests, seems to have no effects. The surface seems to use only DisplayMode to select the surface size.
The new api should be more simply while keeping all the features offered by vulkan. It is also less vulkan-specific, maybe allowing in the future to other rendering api to use the direct display path.
…ome checks on the create_display_plane_surface to deny undefined behaviours.
…_KHR_display extension.

Changed the error structure returned by enumerate_available_displays to include the possibility that the system does not support the feature.
Since the enumerate_available_displays is the "entry point" to take advantage of the feature (and others related functions cannot be called before that), the checking has been added only to that function.
…splay, and it require ust V1.0 .

So checks about vulkan version on enumerate_available_displays has been removed.
Also removed the vulkan version check on the instance creation when VK_KHR_display is added to the extension list.
Merged a function from src/backend/vulkan/src/display.rs into the only function that use it and removed the file.
Moved all the new functions except `create_display_plane_surface` to `PhysicalDevice`.
Implemented From<(image::Size,image::Size)> for Extent2D.
Implemented From<(image::TexelCoordinate,image::TexelCoordinate)> for Offset2D.
Replaced all the tuple instances in hal::display with the appropriate Offset2D and Extent2D.
Replaced (u32,u32) with Extend2D on the "create_display_plane_surface" function for all the backends.
Added to direct display related function a cfg to target linux only since it is a linux only feature.
Direct dispay related functions for non-linux backend (like dx and metal backends) has been commented since they will never be compiled.
I cannot test all the platform, that's why they are only commented. If the CI tests will pass, i will delete the commented functions.
Removed "enumerate_builtin_display_modes" and builtin modes will be returned in the "enumerate_available_displays" function.
…d by vulkan validation layers that i need to investigate.
Added From<Extent2D> for (image::Size, image::Size) implementation so that Extent2D can be converted into tuple.
Fixed a problem with the resolution of the quad example for the direct display path: it was using the first display mode available instead of using the resolution exposed by the DIMS variable. Now it look for the DIMS resolution into the built-in modes and will be created if not found. If the creation fails, the first available built-in mode will be used and the renderer resolution will be setted accordingly.
On the quad example, setted the display plane size as the resolution of the used display mode, so that if it is not possible to use DIMS, it will be anyway a valid resolution.
Changed direct display function to unsafe
Removed leftover code.
Removed custom partial eq implementation that is no more needed.
Changed the return error of `create_display_plane` to OutOfMemory because the only other possible error cannot be triggered due to the shape of the api.
Fixed non camel case enums on the quad example.
@kvark
Copy link
Member

kvark commented May 30, 2021

@Uniformbuffer3 what is the state of this PR?

@Uniformbuffer3
Copy link
Contributor Author

Generally is ready, display and surface related functions are working correctly, but i have a problem with a function that comes from VK_EXT_display_control: i cannot test Device::register_device_event because my system does not support it. I suppose it is due to the fact that it is a Optimus system, famous to have problems with display synchronization stuff.
For Device::register_display_event i will test it tomorrow.
VK_EXT_display_control also offer the ability to control the display power state (implemented as Device::set_display_power_state), and it works correctly instead.
Tomorrow i will focus more on this PR, so it will probably be ready for review.
I'm sorry to be slow, unfortunately for the project i'm working on i need to build different components in parallel (including the external memory PR)

@Uniformbuffer3
Copy link
Contributor Author

Uniformbuffer3 commented May 31, 2021

I have tested the Device::register_display_event and i got weird validation errors like Couldn't find VkDisplayKHR Object 0x...., but i have checked pointer addresses using the api dump layer and they match, so i don't know what's the cause.

Thread 0, Frame 0, Time 46091 us:
vkGetPhysicalDeviceDisplayPropertiesKHR(physicalDevice, pPropertyCount, pProperties) returns VkResult VK_SUCCESS (0):
    physicalDevice:                 VkPhysicalDevice = 0x55d7616ba940
    pPropertyCount:                 uint32_t* = 1
    pProperties:                    VkDisplayPropertiesKHR* = NULL

Thread 0, Frame 0, Time 49040 us:
vkGetPhysicalDeviceDisplayPropertiesKHR(physicalDevice, pPropertyCount, pProperties) returns VkResult VK_SUCCESS (0):
    physicalDevice:                 VkPhysicalDevice = 0x55d7616ba940
    pPropertyCount:                 uint32_t* = 1
    pProperties:                    VkDisplayPropertiesKHR* = 0x55d7616c9db0
        pProperties[0]:                 VkDisplayPropertiesKHR = 0x55d7616c9db0:
            display:                        VkDisplayKHR = 0xfa21a40000000003
            displayName:                    const char* = "monitor"
            physicalDimensions:             VkExtent2D = 0x55d7616c9dc0:
                width:                          uint32_t = 423
                height:                         uint32_t = 238
            physicalResolution:             VkExtent2D = 0x55d7616c9dc8:
                width:                          uint32_t = 1600
                height:                         uint32_t = 900
            supportedTransforms:            VkSurfaceTransformFlagsKHR = 1 (VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR)
            planeReorderPossible:           VkBool32 = 0
            persistentContent:              VkBool32 = 0

Thread 0, Frame 0, Time 51998 us:
vkGetDisplayModePropertiesKHR(physicalDevice, display, pPropertyCount, pProperties) returns VkResult VK_SUCCESS (0):
    physicalDevice:                 VkPhysicalDevice = 0x55d7616ba940
    display:                        VkDisplayKHR = 0xfa21a40000000003
    pPropertyCount:                 uint32_t* = 1
    pProperties:                    VkDisplayModePropertiesKHR* = NULL

Thread 0, Frame 0, Time 52016 us:
vkGetDisplayModePropertiesKHR(physicalDevice, display, pPropertyCount, pProperties) returns VkResult VK_SUCCESS (0):
    physicalDevice:                 VkPhysicalDevice = 0x55d7616ba940
    display:                        VkDisplayKHR = 0xfa21a40000000003
    pPropertyCount:                 uint32_t* = 1
    pProperties:                    VkDisplayModePropertiesKHR* = 0x55d7616c9780
        pProperties[0]:                 VkDisplayModePropertiesKHR = 0x55d7616c9780:
            displayMode:                    VkDisplayModeKHR = 0xf56c9b0000000004
            parameters:                     VkDisplayModeParametersKHR = 0x55d7616c9788:
                visibleRegion:                  VkExtent2D = 0x55d7616c9788:
                    width:                          uint32_t = 1600
                    height:                         uint32_t = 900
                refreshRate:                    uint32_t = 60062

Thread 0, Frame 0, Time 52202 us:
vkRegisterDisplayEventEXT(device, display, pDisplayEventInfo, pAllocator, pFence) returns VkResult[2021-05-31T16:13:27Z ERROR gfx_backend_vulkan] 
    VALIDATION [UNASSIGNED-Threading-Info (0x5d6b67e2)] : Validation Error: [ UNASSIGNED-Threading-Info ] Object 0: handle = 0xfa21a40000000003, type = VK_OBJECT_TYPE_DISPLAY_KHR; | MessageID = 0x5d6b67e2 | Couldn't find VkDisplayKHR Object 0xfa21a40000000003. This should not happen and may indicate a bug in the application.
    object info: (type: DISPLAY_KHR, hndl: 0xfa21a40000000003)
    
[2021-05-31T16:13:27Z ERROR gfx_backend_vulkan] 
    VALIDATION [UNASSIGNED-Threading-Info (0x5d6b67e2)] : Validation Error: [ UNASSIGNED-Threading-Info ] Object 0: handle = 0xfa21a40000000003, type = VK_OBJECT_TYPE_DISPLAY_KHR; | MessageID = 0x5d6b67e2 | Couldn't find VkDisplayKHR Object 0xfa21a40000000003. This should not happen and may indicate a bug in the application.
    object info: (type: DISPLAY_KHR, hndl: 0xfa21a40000000003)
    
 VK_ERROR_OUT_OF_HOST_MEMORY (-1):
    device:                         VkDevice = 0x55d7616f3340
    display:                        VkDisplayKHR = 0xfa21a40000000003
    pDisplayEventInfo:              const VkDisplayEventInfoEXT* = 0x7ffe093037d8:
        sType:                          VkStructureType = VK_STRUCTURE_TYPE_DISPLAY_EVENT_INFO_EXT (1000091002)
        pNext:                          const void* = NULL
        displayEvent:                   VkDisplayEventTypeEXT = VK_DISPLAY_EVENT_TYPE_FIRST_PIXEL_OUT_EXT (0)
    pAllocator:                     const VkAllocationCallbacks* = NULL
    pFence:                         VkFence* = 0xfab64d0000000002

Still i have tested the other parts of the api and they work. I'm preparing a quick test program if someone want to test these features with a different system.

@Uniformbuffer3 Uniformbuffer3 marked this pull request as ready for review May 31, 2021 17:54
Copy link
Member

@kvark kvark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very nice work, top quality!
Noted a few concerns.

.buildconfig Outdated Show resolved Hide resolved
src/hal/src/adapter.rs Outdated Show resolved Hide resolved
src/hal/src/adapter.rs Outdated Show resolved Hide resolved
src/hal/src/display/mod.rs Outdated Show resolved Hide resolved
src/hal/src/display/mod.rs Outdated Show resolved Hide resolved
src/backend/vulkan/src/conv.rs Outdated Show resolved Hide resolved
src/backend/vulkan/src/conv.rs Outdated Show resolved Hide resolved
src/backend/vulkan/src/device.rs Outdated Show resolved Hide resolved
src/backend/vulkan/src/physical_device.rs Outdated Show resolved Hide resolved
@@ -750,6 +752,20 @@ impl PhysicalDevice {
None
};

let display_control = if enabled_extensions.contains(&vk::ExtDisplayControlFn::name()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to file an issue to Ash to expose this better, similarly to how DrawIndirectCount is exposed

Removed `.buildconfig` that has been added by mistake.
On `DisplayPlane` reduced the members by merging "min max" fields using `std::ops::Range`.
Removed leftover code.
Changed signature of `enumerate_displays` and `enumerate_compatible_planes` so that they return directly `Vec<display::Display<B>>`.
Removed from `create_display_mode` the UnsupportedFeature error since there is no chance it will be returned.
Replaced a check against `std::ptr::null()` with `is_null()`
… because it is not a promoted extension, so there is no need to use `ExtensionFn`, but just `Option<vk::ExtDisplayControlFn>`.
…can be builded from/to vulkan bits. Due to this, the related functions in conv has been removed, the same result can be achieved with just a line now.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants