Implemented interop with externally managed GPU memory

[kekekeks]

See #9925

Also enables OpenGlControlBase on Windows via an ANGLE context powered by a separate Direct3D device.

[avaloniaui-team]

You can test this PR using the following package version. 11.0.999-cibuild0028835-beta. (feed url: https://pkgs.dev.azure.com/AvaloniaUI/AvaloniaUI/_packaging/avalonia-all/nuget/v3/index.json) [PRBUILDID]

[avaloniaui-team]

You can test this PR using the following package version. 11.0.999-cibuild0028883-beta. (feed url: https://pkgs.dev.azure.com/AvaloniaUI/AvaloniaUI/_packaging/avalonia-all/nuget/v3/index.json) [PRBUILDID]

[avaloniaui-team]

You can test this PR using the following package version. 11.0.999-cibuild0028890-beta. (feed url: https://pkgs.dev.azure.com/AvaloniaUI/AvaloniaUI/_packaging/avalonia-all/nuget/v3/index.json) [PRBUILDID]

[TomEdwardsEnscape]

Is VulkanDemoControl in the GpuInterop project supposed to work? It never renders anything, and FreeUsedCommandBuffers deadlocks the process when I resize the window.

I can get a working demo if I switch the window to D3D11DemoControl instead.

[kekekeks]

Which GPU are you using? I've tested the PR with NVIDIA Quadro 2000 on Linux and Radeon Pro 555 on Windows.

[Gillibald]

Freezes for me as well with Intel UHD Grafik 770
D3D11DemoControl works

[TomEdwardsEnscape]

Ah, of course. On laptops the low-power integrated GPU is used by default. I also have an Intel UHD integrated GPU, though I don't know model number.

I added this line to program startup and the Vulkan demo started to run correctly on my discrete NVidia GPU:

NativeLibrary.Load("nvapi64.dll"); // cues Nvidia drivers to run our windows on the system's dedicated GPU, not an integrated one

Unfortunately, activating the correct GPU with AMD's drivers is a lot harder in C#.

For platforms other than Windows I haven't a clue; this is something we need to work out for our product too.

[kekekeks]

We can probably expose GPU selection logic for Win32 backend since ANGLE runs on top of DirectX and have access to IDXGIAdapter.

It will, however, break possible WGL interop since WGL always uses system-selected GPU.

[llfab]

Same here. It works on NVidia. If I switch to the internal Intel GPU it deadlocks. On a pure Intel i7 1165G7 (Intel Iris XE) it crashes in
#######
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
at Silk.NET.Vulkan.Vk.AllocateMemory(Silk.NET.Vulkan.Device, Silk.NET.Vulkan.MemoryAllocateInfo ByRef, Silk.NET.Vulkan.AllocationCallbacks*, Silk.NET.Vulkan.DeviceMemory ByRef)
at GpuInterop.VulkanDemo.VulkanImage..ctor(GpuInterop.VulkanDemo.VulkanContext, UInt32, Avalonia.PixelSize, Boolean, UInt32)
at GpuInterop.VulkanDemo.VulkanSwapchainImage..ctor(GpuInterop.VulkanDemo.VulkanContext, Avalonia.PixelSize, Avalonia.Rendering.Composition.ICompositionGpuInterop, Avalonia.Rendering.Composition.CompositionDrawingSurface)
at GpuInterop.VulkanDemo.VulkanSwapchain.CreateImage(Avalonia.PixelSize)
at Avalonia.Rendering.SwapchainBase1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].BeginDrawCore(Avalonia.PixelSize, System.__Canon ByRef) at GpuInterop.VulkanDemo.VulkanSwapchain.BeginDraw(Avalonia.PixelSize, GpuInterop.VulkanDemo.VulkanImage ByRef) at GpuInterop.VulkanDemo.VulkanDemoControl.RenderFrame(Avalonia.PixelSize) at GpuInterop.DrawingSurfaceDemoBase.UpdateFrame() at Avalonia.Rendering.Composition.Compositor.CommitCore() at Avalonia.Rendering.Composition.Compositor.Commit() at Avalonia.Rendering.Composition.Compositor.<.ctor>b__19_0() at Avalonia.Threading.JobRunner+Job.Avalonia.Threading.JobRunner.IJob.Run() at Avalonia.Threading.JobRunner.RunJobs(System.Nullable1<Avalonia.Threading.DispatcherPriority>)
at Avalonia.Win32.Win32Platform.WndProc(IntPtr, UInt32, IntPtr, IntPtr)
at Avalonia.Win32.Interop.UnmanagedMethods.DispatchMessage(MSG ByRef)
at Avalonia.Win32.Interop.UnmanagedMethods.DispatchMessage(MSG ByRef)
at Avalonia.Win32.Win32Platform.RunLoop(System.Threading.CancellationToken)
at Avalonia.Threading.Dispatcher.MainLoop(System.Threading.CancellationToken)
at Avalonia.Controls.ApplicationLifetimes.ClassicDesktopStyleApplicationLifetime.Start(System.String[])
at Avalonia.ClassicDesktopStyleApplicationLifetimeExtensions.StartWithClassicDesktopLifetime(Avalonia.AppBuilder, System.String[], Avalonia.Controls.ShutdownMode)
at GpuInterop.Program.Main(System.String[])

We have ported this into our own Vulkan bindings and we see the very same effect. However, we had done a Vk 3D renderer integration into WPF in the past using the very same concept. A couple of things differ there as follows:

• D3D Texture is created with ResourceOptionFlags.SharedNthandle (which proved to be critical to the compatibility across different GPU setups
• The sharedHandle is created iwth DXGI.Resource1 interface using resource1.CreateSharedHandle()

That interop we made works on all platforms (pure Nvidia, Nvidia/Intel laptops, pure Intel laptops >= Intel Gen 10). I have put the main portion of the interop code here: https://github.com/llfab/Samples/tree/main/VulkanDxInterop/DXInterop

[BAndysc]

@kekekeks I was wondering, will it be possible to support Vulkan Control on macOS? Of course Vulkan is not officially available on macOS, because Apple... But I believe it could work via moltenVK?

Currently macOS openGL backend (AvaloniaNativeGlPlatformGraphics) doesn't support image sharing (Image sharing is not supported by the current backend), so that's where it fails, I wonder if it would be possible to support it despite begin limited to OpenGL 4.1 only? Or maybe without OpenGL 4.5 and those GL_EXT_memory_object extensions (used on Linux) there is no way to do this? Cheers!