feat: Replace GLES display renderer with native Vulkan compositor

[ghost]

This PR replaces the upstream GLSurfaceView/GLES display renderer with a native Vulkan compositor backed by an Android Surface/ANativeWindow.

The new path keeps rendering compositor-first: windows, cursor, and post-processing effects are composed into a Vulkan swapchain image and presented with vkQueuePresentKHR. It also supports zero-copy AHardwareBuffer import for suitable DRI3/PRESENT pixmaps, but those imported buffers are still sampled by the Vulkan compositor rather than routed through a separate SurfaceControl scanout path.

Vulkan Setup

The compositor creates a Vulkan 1.1 instance with:

• VK_KHR_surface
• VK_KHR_android_surface

It enumerates physical devices through the active Android Vulkan loader/ICD, selects a graphics queue, checks presentation support for the Android surface, and requires VK_KHR_swapchain before creating the swapchain path.

AHardwareBuffer import is enabled only when the required external-memory extensions are all available:

• VK_ANDROID_external_memory_android_hardware_buffer
• VK_KHR_external_memory
• VK_KHR_dedicated_allocation
• VK_KHR_get_memory_requirements2

Optional helpers such as VK_EXT_queue_family_foreign and VK_KHR_sampler_ycbcr_conversion are enabled only when exposed by the driver.

Swapchain And Presentation

Swapchain creation queries surface capabilities, formats, transforms, present modes, and image-count limits.

FIFO is the default present mode because it is guaranteed by Vulkan. MAILBOX or IMMEDIATE can be requested, but the renderer validates support and falls back to FIFO when unavailable.

The swapchain image count is clamped to the surface limits, and each swapchain image gets its own image view, framebuffer, and render-finished semaphore.

Scene Submission

The Java side no longer issues GLES draw calls per window. Instead, it serializes the current display state into a compact direct ByteBuffer containing:

• viewport and scissor state
• screen transform
• window count
• native texture handles
• window geometry
• UV bounds
• cursor state
• swap-RB flag
• screen dimensions
• active effect parameters

Java builds the scene while holding the relevant X/render locks, releases them, then native code snapshots the scene under scene_mutex before acquire, command recording, queue submit, and present under renderer-side synchronization.

Texture Uploads

Texture ownership moved from GLES texture IDs to native Vulkan texture objects.

Software drawable updates are tracked with dirty rectangles. When possible, only the dirty bounds are copied into staging memory and uploaded with vkCmdCopyBufferToImage. Multiple dirty drawable updates can be batched into a single staging upload pass before frame rendering, reducing the cost of small X11 damage updates.

Pending uploads are submitted before the frame render, and same-queue ordering plus transfer-to-shader-read barriers make the updated texture contents visible to the compositor frame.

DRI3 / PRESENT AHardwareBuffer Path

The DRI3/PRESENT path can import suitable AHardwareBuffer pixmaps directly into Vulkan. When a PRESENT pixmap covers the target drawable, the window can use that pixmap as its compositor source instead of copying pixels into the window backing buffer.

Native code imports the buffer with VK_EXTERNAL_MEMORY_HANDLE_TYPE_ANDROID_HARDWARE_BUFFER_BIT_ANDROID, creates a Vulkan image/image view backed by the Android buffer, and samples it as a compositor texture.

External-format buffers are rejected unless the current descriptor/sampler path can represent them correctly, avoiding invalid Vulkan usage on stricter drivers.

Frame Lifecycle

Frame rendering is explicit and ordered:

1. Wait for the current frame fence.
2. Snapshot the scene.
3. Acquire a swapchain image.
4. Record the window, cursor, and effect passes.
5. Submit to the graphics queue.
6. Present with vkQueuePresentKHR.

Texture destruction is deferred through a small graveyard so native texture resources are not freed while still reachable by in-flight GPU work.

Scheduling And Frame Pacing

X/window/PRESENT events now use a coalesced render request that posts to Android Choreographer, so multiple updates before the next display tick collapse into one render.

Renderer-side FPS limiting is performed natively after submit/present. The limiter uses the same absolute-heartbeat pacing model as the X client path:

• 100 ms resync window
• 0.5 ms early-sleep threshold
• 4 ms final precision window

This avoids catch-up bursts after long stalls while keeping renderer pacing aligned with the existing X client timing logic.

Comparison With Ludashi

Compared with Ludashi’s more aggressive SurfaceControl scanout renderer, this implementation is a compositor-first Vulkan path.

It does not try to bypass composition in fullscreen/native-mode cases. Instead, it prioritizes:

• lower Vulkan requirements
• validated swapchain and present-mode behavior
• safer AHardwareBuffer import semantics
• dirty-rect and batched texture uploads
• deterministic frame pacing
• broader compatibility across Android devices and Vulkan drivers

Thanks to Ludashi and StevenMXZ for the inspiration and for proving that a Vulkan-based Winlator display renderer is practical on Android. This implementation takes a more conservative compositor-first approach, but their work helped validate the direction.