Vulkan PVIDEO Resource Lifetime

Vulkan PVIDEO Resource Lifetime

Lifecycle: Performance - Unproven · Type: Resource reuse and GPU conversion
Reviewed source: 322986f4c5502b3fa887a9605ac8cd88e9b2c579
State: Repeated creation and synchronous upload behavior are source-confirmed. Workload impact is unmeasured.

Vulkan throughput pages: Overview · Submission & retirement · Native presentation · Dynamic uniforms · PVIDEO lifetime · GPU-resident consumers · Buffer placement

Status

upload_pvideo_image() calls create_pvideo_image() on every overlay upload. The creation routine destroys resources when absent or dimensions differ, but it has no same-size early return before vmaCreateImage(), vkCreateImageView(), and vkCreateSampler(). The inspected routine also does not publish the new width/height.

The upload then converts packed YUY2-style guest data to RGBA on the CPU, copies the expanded pixels through a staging buffer, and ends with a one-time submission that waits for queue idle.

Source seam:

• display.c

Why it matters

PVIDEO-active workloads can pay for object creation, CPU conversion, doubled upload size, and a synchronous submission on every refresh.

This is sharply bounded and should be measured before larger presentation rewrites.

Current path

flowchart TD
    A[PVIDEO upload] --> B[Create image]
    B --> C[Create image view]
    C --> D[Create sampler]
    D --> E[CPU YUY2 to RGBA conversion]
    E --> F[Flush staging memory]
    F --> G[Copy buffer to image]
    G --> H[Queue submit]
    H --> I[vkQueueWaitIdle]

Loading

Candidate path

Treat this as two independent experiments.

flowchart TD
    A[PVIDEO state] --> B{Existing resource matches dimensions/format?}
    B -->|Yes| C[Reuse image, view and sampler]
    B -->|No| D[Replace resources once]
    C --> E{GPU conversion experiment enabled?}
    D --> E
    E -->|No| F[Reference CPU conversion and upload]
    E -->|Yes| G[Upload packed bytes]
    G --> H[Convert/sample during display composition]
    F --> I[Display batch]
    H --> I

Loading

Stage 1: resource lifetime fix

• Return immediately when existing resources match.
• Replace resources only when dimensions/format actually change.
• Publish width/height only after successful creation.
• Clean up partial creation failures.

Stage 2: GPU conversion

Upload packed video bytes and perform YUV-to-RGB conversion in the display shader or a preceding GPU pass. Merge the upload dependency into the display batch rather than using a separate queue-idle operation.

What changes

Stage 1 removes repeated object creation. Stage 2 additionally removes the CPU pixel loop, reduces transfer bytes from expanded RGBA, and removes a dedicated synchronous submission.

Required guard

• Input width, height, pitch, offset, and packed format match the selected resource plan.
• Color conversion, clamping, chroma interpretation, crop, scale, and color key remain correct.
• Resource replacement cannot destroy an image still used by an in-flight display batch.
• Dirty tracking determines whether a new upload is needed.

Risks

• Treating a size match as proof that format or representation still matches.
• Reusing an object that an earlier command buffer still references.
• Visible rounding differences at color-key or conversion boundaries.
• Mishandling odd widths, unusual pitch, input/output crop, or overlay disable/enable.
• Measuring object-count reduction in a workload that rarely uses PVIDEO.

Evidence

pvideo_uploads
pvideo_image_creations
pvideo_view_creations
pvideo_sampler_creations
pvideo_replacements
CPU_conversion_us
packed_input_bytes
expanded_upload_bytes
pvideo_queue_idle_us
PVIDEO-active frame p95/p99

A constant-resolution sequence should produce many uploads but only one resource creation set.

How to test

1. Run a long constant-resolution overlay sequence and verify stable resource counts.
2. Change resolution and confirm one replacement per real change.
3. Compare converted pixels for color-key edges, crop, scale, pitch, and chroma boundaries.
4. Toggle PVIDEO, resize, reset, restore snapshots, and destroy the renderer with work in flight.
5. Benchmark resource reuse separately from GPU conversion so their effects remain attributable.

Related

• Vulkan Throughput Candidates
• Native Vulkan Presentation
• Submission and Resource Retirement
• Vulkan Renderer

Full technical reference

Primary reference: This subject page is the design and research reference for PVIDEO Resource Lifetime. Pull requests are implementation/test history only.
Lifecycle: Performance - Unproven · Mechanism: Resource reuse plus GPU conversion
Reviewed source: 322986f4c5502b3fa887a9605ac8cd88e9b2c579 · Acceptance class: intended exact or behavior-preserving for its admitted scope

1. Scope

Stop recreating PVIDEO image/view/sampler objects on every upload, then test packed upload and GPU-side YUV conversion in the display batch.

This entry remains Unproven: the source path and opportunity are concrete, but no accepted xemu runtime result has established that the candidate improves representative workloads. The page defines what must be bypassed, what remains mandatory, and how the hypothesis can be rejected.

2. Source seams

File / function	Role
hw/xbox/nv2a/pgraph/vk/display.c	create_pvideo_image(), upload_pvideo_image(), CPU conversion and one-time submission

3. Current execution path

flowchart TD
    N0["Overlay upload"]
    N1["Create image/view/sampler"]
    N0 --> N1
    N2["CPU packed-YUV to RGBA"]
    N1 --> N2
    N3["Upload expanded RGBA"]
    N2 --> N3
    N4["Separate submit"]
    N3 --> N4
    N5["vkQueueWaitIdle"]
    N4 --> N5
    N6["Display sample"]
    N5 --> N6

Loading

4. Candidate execution path

flowchart TD
    N0["Reuse matching resource set"]
    N1["Upload only changed packed bytes"]
    N0 --> N1
    N2["GPU converts/samples packed representation"]
    N1 --> N2
    N3["Order in display batch"]
    N2 --> N3
    N4["Replace resources only on real shape/format change"]
    N3 --> N4

Loading

5. Why it may help

The creation routine lacks a same-size early return before object creation and the inspected code does not publish width/height after creation. This is a sharply bounded source issue. GPU conversion is a separate larger experiment and must not be bundled when measuring the lifetime fix.

6. Work removed

• Per-refresh image/view/sampler creation
• CPU pixel conversion
• Expanded upload bandwidth
• Dedicated synchronous overlay submission

7. Work that remains mandatory

• Overlay state interpretation
• Color key/crop/scale
• Resource replacement on true change
• Reference CPU route
• Display ordering

8. Implementation sketch

if (pvideo.image != VK_NULL_HANDLE &&
    pvideo.width == width && pvideo.height == height &&
    pvideo.format == format) {
    stats.pvideo_resource_reuse++;
} else {
    retire_or_destroy_pvideo_resources(&pvideo);
    create_pvideo_resources(&pvideo, width, height, format);
    pvideo.width = width;
    pvideo.height = height;
    pvideo.format = format;
}

The sketch defines the intended boundary; it is not a complete patch and does not replace source review, ownership design, or focused tests.

9. Correctness contract

• Dimensions/format/usage must match before reuse.
• Old resources cannot be destroyed while an in-flight display batch references them.
• GPU conversion must reproduce pitch, chroma pairing, clamping, crop, scale and color-key behavior within the accepted contract.
• Upload dirtiness must be tracked separately from resource identity.

10. State, ownership, invalidation, and fallback

Stage 1 resources are persistent display-owned objects. Stage 2 packed buffers/images require batch lifetime tracking. Reset/resize/snapshot/renderer teardown invalidate or safely retire them.

The maintained reference path remains the fallback whenever admission, capability, lifetime, or validation is uncertain. A fast result with an unproven dependency is not acceptable as an exact optimization.

11. Advantages

• Very clear object-churn reduction
• Potentially halves upload bytes with packed representation
• Removes CPU conversion loop
• Can fold upload into display submission

12. Costs and failure modes

• Benefits only PVIDEO-active workloads
• Conversion rounding/color-key sensitivity
• In-flight replacement complexity
• Dirty tracking needed to avoid useless uploads

13. Evidence counters

Counters must be cheap, subsystem-owned, and snapshotted outside the hot loop. Record at least:

• pvideo_uploads
• image/view/sampler_creations
• resource_reuse_hits
• CPU_conversion_us
• packed_vs_RGBA_bytes
• pvideo_queue_idle_us
• PVIDEO frame p95/p99

A local reduction is not enough. Evidence must connect route reachability and removed work to fixed guest progress, CPU/GPU time where applicable, and p95/p99 behavior.

14. Qualification matrix

• Long fixed-resolution sequence
• Resolution/format changes
• Odd widths/pitch
• Color-key edges
• Crop/scale
• Enable/disable
• Several frames in flight
• Reset/snapshot/teardown
• Separate lifetime-only and GPU-conversion A/B

Correctness runs may use validation layers, detailed comparisons, and fault injection. Performance acceptance must use optimized Release builds with matching symbols and minimal instrumentation.

15. Acceptance rule

Promote resource reuse once lifecycle tests and creation counters prove it. Promote GPU conversion separately only with pixel/behavior validation and measured total benefit.

16. Rejection rule

Reject combined claims that cannot attribute benefit, or any design that reuses mismatched resources, destroys in-flight objects, or changes overlay behavior beyond contract.

17. Related architecture and history

Vulkan-Renderer · Performance-Unproven · Vulkan-Native-Presentation · Vulkan-Submission-and-Resource-Retirement

PR pages may be linked as supporting evidence, but they are not substitutes for this reference.