Hot Path Vulkan Transient Buffer Growth

Vulkan Transient-Buffer Growth

Lifecycle: Performance - Implemented · Type: Resource policy
State: Present in current main.

Status

Field	State at reviewed main
Underlying optimization	Implemented
Advanced control	Implemented, Grow transient buffers to fit batches
Default	On
Apply boundary	Restart-latched
Reference route	Drain/reset active transient storage, then reuse existing capacity where possible
Evidence counters	Focused growth/resource probes; shared #94 artifact absent
Qualification	Historical resource win; frame-speed effect near neutral/mixed
Reviewed source	322986f4c5502b3fa887a9605ac8cd88e9b2c579, 2026-09-14
Primary history	PR #8 archival provenance; later current-main reset-offset correction

Summary

The Vulkan renderer stages inline vertices, indices, uniforms, textures, and other transient data in host-visible/device buffer storage. A fixed worst-case allocation avoids growth but can reserve enormous amounts of RAM/VRAM that most workloads never use.

Current main creates bounded initial buffers and grows them only when a request cannot fit. Paired staging/device buffers are resized together outside active command buffers. The control determines whether a batch that exceeded current remaining space may grow to keep the larger accumulated batch or should first drain/reset and size from the post-reset request.

The primary goal is resource efficiency and fewer later batch drains—not guaranteed frame-time improvement.

Why It May Help

The mechanism addresses two costs:

1. Unconditional capacity: Historical code requested roughly 2.5 GiB for transient vertex storage at startup.
2. Repeated drains at a stable larger workload: Once a real batch requires more than the initial size, growing can retain enough capacity for later batches rather than submitting/resetting at the same point repeatedly.

Potential benefit:

lower routine allocation pressure
+ fewer future capacity-driven finishes after a high-water mark

Potential cost:

allocation + unmap/destroy/create/remap + active-work finish at new peaks

The best policy depends on workload demand, allocation pressure, and whether the high-water mark recurs.

Observed in xemu

PR #8 documented a historical reduction from a roughly 2.5-GiB fixed request to an initial 8-MiB staging plus 8-MiB device pair with demand-driven growth. In one PGR2 campaign, working set fell from about 2.367 GB to 1.033 GB, private bytes from 7.459 GB to 4.791 GB, and reported device memory from 3,742 MiB to 2,470 MiB. Guest cadence remained near neutral/slightly favorable in the source-scoped cells.

Current main contains equivalent core mechanics plus a later correctness fix: after a capacity-driven finish resets the staging offset, required capacity must be recomputed for the request at offset zero rather than the stale end of the previous batch.

External Precedent

General allocation precedent: Vulkan Memory Allocator supports persistently mapped host-visible allocations and centralizes map/unmap/lifetime rules: VMA memory mapping.

General reuse precedent: Khronos recommends recycling command/buffer management structures rather than repeatedly allocating/freeing them on hot paths, while noting the tradeoff between CPU overhead and retained memory: Command buffer usage.

xemu's paired transient buffers and command-buffer ownership are project-specific; the external guidance supports measuring reuse/growth rather than reserving unbounded worst-case storage.

Emulation Boundary

Xbox draw/state data
    ↓
xemu transient host representation
staging buffers, device buffers, offsets, command-buffer lifetime
    ↓
host allocation policy
small initial capacity + safe growth OR drain/reuse
    ↓
required result
same uploaded vertex/index/uniform data and draw ordering

Transient Vulkan allocation size is not guest-visible. Data contents, offsets bound to the draw, and ordering are guest-visible through the rendered result.

Code and Ownership Map

Location	Role	Owner
hw/xbox/nv2a/pgraph/vk/buffer.c	Initial sizes, capacity checks, paired resize, mapping, append offsets	Renderer/PFIFO thread
vk/draw.c	Handles append failure, finish/reset, policy-specific required-size recomputation	Renderer thread
vk/renderer.h	Storage-buffer objects, offsets, mappings, usage	Renderer
pgraph_vk_finish()	Ends active ownership before replacement	Renderer
ui/xemu-tweaks.[ch]	Restart-latched growth policy	Startup/UI

Actual Current Flow

flowchart TD
    A[Append transient data request] --> B[Compute aligned end at current offset]
    B --> C{Fits current paired buffers?}
    C -->|Yes| D[Copy into mapped staging and advance offset]
    C -->|No| E[Finish active command buffer and reset offsets]
    E --> F{Growth control enabled?}
    F -->|Yes| G[Size pair for request / retained batch policy]
    F -->|No| H[Recompute requirement at reset offset]
    G --> I[Ensure both staging and device buffers meet requirement]
    H --> I
    I --> J[Unmap mapped old allocation]
    J --> K[Destroy/replace undersized buffers outside active command buffers]
    K --> L[Remap persistent staging side]
    L --> M[Retry append]
    M --> D

Loading

General scratch buffers can use power-of-two growth; the paired transient path keeps both sides at a common sufficient size. The exact route depends on the buffer class.

Maintained Reference Flow

flowchart TD
    A[Request exceeds remaining capacity] --> B[Submit/drain current batch]
    B --> C[Reset offset to zero]
    C --> D[Recompute request size at reset offset]
    D --> E{Single request exceeds whole buffer?}
    E -->|No| F[Reuse current allocation]
    E -->|Yes| G[Grow enough for required single request]
    F --> H[Append and record new batch]
    G --> H

Loading

Control Off does not forbid all growth: a single request larger than total storage still needs safe capacity. It avoids retaining extra capacity solely to fit the prior accumulated batch.

Eligibility and Protected Invariants

Invariant	Why required	Failure if weakened
No active primary/aux command buffer during replacement	Recorded commands reference old handles	GPU use-after-free
Paired staging/device capacities remain compatible	Copy source/destination ranges must both fit	Overrun or invalid copy
Mapped pointers cleared before destroy	Host pointer cannot outlive allocation	Use-after-free
Persistently mapped classes remapped after resize	Later append expects a valid pointer	Crash/corruption
Offset reset on replacement/finish	New allocation starts empty	Wrong destination range
Alignment included in requirement	Vulkan binding/copy limits	Misaligned or underestimated capacity
Overflow checked	Size arithmetic is host security/correctness boundary	Wrap and undersized allocation
Requirement recomputed after finish	Finish changes offset and command ownership	Overgrowth or wrong policy decision
Failure remains explicit	Allocation failure cannot silently continue	Memory corruption/device loss

State, Lifetime, Invalidation, and Fallback

Each StorageBuffer owns a Vulkan buffer, VMA allocation, size, current append offset, usage, and optional mapped pointer. Its lifetime extends until renderer teardown or a safe resize outside command-buffer ownership.

The setting is restart-latched so the renderer uses one stable policy for its allocation lifecycle. Already allocated capacities persist until replacement or teardown; changing the saved value while running must not pretend the active buffers changed policy.

If a request does not fit, xemu must first end any submission that references the allocation. It can then reuse after offset reset or replace the pair. There is no safe “continue anyway” fallback.

Expected Profile Signature

A resource-limited workload should show:

much lower initial committed capacity
    + growth only at new high-water marks
    + stable paired capacities/mappings
    + lower RAM/VRAM

A recurring high-demand workload may additionally show fewer capacity-driven finishes after growth.

Potential adverse signature:

• many growth events with little reuse;
• p99/max spikes at allocation peaks;
• larger retained memory after a one-off burst;
• device allocation failures or paging;
• no end-to-end gain despite fewer finishes.

Do not treat lower allocated bytes as an FPS result. Report resource and latency effects separately.

Counters and Evidence

Counter	Unit	Meaning
vk.buffer.initial_capacity.<class>	bytes	Startup request per buffer
vk.buffer.growth_events.<class>	events	Replacements caused by demand
vk.buffer.old/new_capacity	bytes	High-water progression
vk.buffer.capacity_finishes	finishes	Active batches drained for space
vk.buffer.append_bytes.<class>	bytes	Workload demand
vk.buffer.peak_offset.<class>	bytes	Observed batch high-water mark
vk.buffer.remaps	events	Persistently mapped allocation replacements
allocation CPU/wall time	ns/us	Tail cost of growth
process working/private/device memory	bytes	Primary resource outcome

Evidence must identify the buffer class; combining all buffers hides whether vertex, uniform, texture, or scratch demand caused the event.

Risks and Known Bugs

• Replacing a buffer still referenced by a command buffer is a GPU lifetime defect.
• Staging/device pair mismatch can pass local host checks and fail during copy/draw.
• Stale mapped pointers can survive a VMA allocation replacement.
• Using the stale pre-finish offset overestimates the next required allocation; current main includes a correction.
• Exact-to-demand growth can churn if demand rises in small steps; geometric growth retains more memory but reduces allocations.
• Retaining a one-off high-water mark may increase long-term memory pressure.
• Smaller allocations can shift bottlenecks to more frequent finishes.
• Historical PR #8 numbers are source-scoped and primarily a resource result.

See PR #8, PR #37, and Pull Request Lessons.

Qualification

1. Initial capacity/mapping state for every buffer class.
2. Exact-fit, one-byte-over, alignment-boundary, and overflow tests.
3. Paired staging/device capacity equality and usage flags.
4. Active primary and auxiliary command-buffer replacement prohibition.
5. Finish/reset required-size recomputation regression test.
6. Repeated growth sequence and mapping validity after every replacement.
7. Allocation failure/fault injection with clean error path.
8. Large single draw while control Off (must still grow safely).
9. Representative recurring high-water workload versus one-off burst.
10. RAM, private bytes, VRAM/device allocation, growth count/time, p95/p99/max.
11. Vulkan validation, XISO output hashes, PGR2/Morrowind, teardown/restart.

Default policy should balance retained memory and growth tails; the accepted claim may be resource efficiency even when frame speed is neutral.