Hot Path Vulkan Texture Image Sampler Separation

Vulkan Texture Image and Sampler Separation

Lifecycle: Performance - Unproven · Type: Cache/ownership redesign
State: Source-backed; runtime value is not established.

Status: Source-backed optimization candidate; not implemented and not benchmarked.

Reviewed source: 322986f4c5502b3fa887a9605ac8cd88e9b2c579

Scope: Vulkan texture cache identity, sampler reuse, image ownership and upload avoidance.

This candidate separates two classes of state that currently change at different rates:

• the pixels and host image representation;
• the instructions used to sample those pixels.

The existing clean-stage shortcut can reuse the current complete binding when nothing relevant changed. This proposal is different: it asks whether multiple bindings with different sampling behavior can share one uploaded image and whether identical normalized sampler configurations can share one VkSampler.

Why this is applicable to current xemu

In hw/xbox/nv2a/pgraph/vk/texture.c, the texture-cache key contains the image shape and memory identity and sampler-related state:

// FIXME: Separate sampler from texture
key.filter = filter;
key.address = address;
key.border_color = border_color_pack32;
key.max_anisotropy = max_anisotropy;

A cache miss can therefore allocate/upload another host image even when guest pixels are unchanged but filtering or address mode changed. Sampler creation is attached to the same cache entry.

This candidate was not covered by Clean Texture Stages. That page explains how an already-valid complete stage binding may be skipped during a slow bind. It does not change the cache identity or allow one image to be paired with multiple samplers.

Why it may help

The potential avoided work is:

• duplicate texture-cache entries for identical pixels;
• repeat texture decoding or upload;
• repeat image allocation and view creation;
• duplicate sampler creation;
• avoidable cache pressure and eviction;
• larger retained VRAM and host allocation footprints.

The opportunity should be largest in scenes that reuse the same guest image while changing filtering, wrapping, anisotropy or border behavior.

A useful first-order model is:

avoidable image work
    = sampler-only state changes
    × current combined-key miss rate
    × image creation/upload cost

Current processing flow

flowchart TD
    A[Guest texture state and memory] --> B[Build one combined TextureKey]
    B --> C[Shape and source address]
    B --> D[Filter, address mode, border color, anisotropy]
    C --> E[Combined texture-cache lookup]
    D --> E
    E -->|Hit| F[Reuse image, view and sampler entry]
    E -->|Miss| G[Choose host representation]
    G --> H[Decode or retain compressed data]
    H --> I[Allocate and upload VkImage]
    I --> J[Create image view and VkSampler]
    J --> K[Publish complete cache entry]

Loading

A sampling-only difference participates in the same miss decision as a pixel or representation difference.

Proposed two-stage design

Stage 1: normalized sampler cache

Keep the current image cache intact, but create/reuse samplers from a normalized effective sampler key.

flowchart TD
    A[Guest sampling registers] --> B[Apply host capability and fallback rules]
    B --> C[Normalized effective sampler key]
    C --> D{Sampler cache hit?}
    D -->|Yes| E[Reference existing VkSampler]
    D -->|No| F[Create VkSampler]
    F --> G[Publish sampler entry]
    G --> E

Loading

The key must include extension payload values such as a custom border color, not temporary pNext pointer addresses.

Stage 2: independent image resources

flowchart TD
    A[Guest texture source and shape] --> B[Select effective host image representation]
    B --> C[Image-content and representation key]
    C --> D{Image resource hit?}
    D -->|Yes| E[Reference existing image and view]
    D -->|No| F[Decode or retain source data]
    F --> G[Allocate and upload image]
    G --> H[Publish image resource]
    H --> E
    I[Normalized sampling state] --> J[Sampler cache]
    E --> K[Binding references image resource]
    J --> K

Loading

Multiple bindings may then pair one image/view resource with different sampler resources.

Important representation boundary

Filtering is not always sampler-only in this fork. It can affect whether native block-compressed upload is eligible because the host format must support required filtering features.

Therefore the safe split is not “remove filter from the old key.” The renderer must first resolve the effective host representation. Anything that changes the selected image format remains part of image identity. Only state that changes sampling of an otherwise identical representation belongs exclusively to sampler identity.

Ownership and lifetime

Object	Suggested owner	Lifetime rule
Image allocation and view	Image-resource cache	Retain while referenced by bindings or in-flight commands
Sampler	Sampler cache	Retain while referenced by descriptors/in-flight commands
Binding	Texture-stage/cache entry	References image and sampler independently
Guest-content validation	Image resource	Invalidates/reuploads image data, not unrelated samplers

Eviction must respect Vulkan object lifetime and command-buffer use. Separating objects without independent reference/in-flight ownership would create use-after-free risk.

Seen in the wild

Direct API precedent: Vulkan deliberately represents sampled images and samplers as separate objects. Applications commonly cache immutable sampler descriptions separately from image resources and combine them through descriptors.

That precedent supports the ownership split, but it does not prove xemu's guest-state normalization or invalidation rules. NV2A texture representation, palette dependencies, surface aliases and native-BC eligibility remain xemu-specific.

Evidence required

Add counters before changing ownership:

texture_bind_requests
combined_key_hits / misses
image_resource_hits / misses
image_creations
image_upload_bytes
sampler_requests
sampler_cache_hits / misses
sampler_creations
sampler_only_state_changes
sampler_only_changes_that_change_representation
image_cache_entries / sampler_cache_entries

The decisive focused test alternates filter/address/border state while holding guest pixels, dimensions and source address constant.

Expected successful signature:

sampling state changes
    -> sampler request changes
    -> image resource remains a hit
    -> image_creations and uploaded bytes do not increase
    -> output remains equivalent

Risks and known failure surfaces

• Treating representation-affecting filter capability as sampler-only.
• Omitting palette identity or palette dirtiness from image validity.
• Sharing an image across incompatible surface-source/scaling states.
• Incorrect custom-border-color keying.
• Destroying image/sampler objects while descriptors or command buffers still reference them.
• Increasing lookup/refcount overhead enough to offset savings in low-churn workloads.
• Cache growth from separating one bounded cache into two unbounded caches.

Qualification

1. Unit-test normalized sampler-key equality and inequality.
2. Unit-test representation selection before image-key construction.
3. Add a sampler-only state-churn XISO oracle.
4. Verify image creation/upload counts fall while output hashes remain stable.
5. Exercise texture/palette dirtying, DMA remap, surface aliases and native/fallback BC formats.
6. Exercise cache eviction, command-buffer rollover, renderer teardown and snapshot restore.
7. Run Vulkan validation and representative texture-heavy titles.
8. Compare frame tails, CPU time, VRAM and cache occupancy.

A small sampler-cache experiment should be evaluated separately from the larger image-ownership split.

Disposition

Applicable and the highest-value additional graphics candidate from the audit. Implement sampler-object sharing first, then evaluate independent image ownership only after representation and lifetime counters prove the larger split is worthwhile.

Related pages

• Hot-Path Shortcuts
• Using Hot-Path Shortcuts
• Clean Texture Stages
• Vulkan Renderer
• Memory & DMA

Full technical reference

Primary reference: This subject page is the design and research reference for Texture Image / Sampler Separation. Pull requests are implementation/test history only.
Lifecycle: Performance - Unproven · Mechanism: Cache and ownership redesign
Reviewed source: 322986f4c5502b3fa887a9605ac8cd88e9b2c579 · Acceptance class: intended exact or behavior-preserving for its admitted scope

1. Scope

Separate texture pixel representation from sampling state so filter/address/border changes do not duplicate image allocations and uploads when the underlying host image can be shared.

This entry remains Unproven: the source path and opportunity are concrete, but no accepted xemu runtime result has established that the candidate improves representative workloads. The page defines what must be bypassed, what remains mandatory, and how the hypothesis can be rejected.

2. Source seams

File / function	Role
hw/xbox/nv2a/pgraph/vk/texture.c	create_texture(), combined cache key, sampler creation and FIXME

3. Current execution path

flowchart TD
    N0["Build combined image+sampler key"]
    N1["Lookup texture entry"]
    N0 --> N1
    N2["Sampler-only difference causes miss"]
    N1 --> N2
    N3["Allocate image"]
    N2 --> N3
    N4["Upload/convert pixels"]
    N3 --> N4
    N5["Create sampler"]
    N4 --> N5
    N6["Bind combined entry"]
    N5 --> N6

Loading

4. Candidate execution path

flowchart TD
    N0["Select effective host image representation"]
    N1["Lookup/create image by content/representation key"]
    N0 --> N1
    N2["Normalize sampler state"]
    N1 --> N2
    N3["Lookup/create sampler by sampling key"]
    N2 --> N3
    N4["Bind image view + sampler"]
    N3 --> N4

Loading

5. Why it may help

Pixels and sampling instructions change at different rates. The source explicitly notes that sampler and texture should be separated. The complication is that filtering can affect whether a native compressed representation is legal, so representation selection must happen before image identity is declared independent.

6. Work removed

• Duplicate image entries caused only by sampling-state changes
• Duplicate uploads/allocations
• Duplicate sampler objects after normalization

7. Work that remains mandatory

• Content validation
• Image representation selection
• Sampler capability/fallback handling
• Binding update
• Lifetime tracking for both objects

8. Implementation sketch

ImageKey image_key = select_image_representation(pg, guest_texture);
ImageResource *image = image_cache_get_or_create(&image_key);

SamplerKey sampler_key = normalize_sampler_state(pg, guest_sampler,
                                                 &image_key);
VkSampler sampler = sampler_cache_get_or_create(&sampler_key);

bind_texture_stage(stage, image->view, sampler);

The sketch defines the intended boundary; it is not a complete patch and does not replace source review, ownership design, or focused tests.

9. Correctness contract

• Image identity covers every input that changes uploaded representation: format, dimensions, mip layout, palette/source, scale, compression decision, and content version.
• Sampler identity uses effective host behavior, including capability fallbacks and extension data values rather than temporary pointer addresses.
• Image/view/sampler lifetimes must extend through in-flight draws.

10. State, ownership, invalidation, and fallback

Start with sampler-object sharing while retaining the existing image cache. The second stage introduces independent image ownership/reference counting. Reset, eviction, renderer recreation, capability changes, and in-flight use govern destruction.

The maintained reference path remains the fallback whenever admission, capability, lifetime, or validation is uncertain. A fast result with an unproven dependency is not acceptable as an exact optimization.

11. Advantages

• Can eliminate large duplicate image work
• Normalizes and reuses equivalent samplers
• Improves cache occupancy
• Separates lifetimes conceptually

12. Costs and failure modes

• Major ownership/refcount change
• Incomplete image key causes representation mismatch
• Filtering/compression interaction complicates sharing
• More cache lookups and objects to manage

13. Evidence counters

Counters must be cheap, subsystem-owned, and snapshotted outside the hot loop. Record at least:

• texture_requests
• image_cache_hits/misses
• sampler_cache_hits/misses
• image_creations
• sampler_creations
• uploaded_bytes
• entries_per_guest_image
• frame p95/p99

A local reduction is not enough. Evidence must connect route reachability and removed work to fixed guest progress, CPU/GPU time where applicable, and p95/p99 behavior.

14. Qualification matrix

• Hold pixels constant while alternating filter/address/border
• Native compressed vs decompressed representation
• Palette and mip changes
• Capability fallback normalization
• Eviction with several samplers per image
• In-flight destruction
• Reset/snapshot/recreation
• Compare output and upload counts

Correctness runs may use validation layers, detailed comparisons, and fault injection. Performance acceptance must use optimized Release builds with matching symbols and minimal instrumentation.

15. Acceptance rule

Promote sampler sharing first when normalized keys are correct and creation falls. Promote image separation only after representation/lifetime tests and meaningful duplicate upload evidence.

16. Rejection rule

Reject or stage more narrowly if sampler churn does not create image misses, representation choice cannot be separated safely, or ownership complexity outweighs measured savings.

17. Related architecture and history

Vulkan-Renderer · Performance-Unproven · Hot-Path-Vulkan-Clean-Texture-Stages · Hot-Path-Vulkan-Direct-Surface-Sampling

PR pages may be linked as supporting evidence, but they are not substitutes for this reference.