Hot Path Vulkan Precompiled Internal Shaders

Precompiled Vulkan Internal Shaders

Lifecycle: Performance - Unproven · Type: Build-time preparation
State: Source-backed; runtime value is not established.

Status: Source-backed startup/recreation candidate; not implemented as a build-time artifact path and not benchmarked.

Reviewed source: 322986f4c5502b3fa887a9605ac8cd88e9b2c579

Scope: Fixed renderer-owned GLSL compiled during Vulkan initialization, especially clear shaders.

xemu owns the complete source of several internal Vulkan shaders. Unlike guest-derived shaders, their source does not depend on an Xbox title's runtime shader state. Current initialization still sends fixed GLSL through the runtime compiler before creating Vulkan shader modules.

This candidate compiles and validates target-specific SPIR-V during the build and embeds it for normal runtime use.

Why this is applicable

vk/draw.c contains fixed internal vertex and fragment GLSL strings used by clear operations. init_clear_shaders() calls the normal source-compilation helper to construct their modules.

The existing Shader Caching page addresses exact generated sources learned at runtime and reused across launches. This proposal is different:

• the source is known before xemu is built;
• no first-run compiler miss is necessary;
• artifacts can be validated by the build and packaged with the executable;
• likely benefit is initialization or renderer recreation, not sustained gameplay.

Why it may help

Potentially avoided runtime work includes:

• GLSL preprocessing/parsing/linking;
• glslang setup and SPIR-V generation;
• source hashing/cache lookup for fixed shaders;
• startup variance and compiler allocation;
• repeated compilation after renderer reinitialization.

The total number of fixed shaders is small, so this should not be sold as a frame-rate optimization without evidence. Its value is predictable startup/recreation work and a smaller runtime compiler surface.

Current flow

flowchart TD
    A[Vulkan renderer initialization] --> B[Fixed internal GLSL string]
    B --> C[Runtime glslang compilation]
    C --> D[Generated SPIR-V]
    D --> E[Create VkShaderModule]
    E --> F[Create clear/internal pipeline as needed]

Loading

Candidate flow

flowchart TD
    A[Build system] --> B[Fixed internal GLSL]
    B --> C[Compile for each supported target/policy]
    C --> D[Validate SPIR-V and embed artifact]
    E[Vulkan renderer initialization] --> F[Resolve active Vulkan/SPIR-V target and debug policy]
    F --> G{Compatible embedded artifact available?}
    G -->|Yes| H[Create VkShaderModule from embedded SPIR-V]
    G -->|No or developer override| I[Use maintained runtime source compiler]
    H --> J[Create internal pipeline]
    I --> J

Loading

Target and policy identity

One embedded binary is not automatically valid for every current compiler route. Identity must include all build/runtime inputs that can alter accepted SPIR-V or reflection behavior, such as:

• shader stage and entry point;
• Vulkan API/SPIR-V target pair;
• generator ABI;
• compiler validation/profile/message policy;
• debug/optimizer/debug-info options;
• descriptor/push-constant layout contract;
• source revision/hash.

The active renderer chooses a compatible artifact. Unsupported or development/debug configurations retain a source-compile fallback.

Build integration contract

The build must:

1. compile fixed sources with pinned compiler arguments;
2. validate outputs;
3. generate deterministic C/byte-array or packaged resource artifacts;
4. record source and compiler-policy identity;
5. make stale generated outputs fail the build rather than silently persist;
6. support all targets actually shipped by the repository.

Do not commit an opaque binary without a reproducible generation rule and source linkage.

Emulation boundary

These shaders are host implementation details; they do not exist as guest-provided Xbox shader programs. The guest-visible contract is the same clear/internal rendering result and ordering.

Equivalent behavior requires the embedded artifact to use the same shader semantics, descriptor layout, push constants, render-pass compatibility and target assumptions as the runtime-compiled source.

Seen in the wild

Direct Vulkan/build-system precedent: Vulkan consumes SPIR-V modules, and applications commonly compile fixed GLSL/HLSL to SPIR-V as a build step, embedding or packaging the result. Runtime compilation is usually reserved for development, user-supplied code or dynamically generated shaders.

xemu-specific distinction: guest-derived NV2A shaders remain dynamic and are handled by module/persistent caches and the hybrid route. Precompiling a few renderer-owned shaders does not replace those systems.

Code and ownership map

Area	Role
vk/draw.c fixed GLSL strings and init_clear_shaders()	Initial candidate source and module creation
vk/glsl.c compiler helpers	Maintained fallback and policy reference
Renderer initialization	Selects target-compatible artifact
Build files/generated source	Reproducibly create embedded SPIR-V
Shader-module teardown	Same runtime ownership after module creation

Evidence and counters

internal_shader_modules_created
internal_runtime_compiles
internal_embedded_artifact_hits
internal_embedded_artifact_fallbacks
internal_shader_compile_cpu_ns
renderer_init_cpu_ns
renderer_recreation_cpu_ns

Expected signature:

normal release startup
    -> embedded artifact hit
    -> zero runtime compile for covered internal shaders
    -> same module/pipeline/output

Risks

• Artifact compiled for the wrong Vulkan/SPIR-V target.
• Generated file not refreshed after source or compiler-policy change.
• Debug source/line behavior lost in developer builds.
• Embedded reflection/layout assumptions diverge from runtime path.
• Build becomes dependent on an unavailable or unpinned compiler.
• Binary size/build complexity outweighs tiny startup savings.
• Claiming gameplay improvement when only initialization changed.

Qualification

1. Enumerate fixed internal shaders and prove which are source-invariant.
2. Add deterministic generation and stale-output checks.
3. Compare embedded SPIR-V with runtime output for every supported target/policy where deterministic equality is expected.
4. Route embedded and source-compiled artifacts through the same module/reflection/layout validation.
5. Compare clear/internal output hashes and Vulkan validation.
6. Exercise normal startup, renderer creation failure/fallback, teardown and recreation.
7. Retain a developer/runtime-source path and explicitly test it.
8. Measure startup/recreation CPU and wall time; do not use frame FPS as the primary metric.
9. Keep guest-derived shader caching/hybrid qualification separate.

Disposition

Applicable, low-risk if build reproducibility is strong, but likely modest value. Add after higher-frequency candidates unless renderer recreation/startup profiles show meaningful compiler cost.

Related pages

• Hot-Path Shortcuts
• Shader Caching
• Hybrid Ubershader
• Vulkan Renderer

Full technical reference

Primary reference: This subject page is the design and research reference for Precompiled Internal Shaders. Pull requests are implementation/test history only.
Lifecycle: Performance - Unproven · Mechanism: Build-time preparation
Reviewed source: 322986f4c5502b3fa887a9605ac8cd88e9b2c579 · Acceptance class: intended exact or behavior-preserving for its admitted scope

1. Scope

Compile fixed xemu-owned GLSL during the build and embed compatible SPIR-V instead of invoking the runtime compiler at renderer initialization.

This entry remains Unproven: the source path and opportunity are concrete, but no accepted xemu runtime result has established that the candidate improves representative workloads. The page defines what must be bypassed, what remains mandatory, and how the hypothesis can be rejected.

2. Source seams

File / function	Role
hw/xbox/nv2a/pgraph/vk/draw.c	fixed clear/display helper shader strings and init_clear_shaders()
hw/xbox/nv2a/pgraph/vk/glsl.c	runtime GLSL-to-SPIR-V module creation

3. Current execution path

flowchart TD
    N0["Renderer initialization"]
    N1["Known internal GLSL string"]
    N0 --> N1
    N2["glslang/runtime compilation"]
    N1 --> N2
    N3["Create VkShaderModule"]
    N2 --> N3
    N4["Use fixed shader"]
    N3 --> N4

Loading

4. Candidate execution path

flowchart TD
    N0["Build step compiles known GLSL"]
    N1["Validate/embed target-specific SPIR-V"]
    N0 --> N1
    N2["Runtime selects compatible artifact"]
    N1 --> N2
    N3["Create VkShaderModule"]
    N2 --> N3
    N4["Fallback to source compile when policy differs"]
    N3 --> N4

Loading

5. Why it may help

For shaders whose complete source and policy are owned by xemu, runtime compilation adds startup or renderer-recreation work without learning anything from guest state. This is distinct from persistent caching of dynamically generated shaders.

6. Work removed

• Runtime parser/compiler invocation for fixed internal shaders
• Cold-start compiler initialization for those sources

7. Work that remains mandatory

• VkShaderModule creation
• Runtime compilation fallback
• Guest-derived shader compilation
• Pipeline creation
• Policy/target validation

8. Implementation sketch

custom_target(
  'clear-shaders-spv',
  input: ['clear.vert', 'solid.frag'],
  output: ['clear_vert_spv.h', 'solid_frag_spv.h'],
  command: [shader_embed_tool, '@INPUT@', '--target-env', vulkan_target,
            '--output', '@OUTPUT@'])

The sketch defines the intended boundary; it is not a complete patch and does not replace source review, ownership design, or focused tests.

9. Correctness contract

• Embedded artifacts must match Vulkan/SPIR-V target, generator ABI, optimization/debug policy, and expected shader interface.
• Builds must fail on stale generated artifacts.
• Runtime source compilation remains available for unsupported/debug policies.

10. State, ownership, invalidation, and fallback

Artifacts are immutable build outputs. Runtime ownership begins only at VkShaderModule creation. Renderer teardown destroys modules normally; no persistent cache lifetime is involved.

The maintained reference path remains the fallback whenever admission, capability, lifetime, or validation is uncertain. A fast result with an unproven dependency is not acceptable as an exact optimization.

11. Advantages

• Removes deterministic startup compilation
• Build-time failures surface shader errors earlier
• No runtime cache lookup/write needed
• Small, well-defined source set

12. Costs and failure modes

• Build-system/toolchain complexity
• Artifacts multiply across target/debug policies
• Likely no sustained gameplay FPS benefit
• Embedded SPIR-V can become stale if generation checks are weak

13. Evidence counters

Counters must be cheap, subsystem-owned, and snapshotted outside the hot loop. Record at least:

• internal_shader_compile_calls
• internal_shader_compile_us
• renderer_init_us
• renderer_recreate_us
• embedded_artifact_selected
• runtime_fallbacks_by_reason

A local reduction is not enough. Evidence must connect route reachability and removed work to fixed guest progress, CPU/GPU time where applicable, and p95/p99 behavior.

14. Qualification matrix

• Build regeneration/stale-output check
• Validate SPIR-V for every supported target policy
• Compare runtime and embedded module reflection/interfaces
• Debug/nonsemantic-info policy
• Renderer recreation and teardown
• Cold startup timing on clean process

Correctness runs may use validation layers, detailed comparisons, and fault injection. Performance acceptance must use optimized Release builds with matching symbols and minimal instrumentation.

15. Acceptance rule

Promote when artifact identity is robust, all supported configurations have a safe fallback, and startup/recreation cost is measurably reduced or build-time ownership is otherwise desirable.

16. Rejection rule

Reject if policy matrix maintenance outweighs the tiny runtime cost, artifacts frequently go stale, or target-specific behavior cannot be selected safely.

17. Related architecture and history

Vulkan-Shader-Binding-and-Compilation · Performance-Unproven · Hot-Path-Vulkan-Shader-Caching · Vulkan-Renderer

PR pages may be linked as supporting evidence, but they are not substitutes for this reference.