OpenGL Cold Shader Stall Avoidance

OpenGL Cold Shader Stall Avoidance

Lifecycle: Performance - Unproven · Type: Asynchronous compilation and alternate execution research
Reviewed source: 322986f4c5502b3fa887a9605ac8cd88e9b2c579
State: Synchronous compile/link/validate behavior is source-confirmed. The backend already has module/program caches and disk program binaries, so this page targets genuinely first-seen or rejected-cache states only.

OpenGL candidate directory · OpenGL Renderer · Vulkan Hybrid Ubershader

Purpose

The OpenGL backend caches generated shader modules, linked programs, and program binaries. Warm known states are therefore not the primary target.

For a genuinely first-seen state, create_gl_shader() calls glCompileShader() and immediately checks GL_COMPILE_STATUS. generate_shaders() links, immediately checks GL_LINK_STATUS, uses the program, validates it, and retrieves uniform locations. Those checks can force the application thread to observe completion.

The candidate investigates ways to avoid blocking guest progress on first-seen compilation:

1. Use GL_KHR_parallel_shader_compile where supported, allowing the driver to compile/link asynchronously and polling completion at bounded points.
2. Build an OpenGL-specific alternate fragment route, such as an interpreter/ubershader, while exact specialization completes.
3. Improve startup/admission scheduling for known likely states without duplicating the existing program-binary cache.

These are separate subexperiments and must not be merged into one performance claim.

Source seams

• hw/xbox/nv2a/pgraph/gl/shaders.c
  • create_gl_shader()
  • generate_shaders()
  • shader-module cache
  • linked-program cache
  • program-binary disk cache and validation
  • uniform-location initialization
• hw/xbox/nv2a/pgraph/glsl/psh.c
  • shared fragment generator
  • current assertion that ubershader mode requires Vulkan
• OpenGL program/pipeline selection at draw time.

Current execution flow

flowchart TD
    A[Draw needs shader state] --> B{Program cache hit?}
    B -->|Yes| C[Use cached linked program]
    B -->|No| D{Program binary available and accepted?}
    D -->|Yes| E[Load binary and validate program]
    D -->|No| F[Generate/lookup shader modules]
    F --> G[glCompileShader for first-seen modules]
    G --> H[Check compile status]
    H --> I[glLinkProgram]
    I --> J[Check link status]
    J --> K[glValidateProgram]
    K --> L[Query uniform locations]
    E --> C
    L --> C
    C --> M[Draw]

Loading

The cache already solves many warm states. The unproven opportunity is the cold miss path.

Candidate execution flow

flowchart TD
    A[Cold program miss] --> B{Parallel compilation supported and enabled?}
    B -->|Yes| C[Start compile/link and publish pending record]
    B -->|No| D{Exact alternate route admitted?}
    C --> E{Exact alternate route admitted while pending?}
    E -->|Yes| F[Draw through general fallback]
    E -->|No| G[Use synchronous reference route for this draw]
    D -->|Yes| F
    D -->|No| G
    C --> H[Poll completion at bounded processing points]
    H --> I{Specialized program complete and valid?}
    I -->|Yes| J[Publish complete program and promote future draws]
    I -->|No| K[Retain pending/fallback or report failure]
    L[Scheduled warmup] --> C

Loading

The three routes below must be benchmarked independently: parallel driver work, alternate execution, and predicted warmup can overlap, but they remove different costs and carry different correctness risks.

Candidate A: parallel compiler completion

flowchart TD
    A[Cold shader/program request] --> B[Start compile/link with parallel extension]
    B --> C[Store pending program record]
    C --> D{Completion status ready?}
    D -->|No| E[Use admitted fallback or pause only this draw]
    D -->|Yes| F[Check compile/link result]
    F -->|Success| G[Initialize locations and publish program]
    F -->|Failure| H[Record failure and use reference error handling]
    G --> I[Exact program used by later draws]

Loading

Illustrative completion check:

GLint complete = GL_FALSE;
glGetProgramiv(program, GL_COMPLETION_STATUS_KHR, &complete);
if (!complete) {
    return PROGRAM_PENDING;
}

GLint linked = GL_FALSE;
glGetProgramiv(program, GL_LINK_STATUS, &linked);

The initial compile/link calls may still have CPU cost. The extension changes when the driver must complete the work and allows several jobs to progress concurrently.

Candidate B: OpenGL alternate fragment route

The shared generator currently permits ubershader output only when Vulkan options are selected. Reusing the concept requires an OpenGL integration design, not removal of the assertion.

flowchart TD
    A[Cold specialized state] --> B{State admitted by OpenGL fallback?}
    B -->|No| C[Current synchronous specialized route]
    B -->|Yes| D[Use precompiled OpenGL general fragment program]
    D --> E[Upload compact combiner/control state]
    E --> F[Draw continues]
    D --> G[Queue exact specialization]
    G --> H[Compile/link completes]
    H --> I[Build complete compatible program state]
    I --> J[Promote future draws to specialization]

Loading

The fallback must implement the exact guest combiner/texture/depth semantics for admitted states. A shader module alone is not sufficient; program link, locations/layout, and route compatibility must be complete before promotion.

Candidate C: scheduled warmup

A program-binary cache already exists. A useful warmup experiment must target:

• binaries known from the current title/profile;
• likely module/program states not already loaded;
• low-priority compile scheduling outside the critical draw;
• bounded memory/cache capacity;
• exact executable/driver/cache identity.

Blindly compiling every historical state can make startup worse and consume driver resources.

Route classification

Route	Correctness	Main purpose
Existing process/program cache	Exact	Warm known state
Existing disk program binary	Exact if accepted/validated	Cross-run warm state
Parallel compile	Exact final program	Move/overlap cold compilation
OpenGL general fallback	Must be proven for admitted states	Avoid first-seen draw stall
Scheduled warmup	Exact final program	Prepare likely states before first draw

What may disappear

• Application-thread blocking on immediate compile/link status checks.
• First-seen draw stalls when a correct fallback is available.
• Serial cold compilation when the driver can process jobs in parallel.
• Some startup/hitch cost for accurately predicted warmup states.

What remains mandatory

• Exact shader semantics for specialized and fallback routes.
• Complete program/link/uniform state before publication.
• Correct error reporting for failed compilation/link.
• Program and module cache ownership.
• Driver/vendor/version identity for binary reuse.
• Bounded worker/pending-job count.
• Teardown/reset/context-recreation cancellation.
• Synchronous reference route for unsupported extensions or states.

Why another cache is not the first answer

The backend already has:

• process module cache;
• linked program cache;
• program binary disk cache;
• cache reload list and validation.

A new cache does not solve genuinely first-seen source or a program binary rejected by the current driver. Evidence must identify which layer missed before selecting a solution.

OpenGL fallback design constraints

An interpreter/ubershader may need:

• combiner stage inputs/outputs;
• final combiner controls;
• texture-mode control data;
• alpha test/discard;
• fog and depth behavior;
• signed/border/shadow texture semantics;
• uniform/control streaming;
• a stable program and sampler layout;
• exact route-selection and promotion rules.

The current Vulkan controls/layout cannot simply be copied because OpenGL program and uniform ownership differ.

Advantages

• Directly targets first-seen hitches rather than warm cache cost.
• Parallel compile can be capability selected with exact final output.
• Alternate execution can preserve guest progress during specialization.
• Existing caches and shared generator provide reusable components.
• Route counters can distinguish source/module/link/binary misses.

Costs and risks

• Parallel extension support and behavior vary by driver.
• Polling completion too often can create new driver overhead.
• An OpenGL ubershader is a major correctness project.
• General fallback may be slower during steady state or increase resource use.
• Promotion while a draw/program is in use can create lifetime races.
• Warmup prediction can increase startup time and memory.
• Background compilation and context ownership can complicate teardown.

Required counters

shader_binding_requests
program_cache_hits
program_binary_hits
program_binary_rejections
module_cache_hits
module_compiles
program_links
compile_status_wait_us
link_status_wait_us
parallel_jobs_submitted
parallel_jobs_pending_peak
parallel_jobs_completed
fallback_draws
specialized_promotions
warmup_programs_requested
warmup_programs_used_before_eviction
cold_draw_p95/p99

Qualification matrix

Scenario	Required result
Warm process cache hit	No new mechanism invoked
Valid program binary hit	Existing route unchanged
Empty cache, first-seen state	Candidate route counted and hitch measured
Driver rejects binary	Normal compile/fallback succeeds
Parallel extension unsupported	Synchronous reference route
Multiple cold states	Bounded jobs and no starvation
Fallback-admitted state	Pixel/hash parity with specialization
Fallback-ineligible state	Reference synchronous route
Promotion after compile	No wrong program/layout or lifetime race
Reset/shutdown/context loss	Jobs stopped and objects released safely
Cold/warm Release runs	Tail improvements reported separately

Acceptance rule

Parallel compile may move to Researched when exact output is preserved and first-seen tail latency improves on supporting drivers. An OpenGL alternate route requires a much higher bar: semantic oracle coverage, lifecycle safety, and broad title qualification similar to the Vulkan hybrid system.

Rejection rule

Reject a mechanism if traces show cold compilation is not a material bottleneck, if completion polling simply moves the stall, if driver support is too narrow, or if fallback correctness cannot be bounded.

Related

• OpenGL Performance Candidates
• OpenGL Renderer
• Shader Binding & Compilation
• Hybrid Ubershader
• PR #68 Shader Miss Classification
• Performance Methodology