Compiling Vulkan shader code at registration, not at the first draw (VK_EXT_shader_object)

[Raghav314]

Update: the maintainers went with VK_EXT_graphics_pipeline_library instead of shader objects (see adriadam10's reply and the 4090 numbers under it). Phase A landed in #2636; phase B (building the libraries at registration) is next.

Hey @boykopovar, I've been chasing a startup crash in Hades II (PPSA36082) and it led somewhere bigger than one fix, so I'd rather talk it through before writing a huge PR.

The problem

Hades II only boots when the shader caches are warm. On a cold start (first launch, or the first launch after any recompiler change, because the disk cache directory is keyed on a hash of the recompiler sources) it dies 3.2-3.5 s in, every time I've tried.

What happens: the game's loader stages data into a 512 MiB pool and only reuses pool memory once a GPU fence (a RELEASE_MEM on the graphics queue) has landed. On a PS5 that fence comes back within a few ms. Here, the queue worker hits a draw whose pipeline isn't compiled yet and sits in vkCreateGraphicsPipelines for ~65 ms. The fence is on the same queue behind that frame, so it's correctly ordered after the draw and just can't land in time. The loader keeps staging, the pool hits 512 MiB, and the game crashes.

Numbers from my machine (RTX 4090, NVIDIA 615.71, Linux):

• The slow pipeline is one vertex shader (~5.9k SPIR-V words) plus a tiny pixel shader: 64-68 ms cold, every time. Almost all of it is the vertex shader.
• With warm caches the same startup peaks at ~330 MiB in the pool and is fine.
• VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT doesn't help, NVIDIA ignores it (87 ms either way on a test shader).
• I also made the BDA lookup cheaper to compile (one shared lookup body, three fault exits instead of thirteen, same checks and fault reasons). That pipeline drops to 36 ms, which is nice on its own and I can send it as a separate PR, but the game still dies: on a cold start three new pipelines show up back to back (36 + 24 + 36 ms) before the fence gets its turn. Making each compile cheaper can't fix a sum like that.

So as long as the driver compiles shader code when a draw first needs it, a cold start puts tens of ms of host work on the GPU timeline at a moment the game assumes is basically free.

Why I think the fix belongs in the architecture

#639 already says where this should go:

Recompiling them at this stage would allow draw/dispatch to use precompiled SPIR-V without compiling shaders during gameplay … Original shaders read these parameters at runtime. Recompiled shaders must do the same: fixed code, with resources supplied separately.

The RDNA to SPIR-V step already happens at registration. The NVIDIA/AMD compile of that SPIR-V is the one compile left at draw time, and it's the expensive one.

Vulkan has a direct way to do this: VK_EXT_shader_object. Each stage is compiled into a VkShaderEXT on its own, and a draw only binds the stage objects and sets render state as dynamic commands. There's no pipeline object, so nothing compiles at draw. It also maps more closely onto how the PS5 works: precompiled per-stage code, and a draw is register state plus "go".

I checked it with the actual Hades shaders, offline, with the validation layer on (zero messages):

	Pipeline (today)	Shader objects
Compile	~66 ms, at the first draw	~67 ms, at creation (can happen at registration, off the queue)
First draw	+66 ms	0.6 ms
Later draws that change cull mode, blending or vertex format	a new pipeline per combination	~0.1 ms, no hidden recompiles

What would have to change

Here's the catch, and the reason I'm asking first. To build a VkShaderEXT at registration, the stage's code can't depend on anything that's only known at draw time. Today four things are folded in per draw (materializeResult / buildSpecializedModule in Recompiler.cpp):

1. Specialization constants. Descriptor words (vertex and image formats, heap sizes) become OpConstants. In the generic modules I looked at, these are only used as selectors in code that already handles every case (if format == 1 / 4 / 5 …), so they could become values read from a small per-draw parameter block. The one structural use is the bindless heap arrays, whose length is a spec constant (HeapCountBase); those would become runtime-sized arrays.
2. Vertex input types. SpecializeVertexInputTypes rewrites the shader's input variable types (float, int or uint) to match each draw's V# format. A shader object needs fixed types. The faithful option is fetching vertices in the shader from the V#, which is what the hardware does, but it replaces the current Vulkan vertex-input path.
3. Descriptor set layout. The layout is built per draw from the captured descriptors (preparedBindingPlan). A shader object fixes its set layouts at creation, so each stage needs a layout that's stable from registration on.
4. Push data placement. Where a stage's push data sits depends on the draw (pushConstantOffsetBytes is in the module key). It would need a fixed place per stage.

Plus dynamic rendering instead of VkRenderPass for this path, every bit of render state as dynamic commands (extended dynamic state 1-3, vertex input dynamic state), and the same for compute. Devices without VK_EXT_shader_object would keep today's pipeline path as a fallback. NVIDIA ships the extension natively, and RADV has had it enabled by default since Mesa 24.1.

How I'd split it

Every step stands on its own and keeps today's path working:

1. Run-time parameters instead of specialization constants (recompiler), with runtime-sized heap arrays.
2. Stable per-stage descriptor layouts and push data placement.
3. Vertex fetch from the V# in the shader (or another answer to point 2 above, if you have one in mind).
4. Compute stages as shader objects, created at registration on a background thread.
5. Graphics stages as shader objects with dynamic rendering and dynamic state, pipeline path kept as the fallback.

The BDA lookup change is independent and can go in whenever.

Questions

• Does this fit where you want to take perf(agc): baseline shader recompilation on registration #639? Especially points 1-3, since they change how materialization works.
• For vertex input: would you rather fetch in the shader, or keep Vulkan vertex input and accept a few variants per vertex shader (one per numeric type combination)?
• Is it OK to require dynamic rendering and the extended dynamic state extensions for the new path, with pipelines as the fallback?
• Any of this already in progress on your side? I don't want to duplicate it.

Happy to share the measurement harness and the dumped Hades shaders if you want to reproduce the numbers.

Full disclosure: I used an LLM (Claude) to help with this investigation: profiling, writing the offline test harness, reading through the recompiler and drafting this post. All the numbers come from real runs on my machine.

[fisherjoey]

We ran into the same prerequisite from the performance side, in #1612. On Demon's Souls (PPSA01341) most of the per-draw CPU cost is resolving descriptors that the shader could read itself, and moving that resolution onto the GPU needs the same thing your step 1 needs: code that doesn't depend on descriptor contents at draw time. So I think this is one direction with two reasons for it: cold-start correctness here, and per-draw CPU cost there.

Two data points from Demon's Souls on Linux (RTX 5070 Ti, NVIDIA 595.91.07), tutorial level:

• Specialization doesn't buy much in this title. The level's shader cache has 662 variants over 542 distinct code hashes, about 1.2 variants per shader, so dropping descriptor-derived specialization (your step 1) would cost little here.
• What it costs today: on queue 0, the SRT walk is about 71 ms and materialization about 74 ms per frame, out of a 459 ms frame. Both repeat every frame for the same layouts. Fixed per-stage code with resources supplied separately is what removes them.

On the vertex input question I don't have an answer. Our study left vertex fetch on the current fixed-function path and didn't measure either option. I can count how many vertex format combinations each vertex shader actually sees in Demon's Souls, which would show how many variants the "keep vertex input" option needs for this title.

If it helps, I can test steps 1-3 on Demon's Souls on Linux/NVIDIA as they land, with before and after frame numbers.

AI-assisted: yes

[fisherjoey]

Here's the vertex format count I offered. For Demon's Souls the answer is simple: none of its vertex shaders has vertex inputs for SpecializeVertexInputTypes to rewrite.

• Vertex shaders seen: 78 distinct code hashes.
• Draws counted: about 4.7 million draw packets over a 600 s run. About 470 s of that is in the tutorial level (Outpost Passage); the rest is boot and the debug start menu.
• Format combinations per shader: 1 for all 78 (the empty one), 0 with 2, 0 with 3 or more, max 1.
• Shaders with more than one combination: none.

No DS vertex shader declares the AGC vertex attribute and vertex buffer tables (PtrVertexAttribDescTable / PtrVertexBufferTable), so DecodeVertexStageInfo returns no input semantics for any draw. Not one draw had embedded fetch or a fetch shader. The title reads its vertex data with ordinary buffer loads in the shader. On main those shaders get no vertex inputs and no input type specialization, so for this title the vertex input question in your point 2 doesn't come up. Fixed shader object code would work for its vertex stages as they are. Other titles may well differ.

How I counted: a local-only counter in the draw path, keyed by the vertex stage's code hash. For each draw it recorded the per-attribute numeric class (float/sint/uint, using the same mapping as VertexInputNumericClass) and the full V# format of every input semantic. It ran on our working branch (main df16c4c plus a few open PRs and local fixes), because current main doesn't reach a level in this title yet (it stops at registration, #1921). The vertex input decode on that branch is the same function as on main.

Setup: Linux, RTX 5070 Ti, NVIDIA 595.91.07, Demon's Souls PPSA01341 v1.07, booted straight into the tutorial level through the game's debug start menu.

AI-assisted: yes

[adriadam10]

Thanks for the write-up and the numbers. We're taking this on: all five steps, one PR each, with your measurements as the reference. Before choosing the mechanism, I measured it.

Setup. I registered every Shader header in Astro's Playroom's eboot.bin through AgcDriverRegisterShader (the real registration path on current main) and dumped each prepared SPIR-V module. That gave 544 unique graphics modules: 483 VS and 61 PS. I then timed three ways of getting each one ready to draw, on RADV (Radeon 660M, Mesa 26.2.4, all shader caches off):

483 VS	median	p90	max	total
today: vkCreateGraphicsPipelines at the first draw	64.4 ms	297.5 ms	1.39 s	64.3 s
VK_EXT_graphics_pipeline_library: pre-raster library, ahead	28.0	68.1	809 ms	21.4 s
GPL fast link, at the draw	0.01	0.01	0.03	~0
GPL optimized link (background, swapped in later)	2.8	4.2	193 ms	3.1 s
VK_EXT_shader_object: vkCreateShadersEXT, ahead	50.0	154.9	620 ms	36.3 s

PS: today a 13 ms median and a 35.7 s worst case. GPL ahead takes 5.6 ms median and links in 10 µs. Shader objects take 18.7 ms median.

Both approaches take the compile off the draw. On RADV, GPL also costs less ahead of time (the driver defers part of the work to the link), and it keeps VkPipeline/VkRenderPass: the fallback is the same path without the library flags. So the plan is GPL plus extended dynamic state, with today's pipelines as the fallback. Steps 1-3 (runtime parameters instead of spec constants, stable layouts and push placement, vertex fetch from the V# in the shader) are the same for either mechanism, and that's where we start.

Spec constants in those modules, which is what step 1 removes: buffer V# words in 680 of 780 modules, push data offset in 668, vertex formats in 548, image/heap selectors in 179, export mapping in 76, depth compare in 34. Buffers already have a runtime V# path (gpuDescriptor), so most of step 1 is using it everywhere.

What I can't measure here is NVIDIA. @Raghav314, could you run the same benchmark on your 4090? It's one file, with your Hades II modules as input, or any directory of .spv files. It's below (g++ -std=c++20 -O2 bench.cpp -lvulkan, needs glslc; run BENCH_MODE=mono|gpl|so ./bench <dir> once per mode, with the driver's shader cache off). A cold-start Hades run on the PRs as they land would also be the best end-to-end check. @fisherjoey, the same on Demon's Souls would be very welcome, with the vertex-format count you offered.

Your BDA lookup change is independent. Please send it as you planned.

AI-assisted: yes (Claude). All numbers are from real runs; raw TSVs on request.

bench.cpp

// Offline benchmark: cost at first draw of a graphics stage, three ways (RADV/any driver).
//   mono: vkCreateGraphicsPipelines (what AnyPS5 does today, at the first draw)
//   gpl : VK_EXT_graphics_pipeline_library, stage libraries built ahead, fast link at the draw (+ LTO link, background)
//   so  : VK_EXT_shader_object, vkCreateShadersEXT ahead, nothing at the draw
// Input: directory of NNNNN_<stage>_<addr>.spv dumped at registration. Stage under test is paired with a stub.
#include <vulkan/vulkan.h>
#include <algorithm>
#include <chrono>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <filesystem>
#include <fstream>
#include <map>
#include <set>
#include <string>
#include <vector>

namespace fs = std::filesystem;
using Clock = std::chrono::steady_clock;
static double ms(Clock::time_point a, Clock::time_point b) { return std::chrono::duration<double, std::milli>(b - a).count(); }
#define CHECK(x) do { VkResult r_ = (x); if (r_ != VK_SUCCESS) { std::fprintf(stderr, "%s -> %d\n", #x, r_); throw r_; } } while (0)

struct Var { uint32_t location = ~0u, comps = 4, base = 0 /*0 f 1 i 2 u*/; bool flat = false; bool builtin = false; uint32_t arrayLen = 1; };
struct Desc { uint32_t set, binding, count; VkDescriptorType type; };
struct Module {
    std::string name; std::vector<uint32_t> words; uint32_t model = ~0u; std::string entry;
    std::vector<Var> inputs, outputs; std::vector<Desc> descs; bool push = false;
};

static Module Reflect(const std::string& name, std::vector<uint32_t> w) {
    Module m; m.name = name;
    std::map<uint32_t, std::vector<uint32_t>> types, consts;  // id -> instruction words (from opcode)
    std::map<uint32_t, uint32_t> set, binding, location, builtin; std::set<uint32_t> flat, block, bufferBlock;
    std::vector<std::vector<uint32_t>> vars;
    for (size_t c = 5; c < w.size();) {
        uint32_t n = w[c] >> 16, op = w[c] & 0xffff; std::vector<uint32_t> ins(w.begin() + c, w.begin() + c + n);
        if (op == 15 && m.model == ~0u) { m.model = ins[1]; m.entry = reinterpret_cast<const char*>(&ins[3]); }
        else if (op == 71) {
            uint32_t d = ins[2];
            if (d == 34) set[ins[1]] = ins[3]; else if (d == 33) binding[ins[1]] = ins[3]; else if (d == 30) location[ins[1]] = ins[3];
            else if (d == 11) builtin[ins[1]] = ins[3]; else if (d == 14) flat.insert(ins[1]); else if (d == 2) block.insert(ins[1]); else if (d == 3) bufferBlock.insert(ins[1]);
        } else if (op == 72 && ins[3] == 11) builtin[ins[1]] = 1;
        else if (op >= 19 && op <= 39 || op == 5341) types[ins[1]] = ins;
        else if (op == 43 || op == 50) consts[ins[2]] = ins;
        else if (op == 59) vars.push_back(ins);
        c += n;
    }
    auto scalar = [&](uint32_t t, Var& v) {
        auto ty = types[t];
        if (ty[0] >> 16 && (ty[0] & 0xffff) == 28) { v.arrayLen = consts.count(ty[3]) ? consts[ty[3]][3] : 1; t = ty[2]; ty = types[t]; }
        if ((ty[0] & 0xffff) == 23) { v.comps = ty[3]; t = ty[2]; ty = types[t]; } else v.comps = 1;
        uint32_t op = ty[0] & 0xffff;
        v.base = op == 22 ? 0 : (op == 21 && ty[3]) ? 1 : 2;
    };
    for (auto& ins : vars) {
        uint32_t ptr = ins[1], id = ins[2], sc = ins[3]; uint32_t pointee = types[ptr][3];
        if (sc == 1 || sc == 3) {
            Var v; v.builtin = builtin.count(id) || (types[pointee][0] & 0xffff) == 30; v.flat = flat.count(id);
            if (location.count(id)) v.location = location[id];
            if (!v.builtin) scalar(pointee, v);
            (sc == 1 ? m.inputs : m.outputs).push_back(v);
        } else if (sc == 9) m.push = true;
        else if (sc == 0 || sc == 2 || sc == 12) {
            if (!binding.count(id)) continue;
            uint32_t count = 1, t = pointee; auto ty = types[t];
            if ((ty[0] & 0xffff) == 28) { count = consts.count(ty[3]) ? std::max(1u, consts[ty[3]][3]) : 1; t = ty[2]; }
            else if ((ty[0] & 0xffff) == 29) { count = 64; t = ty[2]; }
            ty = types[t]; uint32_t op = ty[0] & 0xffff; VkDescriptorType dt;
            if (op == 25) { bool buf = ty[3] == 5; bool storage = ty[7] == 2; dt = buf ? (storage ? VK_DESCRIPTOR_TYPE_STORAGE_TEXEL_BUFFER : VK_DESCRIPTOR_TYPE_UNIFORM_TEXEL_BUFFER) : (storage ? VK_DESCRIPTOR_TYPE_STORAGE_IMAGE : VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE); }
            else if (op == 26) dt = VK_DESCRIPTOR_TYPE_SAMPLER;
            else if (op == 27) dt = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER;
            else if (sc == 12 || bufferBlock.count(t)) dt = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER;
            else dt = VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER;
            m.descs.push_back({set.count(id) ? set[id] : 0, binding[id], count, dt});
        }
    }
    m.words = std::move(w);
    return m;
}

static std::vector<uint32_t> Glsl(const std::string& src, const char* stage) {
    static int k = 0; std::string base = (fs::temp_directory_path() / ("sobench_stub" + std::to_string(k++))).string();
    std::ofstream(base + ".glsl") << src;
    std::string cmd = std::string("glslc -O -fshader-stage=") + stage + " --target-env=vulkan1.3 " + base + ".glsl -o " + base + ".spv";
    if (std::system(cmd.c_str()) != 0) { std::fprintf(stderr, "%s\n", src.c_str()); throw 1; }
    std::ifstream in(base + ".spv", std::ios::binary); std::vector<char> b((std::istreambuf_iterator<char>(in)), {});
    std::vector<uint32_t> out(b.size() / 4); std::memcpy(out.data(), b.data(), b.size()); return out;
}
static const char* GType(const Var& v, bool vec = true) {
    static const char* t[3][4] = {{"float", "vec2", "vec3", "vec4"}, {"int", "ivec2", "ivec3", "ivec4"}, {"uint", "uvec2", "uvec3", "uvec4"}};
    return t[v.base][(vec ? v.comps : 1) - 1];
}
static std::vector<uint32_t> StubVertexFor(const Module& fs) {
    std::string s = "#version 450\n"; std::string body;
    for (auto& v : fs.inputs) if (!v.builtin && v.location != ~0u) {
        std::string nm = "o" + std::to_string(v.location);
        s += "layout(location=" + std::to_string(v.location) + ") " + (v.flat || v.base ? "flat " : "") + "out " + GType(v) + " " + nm + (v.arrayLen > 1 ? "[" + std::to_string(v.arrayLen) + "]" : "") + ";\n";
        if (v.arrayLen > 1) for (uint32_t i = 0; i < v.arrayLen; ++i) body += nm + "[" + std::to_string(i) + "] = " + GType(v) + "(gl_VertexIndex);\n";
        else body += nm + " = " + GType(v) + "(gl_VertexIndex);\n";
    }
    return Glsl(s + "void main(){ gl_Position = vec4(float(gl_VertexIndex)); " + body + "}\n", "vert");
}

struct Vk {
    VkInstance inst; VkPhysicalDevice phys; VkDevice dev; uint32_t qf = 0; std::string name;
    PFN_vkCreateShadersEXT createShaders; PFN_vkDestroyShaderEXT destroyShader;
};
static Vk Open() {
    Vk vk{}; VkApplicationInfo app{VK_STRUCTURE_TYPE_APPLICATION_INFO}; app.apiVersion = VK_API_VERSION_1_3;
    VkInstanceCreateInfo ici{VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO}; ici.pApplicationInfo = &app;
    const char* layer = "VK_LAYER_KHRONOS_validation"; if (std::getenv("BENCH_VALIDATE")) { ici.enabledLayerCount = 1; ici.ppEnabledLayerNames = &layer; }
    CHECK(vkCreateInstance(&ici, nullptr, &vk.inst));
    uint32_t n = 0; vkEnumeratePhysicalDevices(vk.inst, &n, nullptr); std::vector<VkPhysicalDevice> ps(n); vkEnumeratePhysicalDevices(vk.inst, &n, ps.data());
    int want = std::getenv("BENCH_GPU") ? std::atoi(std::getenv("BENCH_GPU")) : 0; vk.phys = ps.at(want);
    VkPhysicalDeviceProperties pp; vkGetPhysicalDeviceProperties(vk.phys, &pp); vk.name = pp.deviceName;
    VkPhysicalDeviceShaderObjectFeaturesEXT so{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADER_OBJECT_FEATURES_EXT};
    VkPhysicalDeviceGraphicsPipelineLibraryFeaturesEXT gpl{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_GRAPHICS_PIPELINE_LIBRARY_FEATURES_EXT, &so};
    VkPhysicalDeviceVulkan13Features f13{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_FEATURES, &gpl};
    VkPhysicalDeviceVulkan12Features f12{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_2_FEATURES, &f13};
    VkPhysicalDeviceVulkan11Features f11{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_1_FEATURES, &f12};
    VkPhysicalDeviceFeatures2 f2{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2, &f11};
    vkGetPhysicalDeviceFeatures2(vk.phys, &f2);
    f2.features.robustBufferAccess = VK_FALSE;
    uint32_t en = 0; vkEnumerateDeviceExtensionProperties(vk.phys, nullptr, &en, nullptr); std::vector<VkExtensionProperties> ex(en); vkEnumerateDeviceExtensionProperties(vk.phys, nullptr, &en, ex.data());
    std::set<std::string> have; for (auto& e : ex) have.insert(e.extensionName);
    std::vector<const char*> exts;
    for (const char* e : {"VK_EXT_shader_object", "VK_EXT_graphics_pipeline_library", "VK_KHR_pipeline_library", "VK_KHR_fragment_shader_barycentric", "VK_EXT_shader_image_atomic_int64",
                          "VK_EXT_shader_atomic_float", "VK_KHR_shader_clock", "VK_EXT_fragment_shader_interlock", "VK_KHR_shader_maximal_reconvergence", "VK_KHR_shader_float_controls2",
                          "VK_KHR_shader_subgroup_uniform_control_flow", "VK_EXT_shader_stencil_export", "VK_AMD_shader_ballot", "VK_KHR_shader_relaxed_extended_instruction", "VK_EXT_shader_replicated_composites"})
        if (have.count(e)) exts.push_back(e);
    if (!so.shaderObject || !gpl.graphicsPipelineLibrary) { std::fprintf(stderr, "missing shaderObject/GPL\n"); std::exit(2); }
    // Enable every core/known feature the device reports, plus the extension feature structs we enable.
    void* tail = nullptr; std::vector<std::vector<char>> keep;
    auto chain = [&](VkStructureType t, size_t size, const char* ext) {
        if (!have.count(ext)) return;
        std::vector<char> b(size, 0); auto* h = reinterpret_cast<VkBaseOutStructure*>(b.data()); h->sType = t;
        VkPhysicalDeviceFeatures2 q{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2, h}; vkGetPhysicalDeviceFeatures2(vk.phys, &q);
        h->pNext = static_cast<VkBaseOutStructure*>(tail); keep.push_back(std::move(b)); tail = keep.back().data();
    };
    chain(VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FRAGMENT_SHADER_BARYCENTRIC_FEATURES_KHR, sizeof(VkPhysicalDeviceFragmentShaderBarycentricFeaturesKHR), "VK_KHR_fragment_shader_barycentric");
    chain(VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADER_IMAGE_ATOMIC_INT64_FEATURES_EXT, sizeof(VkPhysicalDeviceShaderImageAtomicInt64FeaturesEXT), "VK_EXT_shader_image_atomic_int64");
    chain(VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADER_ATOMIC_FLOAT_FEATURES_EXT, sizeof(VkPhysicalDeviceShaderAtomicFloatFeaturesEXT), "VK_EXT_shader_atomic_float");
    chain(VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADER_CLOCK_FEATURES_KHR, sizeof(VkPhysicalDeviceShaderClockFeaturesKHR), "VK_KHR_shader_clock");
    chain(VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FRAGMENT_SHADER_INTERLOCK_FEATURES_EXT, sizeof(VkPhysicalDeviceFragmentShaderInterlockFeaturesEXT), "VK_EXT_fragment_shader_interlock");
    chain(VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADER_MAXIMAL_RECONVERGENCE_FEATURES_KHR, sizeof(VkPhysicalDeviceShaderMaximalReconvergenceFeaturesKHR), "VK_KHR_shader_maximal_reconvergence");
    chain(VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADER_FLOAT_CONTROLS_2_FEATURES_KHR, sizeof(VkPhysicalDeviceShaderFloatControls2FeaturesKHR), "VK_KHR_shader_float_controls2");
    chain(VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADER_SUBGROUP_UNIFORM_CONTROL_FLOW_FEATURES_KHR, sizeof(VkPhysicalDeviceShaderSubgroupUniformControlFlowFeaturesKHR), "VK_KHR_shader_subgroup_uniform_control_flow");
    chain(VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADER_RELAXED_EXTENDED_INSTRUCTION_FEATURES_KHR, sizeof(VkPhysicalDeviceShaderRelaxedExtendedInstructionFeaturesKHR), "VK_KHR_shader_relaxed_extended_instruction");
    so.pNext = tail;
    float prio = 1; VkDeviceQueueCreateInfo q{VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO}; q.queueCount = 1; q.pQueuePriorities = &prio;
    VkDeviceCreateInfo dci{VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO, &f2}; dci.queueCreateInfoCount = 1; dci.pQueueCreateInfos = &q;
    dci.enabledExtensionCount = exts.size(); dci.ppEnabledExtensionNames = exts.data();
    CHECK(vkCreateDevice(vk.phys, &dci, nullptr, &vk.dev));
    vk.createShaders = (PFN_vkCreateShadersEXT)vkGetDeviceProcAddr(vk.dev, "vkCreateShadersEXT");
    vk.destroyShader = (PFN_vkDestroyShaderEXT)vkGetDeviceProcAddr(vk.dev, "vkDestroyShaderEXT");
    return vk;
}

struct Layout { std::vector<VkDescriptorSetLayout> sets; VkPipelineLayout pipe; VkPushConstantRange push; };
static Layout MakeLayout(Vk& vk, const Module& a, const Module& b) {
    std::map<uint32_t, std::map<uint32_t, VkDescriptorSetLayoutBinding>> sets;
    for (auto* m : {&a, &b}) for (auto& d : m->descs) {
        auto& bnd = sets[d.set][d.binding]; bnd.binding = d.binding; bnd.descriptorType = d.type; bnd.descriptorCount = std::max(bnd.descriptorCount, d.count); bnd.stageFlags = VK_SHADER_STAGE_ALL_GRAPHICS;
    }
    Layout l{}; uint32_t maxSet = sets.empty() ? 0 : sets.rbegin()->first + 1;
    for (uint32_t s = 0; s < maxSet; ++s) {
        std::vector<VkDescriptorSetLayoutBinding> bs; for (auto& [k, v] : sets[s]) bs.push_back(v);
        std::vector<VkDescriptorBindingFlags> fl(bs.size(), VK_DESCRIPTOR_BINDING_PARTIALLY_BOUND_BIT);
        VkDescriptorSetLayoutBindingFlagsCreateInfo bf{VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_BINDING_FLAGS_CREATE_INFO}; bf.bindingCount = fl.size(); bf.pBindingFlags = fl.data();
        VkDescriptorSetLayoutCreateInfo ci{VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO, &bf}; ci.bindingCount = bs.size(); ci.pBindings = bs.data();
        VkDescriptorSetLayout sl; CHECK(vkCreateDescriptorSetLayout(vk.dev, &ci, nullptr, &sl)); l.sets.push_back(sl);
    }
    l.push = {VK_SHADER_STAGE_ALL_GRAPHICS, 0, 256};
    VkPipelineLayoutCreateInfo pci{VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO}; pci.setLayoutCount = l.sets.size(); pci.pSetLayouts = l.sets.data(); pci.pushConstantRangeCount = 1; pci.pPushConstantRanges = &l.push;
    CHECK(vkCreatePipelineLayout(vk.dev, &pci, nullptr, &l.pipe));
    return l;
}

static VkFormat AttribFormat(const Var& v) {
    static const VkFormat f[3][4] = {{VK_FORMAT_R32_SFLOAT, VK_FORMAT_R32G32_SFLOAT, VK_FORMAT_R32G32B32_SFLOAT, VK_FORMAT_R32G32B32A32_SFLOAT},
                                     {VK_FORMAT_R32_SINT, VK_FORMAT_R32G32_SINT, VK_FORMAT_R32G32B32_SINT, VK_FORMAT_R32G32B32A32_SINT},
                                     {VK_FORMAT_R32_UINT, VK_FORMAT_R32G32_UINT, VK_FORMAT_R32G32B32_UINT, VK_FORMAT_R32G32B32A32_UINT}};
    return f[v.base][v.comps - 1];
}
static VkFormat ColorFormat(const Var& v) { return v.base == 0 ? VK_FORMAT_R16G16B16A16_SFLOAT : v.base == 1 ? VK_FORMAT_R32G32B32A32_SINT : VK_FORMAT_R32G32B32A32_UINT; }

struct Pair { const Module* vs; const Module* fs; };
struct State {
    std::vector<VkVertexInputBindingDescription> bindings; std::vector<VkVertexInputAttributeDescription> attrs; std::vector<VkFormat> colors;
    VkPipelineVertexInputStateCreateInfo vi{VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO};
    VkPipelineInputAssemblyStateCreateInfo ia{VK_STRUCTURE_TYPE_PIPELINE_INPUT_ASSEMBLY_STATE_CREATE_INFO};
    VkPipelineViewportStateCreateInfo vp{VK_STRUCTURE_TYPE_PIPELINE_VIEWPORT_STATE_CREATE_INFO};
    VkPipelineRasterizationStateCreateInfo rs{VK_STRUCTURE_TYPE_PIPELINE_RASTERIZATION_STATE_CREATE_INFO};
    VkPipelineMultisampleStateCreateInfo ms{VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO};
    VkPipelineDepthStencilStateCreateInfo ds{VK_STRUCTURE_TYPE_PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO};
    std::vector<VkPipelineColorBlendAttachmentState> blends; VkPipelineColorBlendStateCreateInfo cb{VK_STRUCTURE_TYPE_PIPELINE_COLOR_BLEND_STATE_CREATE_INFO};
    VkDynamicState dyn[2] = {VK_DYNAMIC_STATE_VIEWPORT, VK_DYNAMIC_STATE_SCISSOR}; VkPipelineDynamicStateCreateInfo dy{VK_STRUCTURE_TYPE_PIPELINE_DYNAMIC_STATE_CREATE_INFO};
    VkPipelineRenderingCreateInfo ri{VK_STRUCTURE_TYPE_PIPELINE_RENDERING_CREATE_INFO};
    explicit State(const Pair& p) {
        for (auto& v : p.vs->inputs) if (!v.builtin && v.location != ~0u) {
            bindings.push_back({(uint32_t)bindings.size(), 64, VK_VERTEX_INPUT_RATE_VERTEX});
            attrs.push_back({v.location, bindings.back().binding, AttribFormat(v), 0});
        }
        for (auto& v : p.fs->outputs) if (!v.builtin && v.location != ~0u) { if (colors.size() <= v.location) colors.resize(v.location + 1, VK_FORMAT_UNDEFINED); colors[v.location] = ColorFormat(v); }
        vi.vertexBindingDescriptionCount = bindings.size(); vi.pVertexBindingDescriptions = bindings.data(); vi.vertexAttributeDescriptionCount = attrs.size(); vi.pVertexAttributeDescriptions = attrs.data();
        ia.topology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST; vp.viewportCount = vp.scissorCount = 1;
        rs.lineWidth = 1; ms.rasterizationSamples = VK_SAMPLE_COUNT_1_BIT;
        blends.resize(colors.size()); for (auto& b : blends) b.colorWriteMask = 0xf;
        cb.attachmentCount = blends.size(); cb.pAttachments = blends.data();
        dy.dynamicStateCount = 2; dy.pDynamicStates = dyn;
        ri.colorAttachmentCount = colors.size(); ri.pColorAttachmentFormats = colors.data(); ri.depthAttachmentFormat = VK_FORMAT_D32_SFLOAT;
    }
};

static VkShaderModule ModuleOf(Vk& vk, const Module& m) {
    VkShaderModuleCreateInfo ci{VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO}; ci.codeSize = m.words.size() * 4; ci.pCode = m.words.data();
    VkShaderModule s; CHECK(vkCreateShaderModule(vk.dev, &ci, nullptr, &s)); return s;
}
static VkPipelineShaderStageCreateInfo StageOf(VkShaderStageFlagBits st, VkShaderModule m, const Module& mod) {
    VkPipelineShaderStageCreateInfo s{VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO}; s.stage = st; s.module = m; s.pName = mod.entry.c_str(); return s;
}

struct Result { double mono = -1, gplVs = -1, gplFs = -1, gplVi = -1, gplFo = -1, gplLink = -1, gplLto = -1, soVs = -1, soFs = -1; std::string error; };

static Result Run(Vk& vk, const Pair& p) {
    Result r; State st(p); Layout l = MakeLayout(vk, *p.vs, *p.fs);
    VkShaderModule vsm = ModuleOf(vk, *p.vs), fsm = ModuleOf(vk, *p.fs);
    VkPipelineShaderStageCreateInfo stages[2] = {StageOf(VK_SHADER_STAGE_VERTEX_BIT, vsm, *p.vs), StageOf(VK_SHADER_STAGE_FRAGMENT_BIT, fsm, *p.fs)};
    const std::string mode = std::getenv("BENCH_MODE") ? std::getenv("BENCH_MODE") : "all";
    try {
        if (mode == "all" || mode == "mono") {
        VkGraphicsPipelineCreateInfo g{VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO, &st.ri};
        g.stageCount = 2; g.pStages = stages; g.pVertexInputState = &st.vi; g.pInputAssemblyState = &st.ia; g.pViewportState = &st.vp; g.pRasterizationState = &st.rs;
        g.pMultisampleState = &st.ms; g.pDepthStencilState = &st.ds; g.pColorBlendState = &st.cb; g.pDynamicState = &st.dy; g.layout = l.pipe;
        VkPipeline mono; auto t0 = Clock::now(); CHECK(vkCreateGraphicsPipelines(vk.dev, VK_NULL_HANDLE, 1, &g, nullptr, &mono)); r.mono = ms(t0, Clock::now());
        vkDestroyPipeline(vk.dev, mono, nullptr);
        }
        if (mode == "all" || mode == "gpl") {
        // gpl: four libraries ahead, link at the draw
        auto lib = [&](VkGraphicsPipelineLibraryFlagsEXT part, double& out) {
            VkGraphicsPipelineLibraryCreateInfoEXT li{VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_LIBRARY_CREATE_INFO_EXT, &st.ri}; li.flags = part;
            VkGraphicsPipelineCreateInfo c{VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO, &li};
            c.flags = VK_PIPELINE_CREATE_LIBRARY_BIT_KHR | VK_PIPELINE_CREATE_RETAIN_LINK_TIME_OPTIMIZATION_INFO_BIT_EXT; c.layout = l.pipe;
            if (part == VK_GRAPHICS_PIPELINE_LIBRARY_VERTEX_INPUT_INTERFACE_BIT_EXT) { c.pVertexInputState = &st.vi; c.pInputAssemblyState = &st.ia; }
            if (part == VK_GRAPHICS_PIPELINE_LIBRARY_PRE_RASTERIZATION_SHADERS_BIT_EXT) { c.stageCount = 1; c.pStages = &stages[0]; c.pViewportState = &st.vp; c.pRasterizationState = &st.rs; c.pDynamicState = &st.dy; }
            if (part == VK_GRAPHICS_PIPELINE_LIBRARY_FRAGMENT_SHADER_BIT_EXT) { c.stageCount = 1; c.pStages = &stages[1]; c.pMultisampleState = &st.ms; c.pDepthStencilState = &st.ds; }
            if (part == VK_GRAPHICS_PIPELINE_LIBRARY_FRAGMENT_OUTPUT_INTERFACE_BIT_EXT) { c.pMultisampleState = &st.ms; c.pColorBlendState = &st.cb; }
            VkPipeline p_; auto t = Clock::now(); CHECK(vkCreateGraphicsPipelines(vk.dev, VK_NULL_HANDLE, 1, &c, nullptr, &p_)); out = ms(t, Clock::now()); return p_;
        };
        VkPipeline libs[4] = {lib(VK_GRAPHICS_PIPELINE_LIBRARY_VERTEX_INPUT_INTERFACE_BIT_EXT, r.gplVi), lib(VK_GRAPHICS_PIPELINE_LIBRARY_PRE_RASTERIZATION_SHADERS_BIT_EXT, r.gplVs),
                              lib(VK_GRAPHICS_PIPELINE_LIBRARY_FRAGMENT_SHADER_BIT_EXT, r.gplFs), lib(VK_GRAPHICS_PIPELINE_LIBRARY_FRAGMENT_OUTPUT_INTERFACE_BIT_EXT, r.gplFo)};
        auto link = [&](bool lto, double& out) {
            VkPipelineLibraryCreateInfoKHR li{VK_STRUCTURE_TYPE_PIPELINE_LIBRARY_CREATE_INFO_KHR}; li.libraryCount = 4; li.pLibraries = libs;
            VkGraphicsPipelineCreateInfo c{VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO, &li}; c.layout = l.pipe; if (lto) c.flags = VK_PIPELINE_CREATE_LINK_TIME_OPTIMIZATION_BIT_EXT;
            VkPipeline p_; auto t = Clock::now(); CHECK(vkCreateGraphicsPipelines(vk.dev, VK_NULL_HANDLE, 1, &c, nullptr, &p_)); out = ms(t, Clock::now()); vkDestroyPipeline(vk.dev, p_, nullptr);
        };
        link(false, r.gplLink); link(true, r.gplLto);
        for (auto x : libs) vkDestroyPipeline(vk.dev, x, nullptr);
        }
        if (mode == "all" || mode == "so") {
        // shader objects, unlinked, created one by one (as registration would)
        auto obj = [&](const Module& m, VkShaderStageFlagBits stage, VkShaderStageFlags next, double& out) {
            VkShaderCreateInfoEXT c{VK_STRUCTURE_TYPE_SHADER_CREATE_INFO_EXT}; c.stage = stage; c.nextStage = next; c.codeType = VK_SHADER_CODE_TYPE_SPIRV_EXT;
            c.codeSize = m.words.size() * 4; c.pCode = m.words.data(); c.pName = m.entry.c_str(); c.setLayoutCount = l.sets.size(); c.pSetLayouts = l.sets.data();
            c.pushConstantRangeCount = 1; c.pPushConstantRanges = &l.push;
            VkShaderEXT s; auto t = Clock::now(); CHECK(vk.createShaders(vk.dev, 1, &c, nullptr, &s)); out = ms(t, Clock::now()); vk.destroyShader(vk.dev, s, nullptr);
        };
        obj(*p.vs, VK_SHADER_STAGE_VERTEX_BIT, VK_SHADER_STAGE_FRAGMENT_BIT, r.soVs); obj(*p.fs, VK_SHADER_STAGE_FRAGMENT_BIT, 0, r.soFs);
        }
    } catch (VkResult e) { r.error = "VkResult " + std::to_string(e); }
    vkDestroyShaderModule(vk.dev, vsm, nullptr); vkDestroyShaderModule(vk.dev, fsm, nullptr);
    vkDestroyPipelineLayout(vk.dev, l.pipe, nullptr); for (auto s : l.sets) vkDestroyDescriptorSetLayout(vk.dev, s, nullptr);
    return r;
}

int main(int argc, char** argv) {
    std::vector<Module> mods;
    std::vector<fs::path> files; for (auto& e : fs::directory_iterator(argv[1])) if (e.path().extension() == ".spv") files.push_back(e.path());
    std::sort(files.begin(), files.end());
    std::set<std::vector<uint32_t>> unique;
    for (auto& f : files) {
        std::ifstream in(f, std::ios::binary); std::vector<char> b((std::istreambuf_iterator<char>(in)), {});
        std::vector<uint32_t> w(b.size() / 4); std::memcpy(w.data(), b.data(), b.size());
        if (w.size() < 5 || !unique.insert(w).second) continue;
        auto m = Reflect(f.filename().string(), std::move(w)); if (m.model == 0 || m.model == 4) mods.push_back(std::move(m));
    }
    const size_t limit = argc > 2 ? std::atoi(argv[2]) : mods.size();
    Vk vk = Open(); std::fprintf(stderr, "device %s, %zu unique graphics modules\n", vk.name.c_str(), mods.size());
    Module stubFs = Reflect("stubfs", Glsl("#version 450\nvoid main(){}\n", "frag"));
    std::map<std::string, Module> stubVs;
    std::printf("name\tstage\twords\tmono\tgpl_vi\tgpl_vs\tgpl_fs\tgpl_fo\tgpl_link\tgpl_lto\tso_vs\tso_fs\terror\n");
    size_t done = 0;
    for (auto& m : mods) {
        if (done++ >= limit) break;
        Pair p{};
        if (m.model == 0) p = {&m, &stubFs};
        else {
            std::string key; for (auto& v : m.inputs) if (!v.builtin) key += std::to_string(v.location) + ":" + std::to_string(v.base) + std::to_string(v.comps) + std::to_string(v.flat) + "x" + std::to_string(v.arrayLen) + ",";
            if (!stubVs.count(key)) stubVs.emplace(key, Reflect("stubvs", StubVertexFor(m)));
            p = {&stubVs.at(key), &m};
        }
        Result r = Run(vk, p);
        std::printf("%s\t%s\t%zu\t%.3f\t%.3f\t%.3f\t%.3f\t%.3f\t%.3f\t%.3f\t%.3f\t%.3f\t%s\n", m.name.c_str(), m.model == 0 ? "vs" : "fs", m.words.size(),
                    r.mono, r.gplVi, r.gplVs, r.gplFs, r.gplFo, r.gplLink, r.gplLto, r.soVs, r.soFs, r.error.c_str());
        std::fflush(stdout);
    }
}

[Raghav314]

@adriadam10 I reran PPSA36082 cold starts on the current head (14b52c1) on my machine, same setup as above: RTX 4090, Linux, fresh shader, pipeline and driver caches every run, APS5_HOST_IMPORT_MIB=6784.

The new commits don't change the compute shader from my earlier comment. formattedCall's helper still takes the V# words as parameters in the specialized form, so it's still 236,457 words with a 794 ms pipeline create, and the stock boot still dies loading GUI.pkg. The helper change from my earlier comment applies cleanly on 14b52c1 and still fixes it:

	14b52c1	14b52c1 + helper change
stock eboot, cold	3/3 segfault loading GUI.pkg	3/3 run the full 15 s
frame 1 (first submit to flip)	1715 ms (bigger-pool eboot, 8832)	782 / 801 / 792 ms
the frame's one compute pipeline	236,457 words, 794 ms	51,734 words, 22 ms
graphics libraries at draws over the boot	16, 258 ms	15, 253 ms

For comparison, main was 1041 ms for frame 1 with 267 ms of graphics compiles in it, and it crashes the same way.

Tests, the Linux job of build.yml on 14b52c1 + the change, on the 4090: 921/923. The two failures (agc_vector_window, agc_driver_prepared_shaders_registered_workgroup_sync) fail the same way on main.

agc_driver_pixel_lds hung the GPU again on 14b52c1 as pushed, without my change: VK_ERROR_DEVICE_LOST and Xid 109 (CTX SWITCH TIMEOUT) on the first run. Across this PR's heads that's 3 hangs in 4 runs on my machine (it passed once inside a full test run), and main has passed it 7 out of 7 on a freshly booted GPU. I don't know the cause yet. After a hang every GPU test fails until a reboot, so I couldn't repeat it more today.

LLM assistance used.

[Raghav314]

@adriadam10 one more on the same helper, this time in registration form. I dumped every shader Demon's Souls (PPSA01341) registers in its first two minutes on my machine (2,388 requests: 2,106 compute, 199 vertex, 83 pixel) and compiled each graphics one offline in both forms with agc_shader_replay, on 14b52c1 + the helper change from my earlier comment (+ #4049, which the game needs to get through start-up).

The vertex shaders' registration form was 3.3x the size of their specialized form on average, and 39 of the 58 largest were over registrationWordLimit. Almost all of it is formattedCall's helper again. 14b52c1 specializes the format word at the call site for descriptorFormatted buffers, but the helper is DontInline and still takes word 3 as a parameter, so its body keeps the full format switch. The largest shader has 10 of these helpers at about 21,800 lines each, with 12 OpSwitches over bits 12-18 of that parameter, roughly 90% of the module.

Reading word 3 as the specialization constant inside the helper when the format is specialized, on top of the earlier change:

change

             state.bufferMisalignment = parameters[9];
-            shared = {ConstantU32(state, 0u), parameters[3], parameters[5], parameters[6], nonzero(state, parameters[7]), parameters[0], nonzero(state, parameters[8]), parameters[1], parameters[2]};
+            const bool runtimeFormat = memory.gpuDescriptor || !state.program.Info().buffers.at(memory.resource).descriptorFormatted;
+            const auto word3 = runtimeFormat ? parameters[6] : state.module.SpecializationConstant(u32, PipelineSpecialization::BufferBase + memory.resource * PipelineSpecialization::BufferWords + 1u, 0u);
+            shared = {ConstantU32(state, 0u), parameters[3], parameters[5], word3, nonzero(state, parameters[7]), parameters[0], nonzero(state, parameters[8]), parameters[1], parameters[2]};

58 largest graphics shaders	14b52c1 + earlier change	+ this change
registration form vs specialized form, mean	3.26x	1.10x
over 200k words in registration form	39	0
largest	1,040,760 words	176,445 words

The pixel shaders don't use this helper much: their registration form is about 2.1x the specialized one and none of the 82 is over 200k, with or without the change. Since RegistrationResult skips shaders with descriptorFormatted buffers, these vertex shaders still aren't prebuilt; what changes is the size of the module compiled at their draw. These are word counts only, I haven't timed the library compiles.

The tests that cover this (vertex fetch, buffers, formats, prepared shaders and registration, 51 of them) pass on my 4090 except agc_driver_prepared_shaders_registered_workgroup_sync, which fails the same way on main.

Your 60-shader table was measured before 14b52c1. If Astro's large pixel shaders go through the same helper, it might be worth rerunning that sample with both changes, since it could move the cliff the limit is based on.

LLM assistance used.

[Amoenus]

Independent Windows / RTX5080 check of exact #4152 head 14b52c1ac versus merge-base bca7a29d2: agc_driver_pixel_lds passes five isolated repeats on each revision and both GPL-on/off suites. No timeout or Vulkan device-loss error logged. The same two unrelated test failures reproduce on both revisions; GPL-off skips also match. Exact counts, configuration and limitations.

This uses Windows / NVIDIA610.62 on an already running GPU; it is not a fresh-boot/Linux4090 reproduction, and does not rule out your Xid109 observation. Driver disk-cache coldness and title cold boots remain untested. AI-assisted: Codex.

[adriadam10]

@Raghav314 thanks, both helper changes are in #4152 (head b1ee794, rebased on current main), plus one more aimed at agc_driver_pixel_lds:

• Helper, specialized form: your first change. The V# words are specialization constants inside the helper again. FormattedBufferOnce now pins it: one formatted load gives 26.5k words, main 26.5k, 14b52c1 117.5k.
• Helper, registration form: your second change. The format word is a constant inside the helper when the buffer is accessed with the descriptor's format. Pinned too: a vertex shader's registration form stays under 1.5× its specialized form.
• pixel_lds: comparing that test's pixel shader in both forms, registration form had 6 loops and 6 OpAtomicCompareExchange, specialized none. That's the unaligned store's read-modify-write path, kept alive because the base misalignment was read at run time. It only runs for unaligned bases, but it's the only structural difference I found, so the misalignment is a 1-bit specialization again (both forms: 0 loops, 0 atomics). I can't check it against the Xid 109 without an NVIDIA GPU. Amoenus ran 14b52c1 on an RTX 5080 (Windows) and pixel_lds passed 5/5 there, so it may be specific to your setup.

On the limit: re-measured on Astro's Playroom with both helper changes. Below 200k words, registration form compiles ×1.84-2.24 slower, with modules only ×1.10-1.14 the size. Above, with the limit off, 20 shaders still compiled ×5.9 slower (×20 on the optimized link), so the limit stays where it is. The cost is runtime values the driver can't fold, not size.

Local: suites 433/433 both modes, validation layer clean, CI's lavapipe 430/430, CI's configuration 439/439. If you get a chance: pixel_lds on the 4090, and a PPSA36082 cold start on b1ee794.

AI-assisted: yes (Claude).

[adriadam10]

#4152 is merged (2979ae3). On GPL devices, vertex and pixel shaders now get their pipeline libraries built at registration.

@Raghav314 your cold start with the helper change booted PPSA36082 3/3, and both helper changes are in. The one thing I couldn't confirm is agc_driver_pixel_lds on your 4090: the unaligned-store loops that only registration form had are gone, but if it still hangs there with Xid 109 please tell me and I'll take it from there.

Next on my side: #2866 (the background LTO relink losing the device on NVIDIA with a depth attachment), which comes from #2713.

AI-assisted: yes (Claude).