Guard NEO's scratch allocation with a spill-triggered drain - #609
Draft
michel2323 wants to merge 1 commit into
Draft
Guard NEO's scratch allocation with a spill-triggered drain#609michel2323 wants to merge 1 commit into
michel2323 wants to merge 1 commit into
Conversation
NEO allocates a queue's scratch buffer at the first submission of a kernel whose spill exceeds what is already allocated, and that allocation has no error path: allocateGraphicsMemoryWithProperties is called without a null check, and programSurfaceState aborts the process (UNRECOVERABLE_IF) when it failed. Whether it fails depends on how much dead-but-unfinalized driver memory happens to be live at that one instant — a GC lottery. Remove the lottery: track a per-queue high-water mark of kernel spill sizes, and before the first submission that crosses it, retire in-flight work, flush deferred releases, and run finalizers, so the fatal allocation happens at the cleanest reachable moment. Fires once per (queue, scratch tier); costs one cached-Int compare on the non-spilling fast path. Opt out with ONEAPI_SCRATCH_HEDGE=0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WSFxSBXtckG3BVAT12wYkf
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #609 +/- ##
==========================================
+ Coverage 78.51% 78.80% +0.29%
==========================================
Files 50 50
Lines 3490 3505 +15
==========================================
+ Hits 2740 2762 +22
+ Misses 750 743 -7 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NEO allocates a queue's scratch buffer at the first submission of a kernel whose spill exceeds what is already allocated, and that allocation has no error path:
prepareScratchAllocation()callsallocateGraphicsMemoryWithProperties()without a null check, andprogramSurfaceState()aborts the process (UNRECOVERABLE_IF,scratch_space_controller_xehp_and_later.cpp:78, byte-identical from the 25.18 LTS driver through current master) when it failed. Whether it fails depends on how much dead-but-unfinalized driver memory happens to be live at that one instant — a GC lottery. We have logged repeated aborts of exactly this shape on Aurora (PVC, LTS NEO 25.18) under launch storms from real workloads (ExaModels), where the process dies insidezeCommandQueueExecuteCommandListswith no Julia-side recourse.This PR removes the lottery instead of the (driver-side) bug:
ZeKernelcaches itsspillMemSize(seeded byproperties(), one Int load on the launch path)._run_reclaim_callbacks), and runsGC.gc(false)— so the one allocation the driver cannot recover from happens at the cleanest reachable moment.The hedge fires once per (queue, scratch tier): one synchronize per workload in practice, and a cached-Int compare on the non-spilling fast path. Opt out with
ONEAPI_SCRATCH_HEDGE=0(the high-water mark is maintained regardless).Tested with a deliberately spilling kernel (256 live accumulators): the hedge fires exactly once, never for no-spill kernels, and the knob-off path still tracks the mark. Suite green on Aurora LTS (PVC) and the kernel spills on both PVC and DG2.
This is deliberately independent of the immediate-command-list rework (#610): it guards a different resource (scratch) against a different garbage source (USM churn), and remains useful on current drivers — the missing null check is in NEO's shared code, not LTS-specific.
🤖 Generated with Claude Code
https://claude.ai/code/session_01WSFxSBXtckG3BVAT12wYkf