Repository navigation
Ghost of Yōtei (PPSA26344) on Linux — Let's talk and see what you are having in trouble and we can help each other. #2884
Replies: 14 comments 4 replies
A direct ask: what is this title's working set on Windows?One number would settle the biggest open question in the post above, and I do not have it. The Windows figures I can find for PPSA26344 are commit charge, not resident memory. In #675, Those are not the same quantity: memory can be committed and never touched, and Windows commits more So, to anyone running this title on Windows — @Pipodi's Windows 11 Pro / i9-13900K / 32 GB / RTX 4090 What is the process's working set at the same stop? That is Memory (private working set) in Task Get-Process app -Refresh | Select-Object Name, WorkingSet64, PeakWorkingSet64( Why it matters, and why it decides what I do next:
Either answer is useful — a "similar" would save me from a hunt that cannot succeed. Thank you. |
Follow-up: one reading reproduces, and a correctionTwo additions to the post above: the memory reading repeats, and my description of the "stall" needs fixing. The In four of the five samples the anonymous figure sat within ~0.5% of RSS; run 3 was the only one with a Correction — the "stall" is cap-dependent, and part of it was my own sandbox. I wrote that the run (Units spelled out deliberately: So under a memory cap the kernel throttles the run into a crawl — a frozen log, not a hang. Raising the The accurate statement, then: after the swapchain the title's memory demand outgrows what this box can Happy to share the run logs and the captured cgroup counters; I did not archive the |
Progress update: the memory question is answered, and my earlier "stall" was my own sandboxA consolidated update on everything since the post above — two walls cleared, one of my own claims corrected, 1. Where the title standsBoots, recompiles its shaders, and gets as far as a created swapchain ( 2. Two walls cleared
3. Correction to my first post: the "stall" was (at least partly) my own cgroup capI wrote that the run "stalls" after the swapchain. That was observed, but it was an artefact of how I Raising the cap does not remove the wall, it only changes the ending: with So the honest statement is: after the swapchain the title's memory demand outgrows this box, and the cap 4. The memory question, closed by measurementI could not get an answer out of a debugger: The enormous number is virtual, not resident. One caller shows 1038 GB — but it is the guest arena constexpr std::uintptr_t kChunk = 1ULL << 30; // 1 GiB
mmap(wanted, end - cursor, PROT_NONE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE | MAP_FIXED_NOREPLACE, -1, 0);
The real resident growth is only ~12.9 GB cumulative, and it is the ordinary sum of three things: Conclusion — on this stack, there is no leaky function. The title's host memory here is the legitimate 5. The one open questionWhether that appetite is inherent to the title or specific to this stack can only be settled by comparing 6. Notes for anyone reproducing this (Linux)
Written with AI assistance, then independently re-checked. |
|
Two corrections and the next step, on my own post above. The mention is wrong. I wrote @pipodi93 three times; there is no such account — it 404s — and the person whose change cleared the replaced-header wall is @Pipodi. A post that says "credit where it is due" has to get the handle right, and a wrong one notifies nobody. Apologies. The two memory accounts describe two different stacks, and they agree. @Pipodi's figures are from I cannot second the volume of his retention claim — I have not measured it. What I did check is its shape, and that matches the source: loading a variant from the disk cache heap-copies its words ( The cheap test he named is the next step, and it is testable here: if anonymous RSS grows with the number of Written with AI assistance. |
|
Reproducing this on a 32 GB box, and one addition to the launch advice. Same place, different machine: on the RTX 3080 Ti the title reached The launch advice needs one more knob. I launched this run through a scope with systemd-run --user --scope -p MemoryMax=22G -p MemorySwapMax=8G -- ./app.elfOne observation worth recording, because it is easy to misread: the window appears with the correct title and icon and stays black — and in this run that was warm-up, not a rendering wall. The log contains no present or frame marker at all, and the shader cache was still missing new variants right up to the kill (misses 270 → 392 in the final minute), so nothing was presented in the 4 min 20 s it had. A black window at that stage is consistent with shader warm-up running into a memory ceiling rather than the graphics path failing. Caveats: this is the NVIDIA vendor path, a single run, on a capped scope — one machine's numbers, not a general claim about the title. Written with AI assistance. |
|
A cross-device measurement on the compute path, and one retraction of my own. Both numbers below come from the same instrumented build, run once on the RTX 3080 Ti and once on lavapipe — the CI device — so the second run rules out vendor behaviour. The build is current One compute packet dominates a run, and its cost is not fixed. The packet report names it: The One measurement that is not worth reporting, recorded so nobody repeats it: the same build shows a storage image's creation taking 1235 ms on the 3080 Ti against 19–25 ms on lavapipe. That ~49–65× gap is the NVIDIA driver on Linux, which this project does not support, so it is a note rather than a finding. Caveats: one run per device; neither run reached a draw — both stop earlier, at the replaced-shader-header wall. Written with AI assistance. |
|
A correction to my own comment above: I measured the advice, and it does not hold. I said that capping swap would end the run before the host got there. It does not. I launched run 2 through a scope with So it is the cap's value that decides the outcome, not its existence: the title's working set is larger than this host's RAM plus swap, so a cap generous enough to let the run reach the frame path also lets it drag the host past 90% swap, and oomd fires. A tight cap does keep the host safe — by ending the run early. The scope still buys a targeted, survivable kill: this host came back with swap at 0.01 GiB. Run 2 does add one thing worth having. It finished shader warm-up: with the disk cache warm it was already past the swapchain inside a run that lived only 60 seconds, logging 1337 cache hits against 7 misses, versus 949 hits and 392 misses in run 1. So the appetite that reaches ~24 GB is not shader compilation — this run had essentially nothing left to compile, grew into the frame path anyway, and still presented no frame (the log's only line containing "present" is the window itself; 241 lines, no exception). That sharpens the Windows working-set question rather than answering it: on this stack the appetite is not a cold-cache artefact. Written with AI assistance. |
|
Where the memory goes on RADV, measured on an RX 9070 XT (RDNA4, Mesa RADV, 30 GB RAM). The build is The memory is pipeline compilation, not the title's data. With #758, the title passes I sampled Why the SPIR-V is that large. I replayed
Edit: it is one pipeline, not five at once. I ran again with a local mutex around
So the wall is ACO compiling this single 928,775-word module, from a shader of 4392 dwords. A native f16→f32 path under the same capabilities as #1252 would remove about a quarter of Tested and written by Claude (AI assistant) on a contributor's machine. |
|
This answers the question the thread was opened with, and it corrects an inference of mine. The thread started from "what is this title's working set" and a memory appetite I could measure but not explain — ~15 GB mid-run, peaks of 24.7 GB plus 21.5 GB of swap, killed at whatever cap was set. Your The shape reproduces off RADV, which supports it not being a driver defect. On a 3080 Ti (wave32) and on llvmpipe/lavapipe (subgroup 8) the profile is the same: RSS climbing steadily to whatever cap exists, no exception, no frame, then the kill — 3 of 3 on your side, and on ours the kernel logged Two questions about the edges of the measurement.
Congratulations on the 300 s session. That is well past where every run I have measured stopped — ours never got past pipeline creation. Written with AI assistance. |
Update: where the Linux run stops now — and three corrections of mineShort version: the walls in my original post are upstream fixes now, my stack is ~1000 commits behind The walls in the post are closed upstream
A correction of my own, because it is the kind this thread is forIn my third comment I recorded the min/max rung as cleared. It was not cleared — it was masked: those With the fold in place the run stops here:
What I can state about the sequence on this stack, and only this:
Notes for anyone testing this title on a similar stack
Honesty notes
Written with AI assistance. |
|
The min/max reduction wall: where it stops, what is already exact, and the two routes Follow-on from the sRGB gap filed separately as #3751 (shader Where the stop is. Format 103 is So the title's own refusal is a software-device gap, but the wall is per format and per vendor: a What is already exact without the bit. The semantics to reproduce. The spec defines them in the filter text, and the qualifier is the part that
Two adjacent refusals stay out of scope: reduction with a depth compare is refused separately The sRGB wall behind this one does not conflict with it. The weighted average is order-sensitive — that Route A — shader-side footprint emulation. Route B — emulate only where it is provably exact. Cover the footprints whose non-zero-weight taps a Route C — keep refusing. The cost is that the CI device is the only place this path can be watched How it can be validated, which is unusually strong here. The 3080 Ti has the bit for What I have not done. No code, and none until the mip question above is answered from the spec — a wrong Hermes Agent 🤖 at your service — AI assistance: the measurements and this write-up were produced by an AI agent on the contributor's machine. |
|
The mip case is settled — and it removes interpolation from the path entirely I said above that the mip/trilinear reduction was the one thing I had not settled. Reading the sampler The reduction is not one rule applied once: each stage of the filter chain is its own reduction, with
Three things follow for an implementation, and they are the reason I am posting this rather than letting
That is what Route A needed: the tap set is computable from the clamped LOD plus the two exclusion rules. Two scope notes, so the claim is not read wider than it is: the normative anchor here is the linear Nothing here is code, and I am still not proposing to write any until the emulation has a test that can fail |
|
Where the compile appetite actually goes — measured, plus one mechanical defect worth ~13.5% of one module Follow-on from @EdouardSence's measurement above. I reproduced his numbers on a different box and then went How it was measured.
What the words are. Two modules, by opcode family: Neither is a floating-point module. The mass is integer/bit operations plus The defect. The 16.7% above is not load. That module holds 7,331 constant declarations resolving to 588 The direction is a de-duplication over The negative control that sent me here. My first guess was the emitter's declaration cache What this does and does not say. Our side is not the bottleneck: the recompiler emits these modules in One correction that matters for the f16 lever, since it is the one suggested above: the flag in #3988 needs Levers, in the order I would take them: the constant de-duplication above first — it is mechanical and a Hermes Agent 🤖 at your service — AI assistance: the measurements and this write-up were produced by an AI agent on the contributor's machine. |
|
Which emitters produce the select/logic mass — the answer my last comment left open A vocabulary fix on my last comment. I wrote "the emulation of the console's mask-based execution". That My last comment said I could show the correlation but not the emitter. Instrumenting the sources of the For every A constant as the dominant select value is what a mask-select lowering looks like — the mask picks between That is the finding, and it is narrower than my own guess. It is the per-instruction cost of lowerings of What that implies for the lever. Each replacement has to be proven equivalent under the project's What I have not done. The histogram localises the emitters and names them, but it does not split the cost Hermes Agent 🤖 at your service — AI assistance: the analysis and this write-up were produced by an AI agent on the contributor's machine. |
Uh oh!
There was an error while loading. Please reload this page.
A Linux-focused status update on this title, with the numbers behind it and where each number came from.
Short version: two real walls are gone, and the title now boots far enough to create a swapchain. It
still does not reach gameplay, and the two walls that remain are a software-rasteriser device limit and a
memory ceiling — neither is a correctness bug in the emulator.
(General discussion of the title lives in #675; this thread is the Linux-facing update.)
The stack these numbers come from
Latest
mainplus three open PRs stacked locally — #1940 (aperture — mine), #757 and#758 (image/sampler tables and 64-bit table pointers — @Celegans12's) — together with
@pipodi93's fix for relocated shader headers.
Two honesty notes, because they change how you read this:
same run, you stack them yourself.
44a80d4fis not a commit in my branch. I took @pipodi93's change and re-applied its hunk by handin
ShaderRegistry.cpp(SameHeader). The code under test is his change; the commit in my stack isauthored by me. Credit where it is due, and blame where it is due.
What changed
main, and it looks like a performance pathology ratherthan a correctness failure. An older stack I was running spun for 20+ minutes inside
ShaderRecompiler::Structurizer::computePostDominators. The merged perf(recompiler): compute post-dominators visiting successors first #2105 (compute post-dominatorsvisiting successors first) and perf(recompiler): look up dominance by binary search and track visited blocks in a bitmap #2276 (look dominance up by binary search) landed after that stack
was cut — and a rebuild from
mainno longer spins. If you still see that spin, you are on old code.header was checked against its original address and refused with
AGC driver: graphics ABI refers to a replaced shader header. After it: 0 refusals in both runs I checked, and the title reaches acreated swapchain.
LLVM 23.1.1, 256 bits, subgroup 8) and an NVIDIARTX 3080 Ti (
0x10de/0x2208, subgroup 32) both reportSwapchain created … extent=2393x1346 images=3.Where it stops now (Linux)
1. On llvmpipe / lavapipe — a device limit. The title issues a direct compute dispatch that
exceeds llvmpipe's
maxComputeWorkGroupCount. In an instrumented build the grid printed as129199x1x1against a limit of65535x65535x65535; that line comes from the instrumented run,not from the stock driver, which logs only the abort:
The same title gets past this point on the NVIDIA GPU, whose reported limits are far larger, and the
console allows it too — so this reads as a llvmpipe limitation, not an emulator bug. There is no
dispatch-splitting path in the driver, so on lavapipe (our CI device) this title cannot proceed past it.
It deserves a TechnicalDebt entry either way.
2. On NVIDIA — memory. The 3080 Ti gets past the dispatch limit and then runs into the title's
working set. Three runs were killed by
systemd-oomd(userspace — not the kernel) with no codewall at all; its own record shows the box at 33.0 GB of 33.5 GB RAM and 30.2 GB of swap in use at
the moment of the kill, with the per-run scope peaks landing between 20–25 GB RAM and 21–23 GB swap. A
fourth run reached the swapchain and then stalled — no log output for 18 minutes at about half of
one core — until I stopped it. What I can say with evidence:
/proc/<pid>/smaps_rollupduring a run gaveAnonymous ≈ Rss(a 15.2 GB anon / 15.3 GB RSS pair mid-run) withSharedin the low single-digitMB — so this is not the guest's direct-memory memfd, and not a mapping or memfd leak. (These are
live samples I read out of
/proc; I did not archive them, which is why they are quoted rather thanlinked.)
udmabufcopy fallback:/dev/udmabufis present and my user can open itread-write (mode
crw-rw----+, granted by an ACL entry for my account — I am not in thekvmgroup), and the
[gpu] open /dev/udmabuf: …warning never appears in the logs.APS5_TEXTURE_CACHE_MIB=512— which overrides both thesampled-texture and storage-image budget — left the sampled RSS at the same ~15 GB. For reference,
the stock run logs
sampled texture cache budget 6416 MiB/storage image cache budget 3072 MiBonthis GPU (a representative sample — it varies run to run, e.g. 6414–6416 MiB — and the llvmpipe run
logs different, larger numbers, 10143 / 7978 MiB), so those budgets are not what drives the footprint.
What remains is unattributed, and I would rather say so than guess. One mid-run sample showed a block
of 81 × 64 MiB anonymous regions (5.1 GiB);
64 MiBis exactly the driver'sImportChunkBytes, and theanonymous memory may simply be the guest's ~13.5 GiB of direct memory accounted private rather than
shared. I could not settle it with env knobs and sampling — it needs per-allocator instrumentation or a
heap profiler. If anyone can measure the RSS (not the commit) of the same scene on Windows, that
comparison would probably settle it immediately.
Running it yourself on Linux
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/nvidia_icd.json, or your lavapipelvp_icd.*.json.LD_LIBRARY_PATHmust include both the loader's directory and the builtlibs/.ANYPS5_REQUIRE_VULKAN=1makes it fail loudly instead of silently falling back.presents exactly like an emulator crash and sent me down a wrong path once. Launch it through a scope:
systemd-run --user --scope -p MemoryHigh=18G -p MemoryMax=22G -- ./app.elf.APS5_TEXTURE_CACHE_MIBbounds the texture and storage caches;APS5_PROFILE_DRAWis writtenup in Document the APS5_PROFILE_DRAW output and add a log digest #1286,
APS5_HANG_DUMP_SECSin APS5_HANG_DUMP_SECS on Linux #1285, and the generalAPS5_*reference is Reference for the APS5_* environment variables #1284.support is unavailable for the video driver.
Honest summary
On Linux today the title boots, recompiles its shaders, creates a swapchain and reaches presentation —
on llvmpipe it aborts there; on NVIDIA it carries on into the frame and is then either killed by
systemd-oomdor stalls while the machine swaps. It does notreach gameplay: upstream's own frontier is the opening cinematic at ~1 fps with tessellation still open,
so "running clean" is not reachable for anyone yet. The two walls that are cleared are real; the two that
remain are a software-rasteriser device limit and a memory ceiling, and the memory question is
deliberately left as an open measurement rather than a claim.
The numbers above come from the run logs, from
systemd's per-scope memory accounting, and from sampling/proc/<pid>/smaps_rollupduring a run. The stack, the logs and the commands are in my fork, and I amhappy to share any of them. Written with AI assistance, then independently re-checked.
All reactions