fossilize-replay uses up too much memory (Bioshock Infinite)

[Frogging101]

[  230.058391] Out of memory: Killed process 2954 (fossilize_repla) total-vm:2958052kB, anon-rss:2297484kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:5272kB oom_score_adj:0

Attempting to replay the steam_pipeline_cache.foz from Bioshock Infinite (8870) distributed by Steam (Proton 7.0-4; AMD VEGA10, also Vangogh) results in excessive memory usage by fossilize-replay. The memory usage seems to scale with the number of threads. For reasons I do not know, Steam on my desktop requests 7 threads but the rest of the threads get killed off shortly after starting and does not trigger an OOM. On the Deck, however, the resulting memory usage causes an OOM condition, which takes down most of the userspace and causes Steam to restart.

Here is the foz db for ease of reproducing: http://www.fastquake.com/files/tmp/steam_pipeline_cache.bioshockinfinite.vega10.proton7-0-4.foz

[Frogging101]

I've done some investigation. The high memory usage is due to the sheer number of unique VkSampler objects in the fossilize DB. Replaying the DB produces over 9 million unique VkSamplers with over 8.3M unique values for minLod and 5.5M unique values for maxLod.

I added print statements to DXVK, and it looks like the game creates new samplers every time the camera moves, with minLod and maxLod calculated to full precision from what I assume is the camera position. This creates new, likely unique samplers for everywhere the player can be in the game.

I'm still exploring what to do about this.

[HansKristian-Work]

That is an annoying problem, ye ... The problem is that we don't really know if a VkSampler will be used as an immutable sampler down the line, but we only need to record those samplers. We'd probably need a heuristic to not samplers that are unlikely to be immutable (odd values of minLod, etc). We can also app-opt DXVK since it does not use immutable samplers at all.