Anticipatory shader cache loading

[roothorick]

It would dramatically reduce stutters on subsequent launches in some games if dxvk keeps track of shaders and pipelines that were used on previous runs, and recreates them on the side before the application requests them. This way they're ready when the app needs them, avoiding hitches that could potentially be lengthy if the app suddenly needs many and/or large shaders.

The obvious approach is flat files stored somewhere (probably yet another environment variable for now) with the executable's full path and a table of remembered pipelines and shaders. Upon device creation, dxvk would check for a file corresponding to the executable path, and if found, recreate the listed pipelines from shader files stored on disk, preferably in one or more dedicated worker threads.

[doitsujin]

The Vulkan pipeline cache should already handle this well enough, the only thing left to do is to actually store it somewhere (an obvious place would be $HOME/.cache on Linux). I think RADV already stores its own shader cache files somewhere already.

[doitsujin]

The Vulkan pipeline cache should already handle this well enough, the only thing left to do is to actually store it somewhere (an obvious place would be $HOME/.cache on Linux). I think RADV already stores its own shader cache files somewhere already.

[SveSop]

Well.. not sure how things tie together, but with nVidia, there is a .nv folder that stores cache... and several games also have their own "cache".. So adding 1 more.. Dunno.

The nVidia implementation of shader cache already causes loads of stuttering when being created, but it is somewhat related to a low amount of storage (dont remember the numbers atm.. But running 4-5 games will cause the shaders to go out of space and be recreated, causing loads of stutters).

Can be worked around by giving each game their own cache, so that should be opted for if its implemented in DXVK i guess. Env. variable or some sort of separation of it to be sure its not overwritten or otherwise "run out of space" :)

[SveSop]

Well.. not sure how things tie together, but with nVidia, there is a .nv folder that stores cache... and several games also have their own "cache".. So adding 1 more.. Dunno.

The nVidia implementation of shader cache already causes loads of stuttering when being created, but it is somewhat related to a low amount of storage (dont remember the numbers atm.. But running 4-5 games will cause the shaders to go out of space and be recreated, causing loads of stutters).

Can be worked around by giving each game their own cache, so that should be opted for if its implemented in DXVK i guess. Env. variable or some sort of separation of it to be sure its not overwritten or otherwise "run out of space" :)

[pchome]

an obvious place would be $HOME/.cache on Linux

Actually $XDG_CACHE_HOME
https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

User may want (or already did) to redefine it. A very common practice to set it as /tmp/.cache for example which (/tmp) will (should) be cleared after reboot. Also /tmp is usually tmpfs - fast in-memory temporary filesystem.

[pchome]

an obvious place would be $HOME/.cache on Linux

Actually $XDG_CACHE_HOME
https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

User may want (or already did) to redefine it. A very common practice to set it as /tmp/.cache for example which (/tmp) will (should) be cleared after reboot. Also /tmp is usually tmpfs - fast in-memory temporary filesystem.

[SveSop]

User may want (or already did) to redefine it. A very common practice to set it as /tmp/.cache for example which (/tmp) will (should) be cleared after reboot. Also /tmp is usually tmpfs - fast in-memory temporary filesystem.

Well.. tbh using tmpfs defeats the purpose of having a cache "for-later-use". The point i THINK @roothorick was trying to achieve with this cache is to not have the driver recompile the shaders each time, and saving shaders to volatile memory would only be valid for that session of computer use. Unless ofc the computer is powered on 24/7...

[SveSop]

User may want (or already did) to redefine it. A very common practice to set it as /tmp/.cache for example which (/tmp) will (should) be cleared after reboot. Also /tmp is usually tmpfs - fast in-memory temporary filesystem.

Well.. tbh using tmpfs defeats the purpose of having a cache "for-later-use". The point i THINK @roothorick was trying to achieve with this cache is to not have the driver recompile the shaders each time, and saving shaders to volatile memory would only be valid for that session of computer use. Unless ofc the computer is powered on 24/7...

[roothorick]

The original idea was to prevent a game that compiles pipelines during gameplay from waiting on the pipeline to become available (resulting in a hitch) by having that pipeline already ready to go.

[roothorick]

The original idea was to prevent a game that compiles pipelines during gameplay from waiting on the pipeline to become available (resulting in a hitch) by having that pipeline already ready to go.

[pchome]

This shaders are not too big so why not to store them in memory? For a while at least.

Both *.dxbc and *.spv generated after Superposition test ~6 Mb and optimized ones ~4 Mb.
TW3's - 25 and 15 after spending some time in game.

So I think about some kind of in-memory-database where all this shaders can be stored and accessed by id (VS_ce8f5bfde13924bc08ecbbb615a50f7b72dce93e.spv or <expected_result_id> for example). Then this DB can be copied/updated to an physical storage and loaded back into memory when application/game launching. This will help with i/o and compilation overhead.

[pchome]

This shaders are not too big so why not to store them in memory? For a while at least.

Both *.dxbc and *.spv generated after Superposition test ~6 Mb and optimized ones ~4 Mb.
TW3's - 25 and 15 after spending some time in game.

So I think about some kind of in-memory-database where all this shaders can be stored and accessed by id (VS_ce8f5bfde13924bc08ecbbb615a50f7b72dce93e.spv or <expected_result_id> for example). Then this DB can be copied/updated to an physical storage and loaded back into memory when application/game launching. This will help with i/o and compilation overhead.

[doitsujin]

@pchome Applications generally perform the DXBC->SPIR-V translation only once for each shader, and usually when loading the game, so I don't see what that is supposed to help with. What @roothorick is referrig to is the stutter when Vulkan pipelines are created, which happens during rendering.

[doitsujin]

@pchome Applications generally perform the DXBC->SPIR-V translation only once for each shader, and usually when loading the game, so I don't see what that is supposed to help with. What @roothorick is referrig to is the stutter when Vulkan pipelines are created, which happens during rendering.

[pchome]

Applications generally perform the DXBC->SPIR-V translation only once for each shader

Sorry for bothering you with stupid questions/suggestions. But what about this debug: Compiling shader VS_... duplicates in log? I know nothing about shaders and how do they handled by dxvk and can suppose they been compiled with different parameters or so.

Here is some numbers #75 (comment)

For superposition :

     61 debug: Compiling shader VS_50dbfbd6521351958857579f020ac5470069ec8d
     32 debug: Compiling shader VS_a7280f2bf16459e952052dbc90c766657567dc37
     26 debug: Compiling shader VS_8b0d83969656d6bcfbd7b807009b297b5445542f
     23 debug: Compiling shader VS_c823713c5f7f2c28be2dda84cd3e2bf2799d49df
     15 debug: Compiling shader PS_9b7240e282984a82c41b2ccfde704b6acb60f283
     11 debug: Compiling shader VS_64546c77b9dd9a8727b2564851fa2874c8c6baef
     10 debug: Compiling shader PS_776858535c6ed5c29a428b6369be9cf9a992ac46

[pchome]

Applications generally perform the DXBC->SPIR-V translation only once for each shader

Sorry for bothering you with stupid questions/suggestions. But what about this debug: Compiling shader VS_... duplicates in log? I know nothing about shaders and how do they handled by dxvk and can suppose they been compiled with different parameters or so.

Here is some numbers #75 (comment)

For superposition :

     61 debug: Compiling shader VS_50dbfbd6521351958857579f020ac5470069ec8d
     32 debug: Compiling shader VS_a7280f2bf16459e952052dbc90c766657567dc37
     26 debug: Compiling shader VS_8b0d83969656d6bcfbd7b807009b297b5445542f
     23 debug: Compiling shader VS_c823713c5f7f2c28be2dda84cd3e2bf2799d49df
     15 debug: Compiling shader PS_9b7240e282984a82c41b2ccfde704b6acb60f283
     11 debug: Compiling shader VS_64546c77b9dd9a8727b2564851fa2874c8c6baef
     10 debug: Compiling shader PS_776858535c6ed5c29a428b6369be9cf9a992ac46

[SveSop]

But what about this debug: Compiling shader VS_... duplicates in log?

What do you mean by "duplicates". Those lines you pasted there are not duplicates.
VS_50dbfbd6521351958857579f020ac5470069ec8d is one shader, VS_c823713c5f7f2c28be2dda84cd3e2bf2799d49df is another one. A app/game/whatever will usually generate loads of them, and they are all "unique".

Sorry if i missunderstand your question tho :)

[SveSop]

But what about this debug: Compiling shader VS_... duplicates in log?

What do you mean by "duplicates". Those lines you pasted there are not duplicates.
VS_50dbfbd6521351958857579f020ac5470069ec8d is one shader, VS_c823713c5f7f2c28be2dda84cd3e2bf2799d49df is another one. A app/game/whatever will usually generate loads of them, and they are all "unique".

Sorry if i missunderstand your question tho :)

[pchome]

What do you mean by "duplicates".

61 times this string appeared in log file.

Command ussed:
$ grep "Compiling shader" dxvk_d3d11.log | sort | uniq -c | sort -nr -k1

[pchome]

What do you mean by "duplicates".

61 times this string appeared in log file.

Command ussed:
$ grep "Compiling shader" dxvk_d3d11.log | sort | uniq -c | sort -nr -k1

[volca02]

Isn't this stuff dependent on settings, though? I mean changing graphical settings would likely change a lot of the stuff in the pipelines?

[volca02]

Isn't this stuff dependent on settings, though? I mean changing graphical settings would likely change a lot of the stuff in the pipelines?

[Guy1524]

@volca02 Yes, but that is not relevant, if you were to change settings, the pipelines would be generated and stored for later use.

[Guy1524]

@volca02 Yes, but that is not relevant, if you were to change settings, the pipelines would be generated and stored for later use.

[jmis]

Path of Exile appears to compile shaders on the fly which makes the game unplayable until a cache is built up. Given the number of shaders required, it's not a good experience. The Wine DX9 to OpenGL translation layer allowed the "UseGLSL" option to disable the compilation.

[jmis]

Path of Exile appears to compile shaders on the fly which makes the game unplayable until a cache is built up. Given the number of shaders required, it's not a good experience. The Wine DX9 to OpenGL translation layer allowed the "UseGLSL" option to disable the compilation.

[dumpsters]

It's true, just a couple minutes of playing with DXVK_SHADER_DUMP_PATH on and there are over 2k files in the directory. One thing I'm curious about is if games that store their own shader cache have problems with DXVK or AMD drivers? for instance with PoE you have two shader cache folders in the game dir + AMD shader cache on windows, does that degrade DXVK performance in any way?

[dumpsters]

It's true, just a couple minutes of playing with DXVK_SHADER_DUMP_PATH on and there are over 2k files in the directory. One thing I'm curious about is if games that store their own shader cache have problems with DXVK or AMD drivers? for instance with PoE you have two shader cache folders in the game dir + AMD shader cache on windows, does that degrade DXVK performance in any way?

[doitsujin]

Commit 4518b1b stores the Vulkan pipeline to disk, which should reduce the stutter in subsequent runs of any application, at least on drivers that do not do this automatically already (such as RADV).

[doitsujin]

Commit 4518b1b stores the Vulkan pipeline to disk, which should reduce the stutter in subsequent runs of any application, at least on drivers that do not do this automatically already (such as RADV).