[Feature Request] Set Nvidia and Mesa Shader Disk Cache by default, per title

[Leopard1907]

Is your feature request related to a problem? Please describe.
By default Nvidia on disk shader cache has a limit and when it is exceed it causes previous caches to get purged. It is a problem due to purging of them leads to stutters over and over again.

Describe the solution you'd like

Either a solution like Lutris does with many installers like this, which sets GL Shader Disk Cache to game directory.

https://lutris.net/games/install/5835/view

Or, like how Heroic has seperate folders for prefixes, game installations; a seperate folder called NV shader cache or something similar which has sub directories per game title.

Both would be beneficial, second option would have the advantage of keeping them more organized but might lead to severe disk space consumption if user never deletes them.

First one seems to be a better option because on every driver update those caches gets invalidated and needs to repopulate. So a user deleting a game and coming back to it months later and still having intact cache is highly unlikely. So Lutris way seems to make more sense.

Relevant NV documentation btw:

http://us.download.nvidia.com/XFree86/Linux-x86_64/510.47.03/README/openglenvvariables.html

Despite it is called GL cache it is not OpenGL only.

Link to message about Mesa driver shader cache management:

#973 (comment)

[flavioislima]

@Leopard1907
Do we still need this? I think the latest drivers allow Nvidia to manage the cache automatically, right?

[Leopard1907]

@flavioislima Still needed, Nvidia always managed cache automatically. Problem is this with this behaviour and limitations of it being combined.

By default shaders gets stored in either one of these locations per Nvidia readme:

If __GL_SHADER_DISK_CACHE_PATH is unset, caches will be stored in $XDG_CACHE_HOME/nvidia/GLCache/ if XDG_CACHE_HOME is set, or in $HOME/.cache/nvidia/GLCache/ if HOME is set. If none of the environment variables __GL_SHADER_DISK_CACHE_PATH, XDG_CACHE_HOME, or HOME is set, the shader cache will be disabled. Caches are persistent across runs of an application. Cached shader binaries are specific to each driver version; changing driver versions will cause binaries to be recompiled.

Note that earlier driver versions kept the default caches in $XDG_CACHE_HOME/.nv/GLCache/ or $HOME/.nv/GLCache/ instead of $XDG_CACHE_HOME/nvidia/GLCache/ or $HOME/.cache/nvidia/GLCache/, respectively. If these earlier locations exist they will be reused instead. It is safe to remove any .nv/GLCache/ directory, after which the driver will populate a fresh cache in the new locations as described in the previous paragraph.

http://us.download.nvidia.com/XFree86/Linux-x86_64/515.76/README/openglenvvariables.html

These default locations doesn't have an unlimited size, rather it has a maximum storage threshold. So when that location hits the limit, it starts to erase caches to make room for new ones, them still being valid (belongs to current driver ) is not a concern of this cache.

An earlier example of how this behaviour leads to bad experience on games with rather high shader count:

https://forums.developer.nvidia.com/t/455-23-04-driver-produces-very-large-shader-caches-results-in-constant-cache-invalidation/156087

I’ve recently noticed Vulkan Developer driver 450.56.02, when compiling shaders for at least Vulkan apps running through Wine compat layer with DXVK, seems to generate GLCaches that are far larger than the GLCache shader caches generated by mainline drivers, and that they get invalidated all the time even though the driver version stays the same.

For instance, in Overwatch, installed through Lutris application with a pre-filled DXVK state cache with roughly 39 thousands of valid shader pipeline entries, launched with __GL_SHADER_DISK_CACHE_PATH environment variable to separate its shader cache location, the size of GLCache under Nvidia Driver 450.57 is 86.6 MB, and it persists just fine through multiple launches of the app.

However, under 450.56.02 and even 450.56.06 driver, the size of GLCache under the same conditions is almost 10 times that - 833 MB. And that shader cache does not persist - it gets invalidated by the driver with every launch of the app and recompiled again, which I find unacceptable as the compilation of so many shaders takes a lot of time, during which the game isn’t playable.

However, if I increase the size at which Nvidia driver invalidates its shader cache with __GL_SHADER_DISK_CACHE_SIZE environment variable by setting it to a value of 1000000000, shader cache doesn’t get invalidated anymore, although its size of course still stays rather large.

As can be seen in those quotes setting __GL_SHADER_DISK_CACHE_SIZE to an absurdly high amount ( doesn't have any bad side effects like consuming drive space etc, that is just an arbitrary threshold) leads to basically eliminating possibility of still valid shader caches being removed like how it will be with the default behaviour.

Setting __GL_SHADER_DISK_CACHE_PATH also and var above with a high enough threshold will lead to being organized and free from con of default limit/ threshold.

Games like God Of War can reach to nearly 1 gb compiled shaders, games like Warframe, Horizon Zero Dawn can reach even more. With everything default they will end up hurting each other because they will fight over space in $HOME/.cache/nvidia/GLCache/.

Setting that shader cache threshold and location will become much more important very soon due to new extension usage on DXVK that will come up with next release ( available on DXVK master as of now, usable with NV Vulkan dev driver ) , VK_EXT_graphics_pipeline_library.

https://github.com/doitsujin/dxvk/wiki/Driver-support#dxvk-next-release

Shaders will be compiled way earlier and mostly fully. So that will just baloon the shader caches even with a game you've just started and sit on the main menu and then quit.

In this case shader caches from each game will be much more likely to trigger that default limit behaviour and start invalidating caches.

Apps like Steam doesn't trigger this behaviour because shader caches for each game gets stored in /.local/share/Steam/steamapps/shadercache with their appid's.

[Leopard1907]

Updating issue to be more general that will benefit all user space drivers ( Nvidia prop, RADV,RadeonSI,ANV,Iris, any upcoming Mesa driver)

https://docs.mesa3d.org/envvars.html

MESA_SHADER_CACHE_MAX_SIZE and MESA_SHADER_CACHE_DIR are equivalents of what was being explained above for Nvidia userspace driver.