Implement Vulkan pipeline caching #76348

warriormaster12 · 2023-04-22T16:10:49Z

An implementation of Vulkan pipeline caching with cache validation when reading the file. Looking for feedback before merging this pr :)

Bugsquad edit: Follow-up to Support for vk graphics pipeline library extension #72682 (this is a different feature, both can be used).

warriormaster12 · 2023-04-24T17:31:50Z

@clayjohn @RandomShaper I asked around about driverABI's purpose and it seems that it is used to check if the os is 32-bit or 64-bit. There aren't any plans to support 32-bits right? If so then I can remove the variable from PipelineCacheHeader.

warriormaster12 · 2023-04-24T17:37:00Z

I've also added a pipeline cache save interval into project settings based on @reduz's one of the suggestions.

clayjohn · 2023-04-24T17:48:59Z

Could you test this PR with a larger project like the TPS-demo and check what the size of the pipeline cache ends up being? I know reduz has stated earlier that he only wants to cache certain pipelines to file size and load time from the cache. I think he specifically wanted to only cache the main specialization constant variants as ideally all the other variants should be compiled on a background thread

warriormaster12 · 2023-04-24T18:00:04Z

Could you test this PR with a larger project like the TPS-demo and check what the size of the pipeline cache ends up being? I know reduz has stated earlier that he only wants to cache certain pipelines to file size and load time from the cache. I think he specifically wanted to only cache the main specialization constant variants as ideally all the other variants should be compiled on a background thread

I wish I could but at the moment at least I'm having a hard time to get the project to open since it gets stuck on importing level.exr

edit: deleting the file fixed the issue.

clayjohn · 2023-04-24T18:08:13Z

I wish I could but at the moment at least I'm having a hard time to get the project to open since it gets stuck on importing level.exr

Ah, ya compressing exr files is super flow in debug builds. If you run it with debug builds you should open up level.exr.import and change the compress/mode to 3. That way it will import as VRAM uncompressed and will be much faster

warriormaster12 · 2023-04-24T18:29:34Z

@clayjohn after doing a quick test with TPS Demo and Gdquest's third person platforming demo. The results are:
TPS Demo: 6.3 mb
GDquest TP: 6.6 mb

And I assume that these graphical artifact are due to me using VRAM uncompressed with level.exr?

warriormaster12 · 2023-05-27T16:45:35Z

One moment

warriormaster12 · 2023-05-27T16:48:39Z

Done

Ansraer · 2023-05-27T21:38:30Z

So I took some time and built this PR (rebased on the current master) locally. The code looks good to me and some quick debugging would imply that Vulkan is correctly loading stuff from the cache (instead of compiling it).

Creating a pipeline without the cache takes on average 30.000 usec. This PR cuts that down to ~260 usec, while my driver's built-in cache averages 80.
If we are really concerned about the cache size it might make more sense to disable this on modern desktop GPUs (they all have a built-in cache) and only enable it on mobile devices out of the box.

warriormaster12 · 2023-05-28T06:45:06Z

@Ansraer I disagree, there are cases in desktop as well where pipeline cache would help. In TPS Demo, shooting a bullet causes a stutter in multiple times when you boot the game before the driver builds up a cache. With pipeline cache, it only happens on a first run.

bitsawer

I'm a bit late to the party, but I made a few comments about the code. Feel free to ignore them if there's a reason behind the issues I pointed out.

bitsawer · 2023-05-28T18:50:24Z

drivers/vulkan/rendering_device_vulkan.cpp

+	float save_interval = GLOBAL_GET("rendering/rendering_device/pipeline_cache/save_interval_mb");
+	VkResult vr = vkGetPipelineCacheData(device, pipelines_cache.cache_object, &pso_blob_size, nullptr);
+	ERR_FAIL_COND(vr);
+	size_t difference = (pso_blob_size - pipelines_cache.current_size) / (1024.0f * 1024.0f);


Should probably use integer division here.

bitsawer · 2023-05-28T18:52:05Z

drivers/vulkan/rendering_device_vulkan.cpp

+				break;
+			}
+		}
+		if (header.data_hash != hash_murmur3_one_64(pipelines_cache.buffer.size()) || header.data_size != (uint32_t)pipelines_cache.buffer.size() || header.vendor_id != props.vendorID || header.device_id != props.deviceID || header.driver_abi != sizeof(void *) || invalid_uuid) {


Should this use hash_murmur3_buffer() instead to hash the full contents of pipelines_cache.buffer? Now we are just hashing the size integer which doesn't seem all that useful as we could just store the integer directly.

Of course, hashing the full buffer and megabytes of data might cause a small slowdown, but hopefully it shouldn't be too bad.

I might have interoperated "robust pipeline cache" article incorrectly so this could indeed be the correct way to hash.

bitsawer · 2023-05-28T18:54:32Z

drivers/vulkan/rendering_device_vulkan.cpp

+	}
+
+	if (FileAccess::exists("user://vulkan/pipelines.cache")) {
+		Vector<uint8_t> file_data = FileAccess::get_file_as_bytes("user://vulkan/pipelines.cache");


This part could use some basic error and data bounds checking just in case, for example if the file exists but is corrupted, truncated or could not be read this will crash soon afterwards.

bitsawer · 2023-05-28T18:56:16Z

drivers/vulkan/rendering_device_vulkan.cpp

+	PipelineCacheHeader header = {};
+	header.magic = 868 + VK_PIPELINE_CACHE_HEADER_VERSION_ONE;
+	header.data_size = pipelines_cache.buffer.size();
+	header.data_hash = hash_murmur3_one_64(pipelines_cache.buffer.size());


Same as above, should this use hash_murmur3_buffer()?

warriormaster12 · 2023-05-28T19:40:20Z

@bitsawer thanks in general for good feedback :)

I'll look at it tomorrow asap

bitsawer · 2023-05-28T19:40:49Z

drivers/vulkan/rendering_device_vulkan.cpp

@@ -8957,6 +8966,102 @@ void RenderingDeviceVulkan::initialize(VulkanContext *p_context, bool p_local_de
 	draw_list_split = false;

 	compute_list = nullptr;
+	_load_pipeline_cache();
+	print_line(vformat("Startup PSO cache (%.1f MiB)", pipelines_cache.buffer.size() / (1024.0f * 1024.0f)));


Might be better to use print_verbose instead of print_line for all logging like this (including the one in _update_pipeline_cache).

warriormaster12 · 2023-05-29T19:05:23Z

@bitsawer @clayjohn @RandomShaper file size has shrunk a bit after changing to hash_buffer so that's possitive.

Let me know if the file error checks that I have added are good enough.

bitsawer · 2023-05-29T20:08:19Z

Changes look mostly good. However, I would write the file check something like this (not tested or compiled):

Error file_error;
Vector<uint8_t> file_data = FileAccess::get_file_as_bytes("user://vulkan/pipelines.cache", &file_error);
if (file_error != OK || file_data.size() <= sizeof(PipelineCacheHeader)) {
    WARN_PRINT("Invalid/corrupt pipelines cache.");
    return;
}
PipelineCacheHeader header = {};
...

Instead of checking multiple possible error codes, just check if the read was not OK. This also checks if we actually read enough data so that the following memcpy() and header check doesn't blow up. For example, reading an empty or partially written file would currently crash because reading would be successful but it would not read enough data.

As a super minor nitpick I didn't notice earlier, you could also tweak the comment formatting like this (space after // and captialized):

// This is mostly for the editor to check if after playing the game, game's pipeline cache size still matches with editor's cache.

After changing those, looks good to me.

warriormaster12 · 2023-05-30T17:44:22Z

@bitsawer Done, hopefully this is good to go for merging

bitsawer

Looks good to me after the changes.

YuriSizov · 2023-05-31T08:19:28Z

@clayjohn @RandomShaper Please make sure you're okay with the changes, when you have time :)

RandomShaper · 2023-05-31T11:04:45Z

I'm finally reviewing this! Sorry for not doing it earlier.

Looks very well!

Now, I'm wondering if it wouldn't be nice to save the cache from another thread; namely, a low priority task in the WorkerThreadPool.

RandomShaper · 2023-05-31T11:05:10Z

doc/classes/ProjectSettings.xml

@@ -2305,6 +2305,9 @@
 		<member name="rendering/rendering_device/driver.windows" type="String" setter="" getter="" default="&quot;vulkan&quot;">
 			Windows override for [member rendering/rendering_device/driver].
 		</member>
+		<member name="rendering/rendering_device/pipeline_cache/save_interval_mb" type="float" setter="" getter="" default="3.0">


This wording makes it sound as if this was about time instead of size. Let me suggest save_increment_mb, for instance.

One of the meanings of interval is a gap between two points. The setting describes a gap between last time we saved and next time. It could be maybe save_gap_mb or save_delta_mb if not interval?

Could be save_chunk_size_mb? Or even just chunk_size_mb. (I didn't check implementation details to see what this is used for exactly so TIWAGOS.)

warriormaster12 · 2023-05-31T11:32:43Z

I'm finally reviewing this! Sorry for not doing it earlier.

Looks very well!

Now, I'm wondering if it wouldn't be nice to save the cache from another thread; namely, a low priority task in the WorkerThreadPool.

I could implement one quickly. I'm not experienced though with multi-threading so I'd like to know, should I call wait_for_ function right after creating task/group id?

RandomShaper · 2023-05-31T12:52:25Z

I could implement one quickly. I'm not experienced though with multi-threading so I'd like to know, should I call wait_for_ function right after creating task/group id?

Nope, because by doing so you would be making the operation synchronous (locking, stalling) regardless the use of threads. You should make the call to add the task and just remember the task id. Then, when it's time to save the cache again, if the task is still in flight, skip. Upon closing the engine, you would wait for any in flight cache save task to complete and then call it normally to ensure the latest state is saved. Does that make sense to you?

warriormaster12 · 2023-05-31T13:37:21Z

It does yeah

warriormaster12 · 2023-05-31T19:21:45Z

@RandomShaper added work threading and updated project setting name.

akien-mga · 2023-05-31T22:47:16Z

Thanks!

warriormaster12 requested a review from a team as a code owner April 22, 2023 16:10

warriormaster12 changed the title ~~Vulkan, implemented pipeline-caching~~ Vulkan, implemented pipeline caching Apr 22, 2023

clayjohn added enhancement topic:rendering labels Apr 22, 2023

clayjohn added this to the 4.1 milestone Apr 22, 2023

clayjohn requested review from RandomShaper and clayjohn April 22, 2023 16:25

warriormaster12 force-pushed the pipeline-cache branch 13 times, most recently from e70f83c to 195bbe8 Compare April 22, 2023 19:14

warriormaster12 force-pushed the pipeline-cache branch from 195bbe8 to 1ae1004 Compare April 24, 2023 17:37

warriormaster12 requested a review from a team as a code owner April 24, 2023 17:37

warriormaster12 force-pushed the pipeline-cache branch from 1ae1004 to 6f14b27 Compare April 24, 2023 17:44

warriormaster12 force-pushed the pipeline-cache branch from 6f14b27 to 486b033 Compare April 24, 2023 17:49

YuriSizov changed the title ~~Vulkan, implemented pipeline caching~~ Implement Vulkan pipeline caching May 27, 2023

warriormaster12 force-pushed the pipeline-cache branch from 1a60b33 to 4483576 Compare May 27, 2023 16:48

bitsawer reviewed May 28, 2023

View reviewed changes

warriormaster12 force-pushed the pipeline-cache branch from 4483576 to c7edfc0 Compare May 29, 2023 19:03

warriormaster12 force-pushed the pipeline-cache branch from c7edfc0 to 2d3d92c Compare May 30, 2023 17:42

bitsawer approved these changes May 30, 2023

View reviewed changes

RandomShaper reviewed May 31, 2023

View reviewed changes

warriormaster12 force-pushed the pipeline-cache branch 2 times, most recently from 08d73a4 to 21bfd7d Compare May 31, 2023 19:21

Implement Vulkan pipeline caching

dded713

warriormaster12 force-pushed the pipeline-cache branch from 21bfd7d to dded713 Compare May 31, 2023 19:24

akien-mga merged commit 3dd0307 into godotengine:master May 31, 2023
13 checks passed

bitsawer mentioned this pull request Jun 1, 2023

LightmapGI : Cannot reimport lightmaps' EXR files after baked lightmaps #77746

Closed

RedworkDE mentioned this pull request Jul 1, 2023

Occasional attempting free on address which was not malloc()-ed crash in CI #78749

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Vulkan pipeline caching #76348

Implement Vulkan pipeline caching #76348

warriormaster12 commented Apr 22, 2023 •

edited by Calinou

warriormaster12 commented Apr 24, 2023

warriormaster12 commented Apr 24, 2023

clayjohn commented Apr 24, 2023

warriormaster12 commented Apr 24, 2023 •

edited

clayjohn commented Apr 24, 2023

warriormaster12 commented Apr 24, 2023 •

edited

warriormaster12 commented May 27, 2023

warriormaster12 commented May 27, 2023

Ansraer commented May 27, 2023

warriormaster12 commented May 28, 2023

bitsawer left a comment

bitsawer May 28, 2023

bitsawer May 28, 2023 •

edited

warriormaster12 May 28, 2023

bitsawer May 28, 2023

bitsawer May 28, 2023

warriormaster12 commented May 28, 2023

bitsawer May 28, 2023 •

edited

warriormaster12 commented May 29, 2023

bitsawer commented May 29, 2023

warriormaster12 commented May 30, 2023

bitsawer left a comment

YuriSizov commented May 31, 2023

RandomShaper commented May 31, 2023

RandomShaper May 31, 2023

warriormaster12 May 31, 2023

akien-mga May 31, 2023 •

edited

warriormaster12 commented May 31, 2023

RandomShaper commented May 31, 2023

warriormaster12 commented May 31, 2023

warriormaster12 commented May 31, 2023

akien-mga commented May 31, 2023

Implement Vulkan pipeline caching #76348

Implement Vulkan pipeline caching #76348

Conversation

warriormaster12 commented Apr 22, 2023 • edited by Calinou

warriormaster12 commented Apr 24, 2023

warriormaster12 commented Apr 24, 2023

clayjohn commented Apr 24, 2023

warriormaster12 commented Apr 24, 2023 • edited

clayjohn commented Apr 24, 2023

warriormaster12 commented Apr 24, 2023 • edited

warriormaster12 commented May 27, 2023

warriormaster12 commented May 27, 2023

Ansraer commented May 27, 2023

warriormaster12 commented May 28, 2023

bitsawer left a comment

Choose a reason for hiding this comment

bitsawer May 28, 2023

Choose a reason for hiding this comment

bitsawer May 28, 2023 • edited

Choose a reason for hiding this comment

warriormaster12 May 28, 2023

Choose a reason for hiding this comment

bitsawer May 28, 2023

Choose a reason for hiding this comment

bitsawer May 28, 2023

Choose a reason for hiding this comment

warriormaster12 commented May 28, 2023

bitsawer May 28, 2023 • edited

Choose a reason for hiding this comment

warriormaster12 commented May 29, 2023

bitsawer commented May 29, 2023

warriormaster12 commented May 30, 2023

bitsawer left a comment

Choose a reason for hiding this comment

YuriSizov commented May 31, 2023

RandomShaper commented May 31, 2023

RandomShaper May 31, 2023

Choose a reason for hiding this comment

warriormaster12 May 31, 2023

Choose a reason for hiding this comment

akien-mga May 31, 2023 • edited

Choose a reason for hiding this comment

warriormaster12 commented May 31, 2023

RandomShaper commented May 31, 2023

warriormaster12 commented May 31, 2023

warriormaster12 commented May 31, 2023

akien-mga commented May 31, 2023

warriormaster12 commented Apr 22, 2023 •

edited by Calinou

warriormaster12 commented Apr 24, 2023 •

edited

warriormaster12 commented Apr 24, 2023 •

edited

bitsawer May 28, 2023 •

edited

bitsawer May 28, 2023 •

edited

akien-mga May 31, 2023 •

edited