Scene testruns with threading/worker_pool/max_threads set to 0 get stuck after recent WorkerThreadPool changes

[smix8]

Godot version

508a5bf

System information

Windows 10, Intel i9-11900K

Issue description

Godot runs fine in the Editor but any test run of a (3D) scene freezes the window up.
Even a test scene with just a single Node3D freezes.

All the test project runs fine with slightly older versions of master / Godot 4.1.

There are no errors but after bisecting it the problem points at changes from pr #76945 so properly some kind of thread deadlock.

edit

Seems to be caused by ProjectSettings threading/worker_pool/max_threads set to 0 and shader_compiler/shader_cache/enabled=false

Steps to reproduce

set ProjectSettings threading/worker_pool/max_threads to 0 and shader_compiler/shader_cache/enabled=false and testrun a scene from the Editor.

Minimal reproduction project

WorkerThreadPoolIssue.zip

[RandomShaper]

Despite how easy it seems to be possible to reproduce it, please share an MRP and commit hash so I can be very sure I'm debugging the exact situation.

[smix8]

I think I found the real cause of the issue.

For some reason in all existing test projects that I tried the ProjectSettings had the threading/worker_pool/max_threads set to 0. Returning it to the default -1 fixed the issue. I did not notice it before cause the AdvancedSettings was still toggled off.

[RandomShaper]

I had a hunch about project settings. Before my PR WorkerThreadPool settings were being ignored.

[smix8]

Does it still make sense to allow this property to be set to 0 now that even the servers are switching to an internal use of the WorkerThreadPool?

[smix8]

Added MRP.
The unhealthy combination of WorkerThreadPool set to zero with shader cache disabled were the ProjectSettings that made the testruns stuck.

[RandomShaper]

Does it still make sense to allow this property to be set to 0 now that even the servers are switching to an internal use of the WorkerThreadPool?

Upcoming PRs are going to fix that. At least I know of one of mine and one of @myaaaaaaaaa.

[akien-mga]

I can confirm the issue using the MRP in the latest master branch, on Linux.

Should we prevent setting max_threads to 0 until this is fixed?

[myaaaaaaaaa]

Should we prevent setting max_threads to 0 until this is fixed?

See the below pseudocode for a suggestion on how to accomplish this

diff --git a/core/object/worker_thread_pool.cpp b/core/object/worker_thread_pool.cpp
index d285be3e70..660a624a43 100644
--- a/core/object/worker_thread_pool.cpp
+++ b/core/object/worker_thread_pool.cpp
@@ -528,18 +528,21 @@ void WorkerThreadPool::wait_for_group_task_completion(GroupID p_group) {
 	}
 
 	task_mutex.lock(); // This mutex is needed when Physics 2D and/or 3D is selected to run on a separate thread.
 	groups.erase(p_group);
 	task_mutex.unlock();
 }
 
 void WorkerThreadPool::init(int p_thread_count, bool p_use_native_threads_low_priority, float p_low_priority_task_ratio) {
 	ERR_FAIL_COND(threads.size() > 0);
+	if (p_thread_count == 0) {
+		p_thread_count = 1;
+	}
 	if (p_thread_count < 0) {
 		p_thread_count = OS::get_singleton()->get_default_thread_pool_size();
 	}
 
 	if (p_use_native_threads_low_priority) {
 		max_low_priority_threads = 0;
 	} else {
 		max_low_priority_threads = CLAMP(p_thread_count * p_low_priority_task_ratio, 1, p_thread_count - 1);
 	}