Add `THREADS_ENABLED` macro in order to compile Godot to run on the main thread #85939

adamscott · 2023-12-08T21:35:41Z

Context

This PR adds the THREADS_ENABLED macro and the ~~use_threads=yes|no~~ threads=yes|no build option.

There could be a way to make the THREADS_ENABLED modifications run time instead of build time, but the Web platform would still need two separate build targets, one with threads, the other without, due to emscripten limitations.

Has the good side-effect to make web builds compatible with macOS / iOS.

Demo

https://adamscott.github.io/truck-town-demo-main-thread/
https://adamscott.github.io/truck-town-demo-main-thread-nosound/ (runs without sound, in order to test performance)

Progress

Threads

Make a WorkerThreadPool that executes single threaded.

File system

Fix issue where file system don't seem to be updated

Audio

Port the Audio Driver from Godot 3
Fix broken driver?!
- Now only works on HTTPS (AudioWorklet is only available for HTTPS contexts), I should maybe investigate if it's possible to make it work on HTTP.
  - Should be another issue/PR

Rendering

RenderingServer should be forced to run single threaded.

Physics

PhysicsServer2D/3D should be forced to run single threaded.
Fix broken physics

Core

Add a variable that say if threads are supported or not
- Added OS.are_thread_enabled() function

Export

Add web_compatible export

Github

Add CI for use_threads=no

Misc

Make the IP Resolver work single threaded.
Make RaycastOcclusionCull work single threaded.

Fixes

Fixes #85938

Known issues

When running in other platforms than the web, sound is broken.

Edit

2024-01-12

Updated use_threads=yes|no to threads=yes|no.
Added non-web sound known issue

Production edit: closes godotengine/godot-roadmap#11

akien-mga

Amazing work!

I haven't tested yet, only did a quick code quality review.

core/os/mutex.h

core/os/thread.h

drivers/unix/thread_posix.cpp

doc/classes/OS.xml

platform/web/export/export_plugin.h

servers/rendering/rendering_server_default.h

core/os/condition_variable.h

core/os/mutex.h

michaldev · 2023-12-11T11:08:28Z

https://adamscott.github.io/truck-town-demo-main-thread/

It works on iOS (17.1.2).

core/object/worker_thread_pool.cpp

core/os/semaphore.h

jordo · 2023-12-11T16:26:04Z

Nice work! Since this adds a THREADS_ENABLED macro, should probably add a use_threads=no CI for this web target? Feels like this target will need CI to catch commits which don't guard future thread dependencies and have a single threaded impl.

adamscott · 2023-12-11T22:08:27Z

Nice work! Since this adds a THREADS_ENABLED macro, should probably add a use_threads=no CI for this web target? Feels like this target will need CI to catch commits which don't guard future thread dependencies and have a single threaded impl.

In fact, it should have CI for both. I'll try to add them to the github-actions.

RandomShaper

I'll do another pass later, once the existing comments have been addressed. As a general advice, I'd make all the no-threads versions of everything as slim as possible (commentless, without inline macros, etc.).

core/os/mutex.h

core/os/condition_variable.h

core/io/ip.cpp

core/os/semaphore.h

servers/rendering/rendering_server_default.h

wojtekmal · 2024-01-16T18:55:17Z

Uncaught ReferenceError: $GODOT_THREADS_ENABLED is not defined
    <anonymous> http://localhost:8060/tmp_js_export.html:141

I'm having the same issue, but on Crazy Games, Game Jolt and when running via the serve.py script from the godot repository. I haven't run this with the one click deploy yet, but this seems to be ubiquitous for me. I checked out to this PR and built the export template today, so maybe this is caused by some recent change.
Edit: I used the template that popcar provided and it works with a bunch of errors, such as
Output buffer has not enough frames! Skipping output frame.
and
Invalid mix rate of 0, consider reassigning setting 'audio/driver/mix_rate'.

Calinou · 2024-01-16T22:20:04Z

Invalid mix rate of 0, consider reassigning setting 'audio/driver/mix_rate'.

You can ignore this message as a mix rate of 0 is valid on the web platform (it lets the browser choose the most suitable mix rate automatically).

adamscott · 2024-01-17T13:52:58Z

Invalid mix rate of 0, consider reassigning setting 'audio/driver/mix_rate'.

You can ignore this message as a mix rate of 0 is valid on the web platform (it lets the browser choose the most suitable mix rate automatically).

I could mask this error from WEB_ENABLED, though.

Edit: Weird, it's already masked for web.
Edit 2: Oh, @wojtekmal used @popcar2 provided template.

adamscott · 2024-01-17T14:06:52Z

@adamscott There's an issue with One-Click Deploy:
Uncaught ReferenceError: $GODOT_THREADS_ENABLED is not defined
    <anonymous> http://localhost:8060/tmp_js_export.html:141
To test, add a "runnable" export preset and configure the debug template to point to one made from this PR.

AH. I understand now why it was working for me. I was only testing with release templates, and the "run in browser" build was using an old (working) debug template.

adamscott · 2024-01-17T14:21:32Z

@akien-mga I'm not able to replicate your issue, even with the debug template build up-to-date.

I tried erasing non-threads templates, setting the templates in the export settings. But clicking One-click deploy for me works without hiccups for me, even on Firefox or Chrome Incognito mode.

akien-mga · 2024-01-17T16:45:39Z

@akien-mga I'm not able to replicate your issue, even with the debug template build up-to-date.

I tried erasing non-threads templates, setting the templates in the export settings. But clicking One-click deploy for me works without hiccups for me, even on Firefox or Chrome Incognito mode.

I tested the current commit (7c4cb14) and it seems to be working fine for me.

I tested:

Custom built Godot editor on Linux
Custom built debug template (scons p=web target=template_debug)
Release template from the GitHub Actions used as Custom debug template

Both my custom built and the GH Actions artifact seem to be working fine.

akien-mga

I think this is good to go!

akien-mga · 2024-01-17T16:50:56Z

Wait I spoke too fast again...

The template from GHA still doesn't work for me. It still tries to enable SharedArrayBuffer.

And my locally built templates lacked threads=no so they're not a good test :P I'll rebuild.

akien-mga · 2024-01-17T17:14:23Z

Yep, local build with scons p=web target=template_debug verbose=yes scu_build=yes threads=no, I still have this error:

So this PR is not working yet for me.

For the record, the One-click deploy mode isn't a good way to test the no-threads builds, as the server we spawn has the required headers for SharedArrayBuffer.

So to be extremely clear, here's what I'm testing:

Compile Linux editor from 7c4cb14 with scons p=linuxbsd dev_build=yes dev_mode=yes linker=mold scu_build=all (dev_*, linker, scu_build shouldn't matter here, but that's just what I use for faster builds)
Compile Web debug template without thread support with scons p=web target=template_debug verbose=yes scu_build=yes threads=no
Edit https://github.com/HarmonyHoney/CandyWrapper with the editor build I made
Add Web export template, configure the Custom debug template path to ~/godot/folder/bin/godot.web.template_debug.wasm32.nothreads.zip
Disable the "Thread Support" checkbox

"Export Project...", "Export With Debug"
In the export folder, python -m http.server
Browse localhost:8000

Same issue if I use the export template artifact from GitHub Actions instead of my custom build.

…n thread

adamscott · 2024-01-17T19:03:35Z

@akien-mga @Smooth-E @popcar2 @wojtekmal I found and fixed the bug. (in platform/web/export/export_plugin.cpp:173)

#ifdef THREADS_ENABLED
	replaces["$GODOT_THREADS_ENABLED"] = "true";
#else
	replaces["$GODOT_THREADS_ENABLED"] = "false";
#endif

That code was setting the requirements for threads as true if the editor itself had threads support. false otherwise. That bug explains why the template from my Github actions weren't working.

I replaced that code with the actual value checked in the editor:

	if (p_preset->get("variant/thread_support")) {
		replaces["$GODOT_THREADS_ENABLED"] = "true";
	} else {
		replaces["$GODOT_THREADS_ENABLED"] = "false";
	}

akien-mga · 2024-01-18T07:46:56Z

Great, that would explain why it worked for me last week when I tested with a Linux editor build with threads disabled, just for fun.

I'll retest today and merge.

akien-mga

Tested successfully! And I don't spot any other misplaced TOOLS_ENABLED check where it's meant to check the target config, so we should be good to go finally.

Changes done.

akien-mga · 2024-01-18T08:36:45Z

Thanks! Amazing work 🎉 🥇

jordo · 2024-02-08T00:56:43Z

nice work! paving the way for our team to eventually move to 4.X! Thanks @adamscott

geekley · 2024-06-04T18:34:51Z

Hi, I came from this post, which said to report errors here.

If you encounter any issue while testing my PR or while testing out the single-threaded samples builds, please report them here.

I tried the catburglar-main-thread-samples on Firefox. It worked fine until completing the (1st?) level, when it stopped responding. That's after getting at least $300 and returning to the entrance.
The browser console logged this error when the game tried to exit that scene/stage after completing the goal:

Uncaught TypeError: AudioNode.connect: Argument 1 is not valid for any of the 2-argument overloads.
    pause https://adamscott.github.io/catburglar-main-thread-samples/index.js:9944
    pauseSampleNode https://adamscott.github.io/catburglar-main-thread-samples/index.js:9814
    sample_set_pause https://adamscott.github.io/catburglar-main-thread-samples/index.js:10380
    _godot_audio_sample_set_pause https://adamscott.github.io/catburglar-main-thread-samples/index.js:10559
    callUserCallback https://adamscott.github.io/catburglar-main-thread-samples/index.js:6639
    runIter https://adamscott.github.io/catburglar-main-thread-samples/index.js:6714
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6600
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
    requestAnimationFrame https://adamscott.github.io/catburglar-main-thread-samples/index.js:7009
    Browser_mainLoop_scheduler_rAF https://adamscott.github.io/catburglar-main-thread-samples/index.js:5893
    Browser_mainLoop_runner https://adamscott.github.io/catburglar-main-thread-samples/index.js:6613
index.js:9944:19

AThousandShips added topic:core regression labels Dec 9, 2023

AThousandShips added this to the 4.3 milestone Dec 9, 2023

adamscott force-pushed the single-threaded-godot-4 branch 3 times, most recently from ebcd020 to b0ba444 Compare December 10, 2023 19:05

adamscott changed the title ~~Add USE_THREADS macro in order to compile Godot to run on the main thread~~ Add THREADS_ENABLED macro in order to compile Godot to run on the main thread Dec 10, 2023

akien-mga added enhancement platform:web topic:porting labels Dec 11, 2023

akien-mga reviewed Dec 11, 2023

View reviewed changes

akien-mga requested review from Faless, RandomShaper and a team December 11, 2023 11:32

adamscott marked this pull request as ready for review December 11, 2023 11:55

adamscott requested review from a team as code owners December 11, 2023 11:55

AThousandShips reviewed Dec 11, 2023

View reviewed changes

core/object/worker_thread_pool.cpp Outdated Show resolved Hide resolved

adamscott commented Dec 11, 2023

View reviewed changes

core/os/semaphore.h Show resolved Hide resolved

adamscott force-pushed the single-threaded-godot-4 branch from 54d1ae2 to 3ad3673 Compare December 11, 2023 15:14

RandomShaper previously requested changes Dec 12, 2023

View reviewed changes

core/os/mutex.h Show resolved Hide resolved

core/os/condition_variable.h Show resolved Hide resolved

core/io/ip.cpp Outdated Show resolved Hide resolved

core/os/semaphore.h Show resolved Hide resolved

servers/rendering/rendering_server_default.h Outdated Show resolved Hide resolved

adamscott force-pushed the single-threaded-godot-4 branch from 31364e7 to 0931a47 Compare December 12, 2023 15:21

adamscott mentioned this pull request Dec 12, 2023

HTML5 export for Godot 4.x takes 1-2 minutes to load on macOS #70691

Open

adamscott force-pushed the single-threaded-godot-4 branch from 26225b4 to 7c4cb14 Compare January 17, 2024 13:54

akien-mga approved these changes Jan 17, 2024

View reviewed changes

Add THREADS_ENABLED macro in order to compile Godot to run on the mai…

bd70b8e

…n thread

adamscott force-pushed the single-threaded-godot-4 branch from 7c4cb14 to bd70b8e Compare January 17, 2024 18:58

akien-mga approved these changes Jan 18, 2024

View reviewed changes

akien-mga merged commit fa81059 into godotengine:master Jan 18, 2024
16 checks passed

akien-mga mentioned this pull request Jan 18, 2024

Cracking audio with Godot 4 no-threads Web builds #87329

Closed

Mickeon mentioned this pull request Feb 29, 2024

HTML5 export doesn't work in 4.3 on MacOS on some projects #88964

Open

Smooth-E mentioned this pull request Mar 9, 2024

Портирование на Godot 4 instant-games-bridge/instant-games-bridge-godot#8

Open

edassis mentioned this pull request Mar 30, 2024

Add missing part for the thread option added in 4.3-dev.3 EsdrasCaleb/godot_scormplugin#1

Merged

This was referenced Apr 29, 2024

Splash image support and new progress bar for HTML5 Web Export #91128

Merged

[SCons] Add option to build without threads godotengine/godot-cpp#1451

Merged

adamscott mentioned this pull request May 1, 2024

Add sample playback support for Web exports #91382

Merged

6 tasks

mihe mentioned this pull request Jun 17, 2024

Add support for web godot-jolt/godot-jolt#477

Open

Faless mentioned this pull request Jun 24, 2024

[Web] Add "threads"/"nothreads" feature tags to export presets #93556

Merged

Nukley mentioned this pull request Jun 29, 2024

Support http for Html5 exports godotengine/godot-proposals#10076

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `THREADS_ENABLED` macro in order to compile Godot to run on the main thread #85939

Add `THREADS_ENABLED` macro in order to compile Godot to run on the main thread #85939

adamscott commented Dec 8, 2023 •

edited

Loading

akien-mga left a comment

michaldev commented Dec 11, 2023 •

edited

Loading

jordo commented Dec 11, 2023

adamscott commented Dec 11, 2023

RandomShaper left a comment

wojtekmal commented Jan 16, 2024 •

edited

Loading

Calinou commented Jan 16, 2024 •

edited

Loading

adamscott commented Jan 17, 2024 •

edited

Loading

adamscott commented Jan 17, 2024

adamscott commented Jan 17, 2024

akien-mga commented Jan 17, 2024

akien-mga left a comment

akien-mga commented Jan 17, 2024

akien-mga commented Jan 17, 2024

adamscott commented Jan 17, 2024 •

edited

Loading

akien-mga commented Jan 18, 2024

akien-mga left a comment

akien-mga commented Jan 18, 2024

jordo commented Feb 8, 2024

geekley commented Jun 4, 2024

Add THREADS_ENABLED macro in order to compile Godot to run on the main thread #85939

Add THREADS_ENABLED macro in order to compile Godot to run on the main thread #85939

Conversation

adamscott commented Dec 8, 2023 • edited Loading

Context

Demo

Progress

Threads

File system

Audio

Rendering

Physics

Core

Export

Github

Misc

Fixes

Known issues

Edit

2024-01-12

akien-mga left a comment

Choose a reason for hiding this comment

michaldev commented Dec 11, 2023 • edited Loading

jordo commented Dec 11, 2023

adamscott commented Dec 11, 2023

RandomShaper left a comment

Choose a reason for hiding this comment

wojtekmal commented Jan 16, 2024 • edited Loading

Calinou commented Jan 16, 2024 • edited Loading

adamscott commented Jan 17, 2024 • edited Loading

adamscott commented Jan 17, 2024

adamscott commented Jan 17, 2024

akien-mga commented Jan 17, 2024

akien-mga left a comment

Choose a reason for hiding this comment

akien-mga commented Jan 17, 2024

akien-mga commented Jan 17, 2024

adamscott commented Jan 17, 2024 • edited Loading

akien-mga commented Jan 18, 2024

akien-mga left a comment

Choose a reason for hiding this comment

akien-mga commented Jan 18, 2024

jordo commented Feb 8, 2024

geekley commented Jun 4, 2024

Add `THREADS_ENABLED` macro in order to compile Godot to run on the main thread #85939

Add `THREADS_ENABLED` macro in order to compile Godot to run on the main thread #85939

adamscott commented Dec 8, 2023 •

edited

Loading

michaldev commented Dec 11, 2023 •

edited

Loading

wojtekmal commented Jan 16, 2024 •

edited

Loading

Calinou commented Jan 16, 2024 •

edited

Loading

adamscott commented Jan 17, 2024 •

edited

Loading

adamscott commented Jan 17, 2024 •

edited

Loading