Refactored entire microbe stage to switch to an ECS approach for performance reasons #4521
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hhyyrylainen
force-pushed
the
performance_refactoring
branch
from
e8a0529
to
2c8cae8
Compare
Setup a version for the native library and tweaked its build Made the FeatureInformation.cs clearer for different platforms by using specific named constants for the platform names Added calling native methods setup and cleanup Non-working native method call interface, it doesn't find the library probably needs to do a mono specific search path hack Trying to override the mono load directory still doesn't work as mono tries to resolve the attributes way too early Kind of working native lib setup when it is put in a very specific folder Added JoltPhysics as a submodule Customized more and added missing clang format options Just one blank line to keep
Made sure API import also defines default visibility and added pool setting Tweaked Jolt build settings and warning options for the thrive lib Started on a few various files, tweaked some stuff, setup clang tidy rules more sensibly
added really basic logger interface Switch to normal quotes for Jolt imports probably better to do it that way and not fight my IDE all the time Added some shape creation helpers Added the new files to compile Finished setting up some basic physical world stuff Got jolt to combine into the shared library Cleaned up the native interop file a bit Setup native log message forwarding Add mesh creation helper method Add new libraries to LICENSE.txt
…sive_ptr Tweaked the precompiled header settings a bit Slightly more conservative RefCounted memory ordering Switched the namespace style and removed the unneeded RefCounted.cpp file
Added mising BOM to NativeInterop.cs Tried to adjust .clang-format penalties Setting up more C API access to physics Physics body and safety checks against used bodies being deleted Unify some nullptr checks Tweaked library header regex to require a folder change this fixes things like JoltCompatibilityStuff.hpp etc. Added a bunch of features and fixed Jolt initialization on the native side Added wrapper classes for the needed physics C++ side things Started making a simple physics test scene Fixed the sphere visual size and the ground offset Did some wanted code reordering by the checks system
and also the needed libs, this adds quite a bit of size but hopefully it's a fine tradeoff to not have to write this and the more libs from boost we use the smaller the size increases will be
…racking Only compile the assert method if Jolt asserts are enabled added the jolt include to ensure it is always included with the interop file A couple of physics tweaks based on the Jolt hello world Relaxed the RefCounted timings to match basically what they were originally as Jolt also had the same values Fixed compile failing Added some comment extra info on the number of threads to use
Putting in some GUI for the physics test Setting up more physics body stuff and started on constraints C# side of more complex physics and constraint setup Almost done physics test for microbe style collisions Forgot to include these needed PhysicsBody.hpp constraint-related changes Fixed indent for a few JPH macros Trying to implement a custom axis lock constraint might be the last commit with it, with the next commits switching to configuring an existing constraint A ton of native physics side changes to constraints, shape generation and body interaction Managed side wrapper changes for previous commit, microbe shape generation now works Some things are now working in the microbe physics test Smoothly interpolate the camera Switched the concave cell shape creator to make pancakes instead of prism like shapes Fixed a long line Don't scale impulse by delta Allow creating microbe shape as spheres
Made microbe convex bodies thicker Added methods for damping and y fixing / moving bodies manually Removed test world gravity for microbes, added explicit Y drift fix fixed microbes not wanting to move in the negative direction Made Y enforcing optional and if disabled now prints drifting bodies Lowered the similarity requirement for target rotation Added pure Godot version of microbe physics test Slightly reduced the physics force for the Jolt test Reduced turn thresshold for Jolt as well to reduce turn jitter Updated jolt version
…all godot visuals when don't need to Added a stress test physics pattern Fixed the third party folder ignores and native folder C++ files as the native files will have a different check for problems in them Renamed body control rotation rate to not refer to seconds as it is very likely not guaranteed to match a real world time Trying to add extra control to fix jittering case near target rotation Cleanup on JVec3.cs Store physics body control variables in the body Added code to dump a physics world state Implemented Jolt debug drawing except text debug rendering is not implemented Implemented a debug line and triangle drawer and hooked it up to the native side Fix critical bug in the RefCounted implementation it didn't have a virtual destructor so a lot of destructor code didn't run correctly Added a single cell physics test Set a more reasonable physics cell rotation speed this ensures that the speed shouldn't overshoot the targets Constants for the debug drawer to automatically turn on for physics Fixed a major mistake of mixing up the convex and sphere bodies for Jolt Made debug drawer follow memory limit to ensure it just doesn't disappear entirely when hitting the limit Fixed physics debug level cycling Made debug drawer show an approximate amount of extra memory needed Added camera zooming in physics test Fixed the test zoom speed Adjusted physics test camera far distance Added debug drawing of contacts Changed microbe convex thickness to 1 to fix collisions getting ignored Added auto adjust for debug physics draw update rate Finally fixed the debug draw allowed use memory limit it might be a bit pessimistic now but at least it follows the limit Implemented not rendering far away physics shapes in debug draw Fixed a body control problem where the overshoot detection never got enabled also made the rotation speed slow down when quite near the target Disabled some warnings that got printed when debug rendering wasn't compiled in Clarified the rotation rate variable in BodyControlState.hpp Increased default debug line draw max distance Increased the close angle threshold to slow down rotation a bit Fixed physics test not printing the right created body count for Godot collision test Removed the option to try to use inertia lock for axis rotation lock as it was not working well at all
with the new external physics simulation system Just about all other files except the Microbe class compile again Everything compiles again but with a ton of disabled code Started on using components (ECS) Brought back a bunch of the microbe code and removed the other entity inheritance tree as it will not be needed at all when using an ECS Started to add a bit of microbe ECS components Implemented a new system of easily reusing predefined visual scenes mainly for ECS based use
Added a few missing usings Made a few cell components compile again to quiet errors temporarily before they are converted Renamed PhysicsBody to NativePhysicsBody to make it nicer to use when Godot is also imported First draft of the collision management system still missing collision callbacks and the actual native backend
Added spatial instance scaling
Constant for toxin projectile physics sphere size Bunch more work on the toxin entity, C# side is almost entirely done just need a bit more of the collision stuff done Made physics collision management compile for now still missing the logic to go from physics bodies to entity references Partially converted the floating chunk entity to ECS Remove trying to compile the task system file which was not anything yet Converted two more systems needed by chunks More work on the ecs conversion Finished converting floating chunk all that's left are things with "microbe" in the name Commenting out some stuff to get closer to compiling code
Include boost smart_ptr and some boost top level headers made the boost_core target include the top boost headers, hopefully this entire thing now fully works Stop ignoring folders named "debug" as boost header files use those and we don't have any folders named debug after all Added accidentally ignored boost_preprocessor files
Everything compiles now with a ton of the microbe code commented out
opcode to load the struct address which does bounds checking
Fixed mixing up the chunk spawn density and new mass density
Removed the no y-drift enforce and just kept the checking Fixed jitter in body rotation control by default disabled the option to slow rotation near target angle Removed some old commented out code about using inverse inertia axis locking approach as this should not really be useful in any situation anymore
…icrobe needed components A bunch more of components that will be needed for microbes to function Removed a really outdated comment on SpawnBacteriaColony Created components for Microbe properties and adjusted a few other ones Removed one extraneous NodePath to string conversion Got most of the microbe spawn component setup done
Fixed issue of not detecting spawned entities in benchmark A more sensible approach to what the microbe stage will need Microbe benchmark now shows membranes Membrane colour apply now works Organelles display now kind of Microbe physics body creation now works probably, this tries to improve the performance for the data passing to C++ which hopefully is correct but it is yet untested Working on hooking up some missing physics C++ methods and how to listen for collision data Disabled the Y-axis enforce option by default as it is not needed with the fixed physics engine y-axis fixing
Updated boost libraries to 1.83 and removed the readme files in the library subfolders
Made our physics wrappers use object pool allocations and modified the RefCounted things to match, hopefully the global pool approach is fine and doesn't require changing to physics world specific pools Aggressively inline the getting of active collisions Implemented the C# side of the first full version of collision ignoring C++ side of the previous commit's APIs Fixed duplicate physics body collision disable flag Fixed one more duplicate property about collision disable and refactored the collision recording registration a bit Simplified the collision filter registering on C# side Implemented more C# side physics methods that were unimplemented calling into the native side
for the physics, these are both needed now by the speed calculation so it is very important to cache these
fixed one organelle visuals update bug
still needs to fix building locally for Windows
after changes to the cell, fixed pilus positioning for eukaryotes
hhyyrylainen
force-pushed
the
performance_refactoring
branch
from
7b85cc8
to
5586d11
Compare
hhyyrylainen
force-pushed
the
performance_refactoring
branch
from
12ce1f2
to
58133b4
Compare
hhyyrylainen
force-pushed
the
performance_refactoring
branch
from
58133b4
to
01f1fa8
Compare
… for that I could not get a build working with clang and msys2 or other tools without resorting to visual studio
This was referenced Oct 28, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Brief Description of What This PR Does
This is the huge PR rewriting / moving all of the microbe stage code to use an entity component system approach for much more performance (and a different physics engine than going through Godot).
I'm opening this PR now as I think I'm at most a few weeks away from being able to merge this to master (even then this will have missing multicellular stage and probably an absolute ton of new bugs but I think this needs to get merged to no longer have a situation where people keep working with the outdated code I've removed / moved).
Related Issues
closes #3016
closes #1282
closes #2112
closes #3770
incidentally this also fixes:
closes #2129
closes #2470
closes #2495
closes #2341
closes #2339
closes #2671
closes #2581
closes #2865
closes #3027
closes #3001
closes #3257
closes #3269
closes #3645
closes #3569
closes #3760
closes #3745
closes #3679
closes #3875
closes #3797
closes #3945
closes #3942
closes #3922
closes #3921
closes #3911
closes #4081
closes #4342
Progress Checklist
Note: before starting this checklist the PR should be marked as non-draft.
break existing features:
https://wiki.revolutionarygamesstudio.com/wiki/Testing_Checklist
(this is important as to not waste the time of Thrive team
members reviewing this PR)
styleguide.
Before merging all CI jobs should finish on this PR without errors, if
there are automatically detected style issues they should be fixed by
the PR author. Merging must follow our
styleguide.