Are component mappers and systems thread safe?

[GMWolf]

I'm thinking of building a parallel invocation handler that would automatically multithread systems. (the idea being that if two systems don't reference the same components, they can be called in parallel).

Would this work? or would it trip up subscriptions?

[junkdog]

The default invocation strategy synchronizes entity subscriptions in-between processing each system, but it could be limited to before and after processing the systems. Note that component removal is immediate per default (unless component is annotated with @DelayedComponentRemoval - a world-wide toggle is discussed in #508).

How many entities are you dealing with?

[GMWolf]

Well actually I'm just experimenting with multithreading in ECS frameworks. Its part of what I'm doing my dissertation on, so I like to experiment with different frameworks. (BTW its Artemis-odb that really sparked my love for ECS systems. So thanks!)

I'll have to look more ar the subscription synchronization. I think I can make it work by synchronizing after each 'batch' of systems. Naturally with multithreading Its hard to tell when problems could occure.

so to make sure: So as long as I handle subscriptions, and parallel systems don't interact with the same components, I should be golden right?

[junkdog]

Ah, cool :) What's your dissertation on (if MT ECS is only a part of it)? original artemis - or ECS design in general - is what got me hooked on gamedev.

(BTW its Artemis-odb that really sparked my love for ECS systems. So thanks!)

Thanks, it makes me happy when artemis *clicks* with someone 😊

Naturally with multithreading Its hard to tell when problems could occure.

Part of the reason why artemis is single-threaded ;)

If you're running a small simulation, you might get away with saving world state every couple of frames during debugging. And you can do whatever bookkeeping you need inside the InvocationStrategy.

so to make sure: So as long as I handle subscriptions, and parallel systems don't interact with the same components, I should be golden right?

Yes, but with one caveat:

When changing component composition (via ComponentMapper's create(e) and remove(e)), the compositionId is updated immediately; this could lead to race conditions if multiple systems make simultaneous updates. It could also lead to similar problems in systems which delegate based on compositionId (for example, a renderer system which delegates to the correct renderer specialization based on compositionId).

To get around this problem, it might be easier/convenient to use a custom ComponentMapper, to delay all composition changes until synchronizing entity subs.

Someone wrote a paper on multi-threaded ECS impl in Rust sometime not too long ago (~6 months?). I remember reading about on /r/rust_gamedev, /r/rust or similar. I only skimmed it, but it seemed pretty interesting. If you haven't read it, I can probably find it buried somewhere in my bookmarks.

[DaanVanYperen]

I've considered pulling entities-per-call and call frequency behaviors out of the system hierarchy as annotations into a rich invocation handler. A multi-threading behavior annotation on systems would be an amazing feature. Low risk as well, as users could just toggle it on/off.

What kind of practical applications are you considering? Typical applications that come up a lot are decoupling render/logic, in-system threaded render pipeline, that sort of thing. I remember reading an interesting discussion about threading on github recently.
Might have been amethyst/amethyst#10

Suspect core might need a bit of retooling or opening up. Fluid entity module wouldn't work out of the box multi-world /multi-threaded for sure, you could consider sticking to odb core, at least initially.

The code for ExtendedComponentMapperPlugin might give you a kickstart to make your own plugin https://github.com/DaanVanYperen/artemis-odb-contrib/wiki/Extended-Component-Mappers.

[GMWolf]

Ah, cool :) What's your dissertation on (if MT ECS is only a part of it)? original artemis - or ECS design in general - is what got me hooked on gamedev.

Something along the lines of "Building an efficient entity component system". Its a 3rd year bachelors dissertation so its nothing huge.
I'm mostly focussing on cache optimization and reducing virtual calls. (using c++, but finding it useful to look at how Artemis is designed. Got the Aspect subscription idea from it).
(I'm not expecting any of the code I write for Artemis to apply to the ECS I'm working on, it's already vastly different. but I enjoy doing it :) )

To get around this problem, it might be easier/convenient to use a custom ComponentMapper, to delay all composition changes until synchronizing entity subs.

That's a good idea. thanks. will try!

Someone wrote a paper on multi-threaded ECS impl in Rust sometime not too long ago (~6 months?). I remember reading about on /r/rust_gamedev, /r/rust or similar. I only skimmed it, but it seemed pretty interesting. If you haven't read it, I can probably find it buried somewhere in my bookmarks.

Thanks that will be useful!

What kind of practical applications are you considering?

I was thinking of building a system that would automatically parallelize systems by looking at what components each system depends on, and what order they are called in. (Of course it would only work if the game is programmed in a strict ECS manner. If the systems make unsynchronized calls to other Data structures, it could break down).

I'll try creating a mapper that delays the composition changes. Thanks a lot for your help!