Some implementation Questions

[SeeSoftware]

I have some questions about the best way to implement some things:

1. Component Dependency:
   for example if i have a "Position" Component and a "Velocity" Componenet and i want to create a new Component called "Physics" to have more complex behaiviour how woud i make Position and Velocity a requirement for having a physics component. I could pass the Registry and the Entity into the Physics component constructor to add the Position and Velocity components if they dont exist but that doesnt seem to be a clean solution.

2. Best way of implementing child entities:
   im kinda thinking about this too i would think that you create a "Parent" component that has a Entity handle to a child and thats it ? but what about passing information down the chain like if i change the position of the parent entity i want to inform the children that the parent moved and update their position accordingly. I would have to check in the "Position" component if it has a "Parent" component and then pass it down or is there a better way?.

3. Getting the registry or the attached entity in the component:
   can you do that without having to pass it into the components that need it? im not sure about that but would it be possible to create a Component base class for those type of things and then have a specialization in the attach function but i dont know if that is possible in C++.

4. What about baseclass Components:
   eg. if i have a Renderable Component and i want to subclass it to implement the draw function or something and add that to the entity and then get all of them with a view by searching all Renderables. is that possible, if not what would be a alternative?

[skypjack]

Uhm... Did you close the issue because you found the answers in the meantime?
If it's not so, I can give you my point of view... Let me know.

[SeeSoftware]

oh i think i missunderstood another thread where you said how to contact you. yes i would love to hear you point of view should i reopen it ?

[skypjack]

Not a problem, it's fine. Just asking.

So far, all what you can do is to create a bunch of factories to assign the right components to an entity when you create a concept.
As an example, when you create a button (if it has any mean in your software) you can assign transform, renderable, sprite, whatever by means of the factory method, then return the entity.
In the TODO list there is already a bullet about making an extension to the registry to register blueprints for concepts, but it isn't high priority because factories model worked fine so far for me and I prefer to work on other features.

It's easier than this.
Children should not have absolute positions. They should have positions relative to their parent.
When you need the world position, combine the two positions from the parent and the child and the job is done. This way, when you move a parent entity, all its children appear as if they have been automatically moved without even updating them.
Moreover you can get rid of the list of children because the parent field is all what you need in this case.

3 and 4.
It looks to me that you are putting logic into your components.
This isn't the way an ECS works and that's why you are experiencing problems like - I don't know to what entity a component belongs.
In a pure ECS approach, components contain only data and systems contain logic. This way systems iterate entities and their components and you have the full pack each and every time.

Honestly I can't help you with a components model based on hierarchies because it's not the way I do it usually and I'm not accustomed at working with hierarchies of components' types.
I'm sorry. If you are still in an early stage with your project, I'd suggest you to reconsider your design and try following a pure ECS approach. It's a bit hard to keep the first time, but it's really satisfying when you get how it works. ;-)

[SeeSoftware]

ye this is a new thing for me i allways implemented a ecs on my own and it allways had logic in entities. i guess imma try this way out thanks for sharing

[skypjack]

You're welcome. Feel free to contact me if you need help.

[Alan-FGR]

Hi there.

Regarding question 2, I'm facing the same challenge. However, my problem isn't transformation but mapping the hierarchy.
How would you guys recommend doing that? There are a couple of ways I could think of, one is storing the parent in a new component on each child entity, what presents some important optimization issues, for example transforming the children on a first pass and then transforming them again in the view that loops the parented objects (so we don't check if the child has a 'parent' component). Another way is having a new entity pool for each hierarchy 'depth', so for example when you want to parent entityB to entityA you simply move entityB to the pool that represents depth1 and add a component that holds the ref to entityA on depth0, that way when you apply your transforms (calc world matrix) you loop the pools from depth0...n and multiply by their parent matrix. This latter seems to be the most cache-friendly way to handle it, and you could handle a local dirty flag without caring about the rest of the hierarchy.

[Alan-FGR]

Maybe another solution to consider is having a 'parent' and 'depth' field in the 'transform' component that holds the id of the parent transform and then sort them by the depth before looping, so transformations are guaranteed to propagate from parent to child? Unless there's a fast way to loop all 'transform' components that don't have a 'parent' component on the same entity, that way it won't be necessary to store that data in the transforms at all

[Alan-FGR]

I think I got it: a Transform and a ParentedTransform component, I loop Transforms to update their positions, then sort ParentedTransforms by depth and update them. That way transformations will propagate and I don't loop parented stuff when updating world position objects. I might add a tag to the Transforms when I parent something to them however, so I at least know that when they're removed I have to search the children ParentedTransform. Unparenting is simply a matter of swapping the ParentedTransform by a Transform.
Edit: NestedTransform is probably a better name 😛

[indianakernick]

This may be of interest. Especially, the Transform Matrix System section.

[Alan-FGR]

Thank you @Kerndog73. Yes, that's exactly what I meant with a new entity pool for each hierarchy 'depth'. However, I realized that has too many pitfalls, for example you have to copy components between registries what's not an easy task and currently not supported in entt. For a tech demo like that it doesn't really matter though, but that won't be a flexible solution for an actual game.

[skypjack]

@Alan-FGR
I'm sorry if I'm late. I see you've already solved, anyway, so ... well, great job!! 😄
Let me know if you're encountering other problems.

for example you have to copy components between registries what's not an easy task and currently not supported in entt

Copying components is straightforward actually:

aRegistry.assign<Component>(anEntity, anotherRegistry.get<Component>(anotherEntity));

But you've to ensure the mapping between the two registries in term of entities and it could be annoying.
What could help probably is a create overload to which users can pass a hint so as to prefer a specific entity identifier during creation. This would ease a lot the mapping between different registries, mainly because one can ensure entity identifiers are consistent between them.
Something like this:

auto anEntity = aRegistry.create();
auto anotherEntity = anotherRegistry.create(anEntity);

anotherRegistry will return the identifier associated to anEntity if available (versions could differ), otherwise it will reserve a new one different from the hint.

Does it sound good? What other problems you encountered along this path?

Do not hesitate to ping me with requests for features on the line EnTT doesn't support it, it helps to improve the library!! Thank you.

[Alan-FGR]

@skypjack thank you for your reply. No worries, you're not late at all and it's great to see such a wonderful dev support here 😃...
The idea is to copy/move not just one component but the entity with all its components to another registry that represents a higher hierarchy depth. Making sure the mapping is valid is certainly something to account for, but besides that there's a general limitation when using a "Parent" component along with a dirty flag optimization. That's where I think using one registry per depth pays off, because in your movement systems (not necessarily internal, could be a physics engine for example) you update all positions and mark dirty, but then at the child objects you might want the world position at any time (say to play sound), not just when rendering, however, checking if the parent is dirty on all of them doesn't sound a very good solution. At the same time just a "Children" component on the parent object doesn't allow you to rebuild the matrices when you only need them on the children, say for example a child object that has a "Renderable", but the parent only has a "Transform" to propagate transformation, the "Transform" world matrix on the parent would never update from other systems, so you need to assure it's up to date when rendering the child object.
It's easy to do all of that naively, but trying to introduce a dirty flag quickly escalates into some insane code complexity if you don't have a traditional double-linked tree hierarchy structure (where parents know the children and vice versa). The problem with that approach is efficiency though.
Please let me know if you have any other ideas on how to efficiently do transformation propagation 😉 and thank you for your reply, I really appreciate it 😃.

[vblanco20-1]

@Alan-FGR What i did (on that space battle article) is designed to work stupidly fast and fully multithreaded on an scenario where everything is dynamic. It was inspired by the paper "pitfalls of object oriented programming).

Im dynamically generating a "scenetree" where each tree level is stored in a separated array. The whole thing is regenerated every frame, and then i do a parallel iteration to build the final matrix from depth 0 to the max depth.

The thing is that you dont have to generate it dynamically every time. In a normal game, you have dynamic objects and static objects. Both ue4 and unity have this distinction. In planning for some even bigger tech demos i thought of having 2 transform types. A static transform, and a dynamic transform. The static transform would be just a reference to the scene tree, while the dynamic transform would get regenerated all the time.

When rendering, first i would iterate over the entites with StaticTransform, and use that handle to get the (already calculated) final matrix to store it on the RenderTransform component. After calculating the static objects, i would do the whole "dynamic scenetree" on the dynamic objects to also calculate their RenderTransform component (dynamic objects as childs of static objects would work completely fine, but not in reverse).

Then the render system just renders everything by sending the RenderTransform to the gpu.

A thing to remember, is that you dont need to keep all your data in the ECS registry. There is nothing wrong with storing things in a specialized data structure (like a scene graph) for efficiency. Ive heard of people that just have a completely normal scenegraph as usual, and they have a "SceneNode" component that just holds a handle to the "actual" scenegraph node (wich has parent + child pointers)

Of course, keeping data outside of the ECS itself means you cant use the snapshot features to serialize the ECS, but would need to augment it with your own.

To help with this kind of things, the unity ECS has added "System State Components" wich are components that are linked to a system instead of an entity, and when the entity is destroyed, its system state components are not destroyed, but they are flagged as orphaned or similar, so the system can perform cleanup accordingly (you can use the events for this with Entt)

[Alan-FGR]

@vblanco20-1 Thank you for your reply 😃.

i thought of having 2 transform types. A static transform, and a dynamic transform. The static transform would be just a reference to the scene tree, while the dynamic transform would get regenerated all the time.

I thought about that too, but even if you've static and dynamic transforms, without a dirty flag you still need to update the dynamic transforms in the correct order, and you get O(n²) anyway.

A thing to remember, is that you dont need to keep all your data in the ECS registry. There is nothing wrong with storing things in a specialized data structure (like a scene graph) for efficiency. Ive heard of people that just have a completely normal scenegraph as usual, and they have a "SceneNode" component that just holds a handle to the "actual" scenegraph node (wich has parent + child pointers)

I've certainly considered some approach like that... keeping a separate tree of entities that simply maps their hierarchy, independently from the 'flat' entities registry. But that's not gonna be very performant.

I have found this however, that describes an approach that I think is by far the best way to go. Basically each transform that's been updated is moved to the end of the registry and a 'dirtyStart' variable is decremented. Then these 'dirty' transforms at the end of the registry are sorted so that parents always come before children, but they're not completely sorted, it doesn't matter if there are other transforms in between as long as overall parents always come before their children. That way you just loop from 'dirtyStart' to the end of the registry and update the world matrices as usual.

I don't know however if I'll face some problems implementing that in Entt, please let me know if there's anything I should be aware of 😉.

---

@skypjack Since this discussion grew quite substantially, maybe I should open a new issue so other people will benefit from it too since my question is specifically about ordered data propagation... maybe you could tag these issues/questions with a "Q&A/FAQ" label too even if they're closed, so later on they can be linked from a Wiki or something...

[vblanco20-1]

@Alan-FGR

I thought about that too, but even if you've static and dynamic transforms, without a dirty flag you still need to update the dynamic transforms in the correct order, and you get O(n²) anyway.

Not O(n²) in any way. You regenerate the tree and iterate it in top to down order. It is exactly O(n) Even the "sorting" to add the dynamic objects to the dynamic tree is still O(n) becouse you arent really sorting, just inserting into the level arrays. I havent tried the static+dynamic version, but the dynamic version ran at a seriously stupid high speed.

My next tech demo is going to be a 2d shooter with time rewind mechanics i want to try to put on the browser, will write about it once i have it.

[Alan-FGR]

@vblanco20-1 I'm sorry but I don't understand how it's O(n) unless you're caching the world transforms. Are you talking about looping the tree from roots to leaves and passing the parent transform as an argument? Other than that if you need it to be CPU-cache friendly by looping a flat list and not cache matrices internally I really don't see how it won't be O(n²) since you're recalculating the whole transformation chain aren't you?

[vblanco20-1]

Looking at the code would explain it easily, but i didnt put it public for now.

The algorithm is really simple:

• you update the RenderTransform component (local space) of all entities, from their own Position etc components or other stuff.
• you add all entities with a "parent" into an array of arrays where the first dimension is the parent "depth". (keep in mind the depth is stored in the "parent" component
• You iterate those arrays, starting from the ones that are depth 1 (depth 0 would be no parent) and basically do RenderTransform *= registry.get(ParentID)

Thats it.

The idea is that you are iterating roots to leaves, and this means that when you do a "get parent transform" it doesnt need to recalculate anything, as it was calculated on the step before.

The total number of iterations is 1 full iteration (to calculate local matrix + add to "tree"), and then a second full iteration (only childs) to calculate the world transform with their parent.

The biggest hit of performance is that on the "get parent" transform, thats probably going to be essentially random access, wich can fuck with cache a bit. Doing it sorted could be faster but would need sorting overhead.

My implementation calculated 140.000 transforms in 4 ms, of wich 1.5ms were spent on the initial iteration and local matrix building (plus insertion into tree), and then 2.5 ms for the parent transformations. scaled to 8 cores on a ryzen. The benchmark was a ton of little spaceships, each of them had the spaceship "core" at root depth, then the left wing as child of the core, and then the right wing as child of the left wing.

[Alan-FGR]

Yes, I see how that would work, it's certainly O(n) but the problem is you're looping all transforms. You don't need to recalculate what didn't change, and when stuff changes you do need to propagate that information (that it changed) to the children transforms. That's where the dirty flag comes handy but then again it won't be too easy to implement that solution in the link especially since Entt loops the array backwards. Maybe a simple solution would be a 'dirty' tag on objects, but then I'd have to store the children in the parent in order to mark them recursively.

[vblanco20-1]

Just have a distinction beetween dynamic objects and static objects. Every engine does that (i dont do a distinction becouse in my tech demo, everything is moving). Usually the amount of dynamic objects ends up being very low.

[Alan-FGR]

The commercial engines I know only have that distinction for static batching. They all have dirty flags so if you don't move an object you don't pay for it. Some even have multiple caching like for example rotation AND mat4x4 transformation (Urho for example). Simply by making a distinction between dynamic and static I'd still have to update all the children of all dynamic transforms each render (or each tick because world position might be used for other systems like AI and sound). I see how that solution works for a simple tech demo but for a real game I don't think it's the most appropriate solution.
That solution in the Stringray blog is just about perfect though, but unfortunately hard to implement in Entt. @skypjack?

---

COPYPASTE OF THE PERTINENT PART:

A better solution is to sort all the dirty objects to the end of the array, so we can loop over just them:

for (int i=first_dirty; i<n; ++i)
    transform(i);

Since we only need a partial sorting of the array, we don’t have to run an expensive O(n log n) sorting algorithm. (It would kind of defeat the purpose to run an O(n log n) sort to avoid an O(n) update.) Instead, we can achieve this by judicious swapping.
When a node becomes dirty we move it to the start of the dirty list by swapping it with the element before the dirty list and decreasing first_dirty:

                                 =============== dirty ==============
|   |   |   | D |   |   |   | X |   |   |   |   |   |   |   |   |   |

                             ================= dirty ================
|   |   |   | X |   |   |   | D |   |   |   |   |   |   |   |   |   |

We do the same for all children of the node and the children’s children, etc.
As we process the items in the dirty array, whenever we find a child that has its parent at a later position in the array, we swap the child and the parent.

                             ================= dirty ================
|   |   |   |   |   |   |   |   |   |   | C |   |   | P |   |   |   |
                                          ^

                             ================= dirty ================
|   |   |   |   |   |   |   |   |   |   | P |   |   | C |   |   |   |
                                          ^

This guarantees that parents are always processed before their children.

[skypjack]

@Alan-FGR
I don't think it's much difficult to implement actually.
To swap with the last (as mentioned in the article) you can just remove and reassign the component to the entity and that's all. We could also expose the swap functionality from the pool up to the registry, even though I'm not sure it's worth it.
Granted, in the meantime other instances of the same component could have been created, therefore you have no guarantees that all the dirty instances are tightly packed to the end of the array. Btw, they are all quite close to the end. You can sort the array using insertion sort in this case to move what remains to the end of the array and trust me, it's be extremely fast.
Now you have all the dirty entities at the end of the array. They will be returned before all the other ones if you use a view, right? That's fine, iterate them until the dirty flag resets and you did it.

Is it a viable approach for you?

---

To be honest, while I was writing the few lines above, another solution that has probably higher performance came to my mind.
Anyway, the basic principle is almost the same, so I won't pollute the answer (but we can discuss of a share of the revenues of your game if you are interested 😂).

[skypjack]

@Alan-FGR

Since this discussion grew quite substantially, maybe I should open a new issue so other people will benefit from it too since my question is specifically about ordered data propagation... maybe you could tag these issues/questions with a "Q&A/FAQ" label too even if they're closed, so later on they can be linked from a Wiki or something...

I wouldn't break the discussion in two issues. I'll add a new label anyway for issues like this one. Good point.

[Alan-FGR]

@skypjack thank you.

To swap with the last (as mentioned in the article) you can just remove and reassign the component to the entity and that's all. We could also expose the swap functionality from the pool up to the registry, even though I'm not sure it's worth it.

Isn't removing and reassigning a component slow compared to just swapping two existing components? Or would it effectively swap them internally if you do that?

Granted, in the meantime other instances of the same component could have been created, therefore you have no guarantees that all the dirty instances are tightly packed to the end of the array. Btw, they are all quite close to the end. You can sort the array using insertion sort in this case to move what remains to the end of the array and trust me, it's be extremely fast.

If there are a few redundant operations due to components being added while I was marking the dirty ones that's no problem because it shouldn't happen too often, so I think the sorting could even be skipped, because otherwise I'd need to keep track of how much the list grew and subtract the value for the first_dirty.

Is it a viable approach for you?

It sure is a viable approach. I think there are other alternatives that could be explored too, for example maybe a dirty "tag" and not a flag in the transform component. If tags are basically dataless components (correct me if I'm wrong) they should be quick to sort in a compound view, and maybe we could then have a "Children" component, so we update them while looping. The problem here is that it's going to be redundant if a child is dirty too, and also you might miss cache when accessing the transform data.

To be honest, while I was writing the few lines above, another solution that has probably higher performance came to my mind.
Anyway, the basic principle is almost the same, so I won't pollute the answer (but we can discuss of a share of the revenues of your game if you are interested 😂).

Sure, I'm especially generous today so I offer 0.00001% after the first 4 bazillion euros, deal?

[skypjack]

@Alan-FGR

Isn't removing and reassigning a component slow compared to just swapping two existing components? Or would it effectively swap them internally if you do that?

Definitely slower, even though not that slow. The fact is that we cannot make the swap member function of the sparse set a virtual function. It would slow down a lot the sort/respect functionalities. The first version was like that and I got something like a lot of performance by removing that virtual function. I won't rollback the change any time soon. :-)
The other way around is to implement a swap member function in the registry class template. Internally, it can do much more than a remove-and-assign operation and we won't have any performance hit. Btw, it's a bit ambiguous from my point of view. Let's consider this:

template<typename Component>
void swap(const entity_type lhs, cont entity_type rhs);

What would you expect from this function? Should it swap the components between the two entities, the two entities and therefore their components within the given pool or whatever? You want the latter, I know, but the former is much more meaningful semantically, right? Thus, ambiguous.

If there are a few redundant operations due to components being added while I was marking the dirty ones that's no problem because it shouldn't happen too often, so I think the sorting could even be skipped, because otherwise I'd need to keep track of how much the list grew and subtract the value for the first_dirty.

Naa, unless you are dealing with 1M+ entities, insertion sort on an almost sorted array is blazing fast. Look at the benchmark in the README file. It takes only 0.0007s on my laptop for 150k entities.
I mean, don't exclude the easiest solution because it will slow down everything. Do it, use the simplest one, you've already an alternative that is a bit more complex to manage and more error-prone. Sort the whole array with a parallel sort or something like this and get away with it. If and when you'll find it's the bottleneck, rework that part.

My two cents.

If tags are basically dataless components (correct me if I'm wrong) they should be quick to sort in a compound view

🤔
The comparison function has the following signature:

bool(const Component &, const Component &);

I suspect it won't be that easy to sort empty components in a compound view, you know... 😄

Sure, I'm especially generous today so I offer 0.00001% after the first 4 bazillion euros, deal?

Well, come on!!! 😂
At least I hope you'll consider to make a donation to EnTT. 😉

[Alan-FGR]

I suspect it won't be that easy to sort empty components in a compound view, you know... 😄

🤣
So would you recommend using dirty 'tags' for that purpose? The idea is to have a 'Children' component and a 'Parent' component that holds the 'depth' too for sorting. When we change a transform we add the 'dirty' tag to it, then on another view we loop all the transforms with the dirty tag and add the tag to their children recursively (prolly tons of cache misses though) so now we simply sort the entities with the dirty tag by depth and then loop and calculate the transforms. Maybe we could use a 'Dirty' component and store the depth there, so when we're recursively marking the children we pass the current_depth+1 and discard that if it's smaller than what's already assigned to it (if any). There are many drawbacks though, the worst one is that we're accessing the parent from the children to get the world transformation, so that's not good at all. We'll also have to be checking whether a tag/component is added to an entity.
Another solution is adding the transforms to a separate list (external) instead of marking/tagging them. Perhaps it could be an unordered_set kind of collection with a priority value... it can't simply be a normal vector that we later sort because if a given transform is added to it twice we just want to keep the one with higher depth since the value is the current transformation depth, not the whole tree depth. We could of course simply store the whole tree depth in the 'Parent' component in order to create a vector, then sort by id AND depth, and simply skip duplicates when applying the world transform.
It's hard to predict how all these complicated solutions are going to perform though, and they take time to implement, so I'm seriously contemplating the idea of simply propagating transforms immediately. I'd still need to know the children (to recurse) and the parent (to get world transform) though. There's no easy way around it.
Btw, of course I'd be more than happy to donate or revshare.

[skypjack]

@Alan-FGR
If I was you and if I was in an early stage of the process (that is your case if I got it right), I'd go with the simplest solution and wouldn't care about cache misses now. Really, the risk is that you spend a lot of time to optimize a part that isn't your bottleneck.
There are plenty of alternatives, right? We discussed some of them. Write down everything and forgot about it. If you'll find in future that this is the real bottleneck of your game, we can resume the discussion starting from this point and find the best solution for you. 😉

[Alan-FGR]

@skypjack you're 100% right ofc 😉... btw, what do you think about exclusion filter/negative selection for views? For example a view that matches all entities with component A but without component B?

[skypjack]

@Alan-FGR Do you mean to create a dedicated filter built-in in the view? Because doing it is straightforward already, you know:

registry.view<A>().each([&registry](const auto entity, const auto &a) {
    if(!registry.has<B>(entity)) {
        // that's all...
    }
});

I don't think we can do more than this honestly, even with a built-in solution. However, if you have an idea of how to improve views along this path, feel free to contact me! 😉

[Alan-FGR]

@skypjack yes I mean a dedicated filter. Maybe there's some trick to optimize that we're not aware of, but if there isn't then that would only be more stuff to maintain for no practical benefit.

[vblanco20-1]

On my testing doing the "has" thing slowed down everything, wich was an issue on my high performance overkill simulation.

[skypjack]

@vblanco20-1 @Alan-FGR
Yeah, actually one should at least use the view for the has because it's a bit faster than the registry.
@vblanco20-1 Did you do it using the registry or with the view?

[vblanco20-1]

I did it with the registry, i did not know that you could do it with the view. That would need some testing to see how it goes.

[skypjack]

@vblanco20-1 No, my fault. The view is constructed only for A, so you can't do has<B> on it. I apologize.
Probably there is room for improvements on such an operation actually. I'll consider it @Alan-FGR .

[indianakernick]

For finding entities that don’t have a component, I usually create a pair of components. A and NotA. I think this is better than having some function in the view because you can give a proper name to the concept of “doesn’t have A”. For example, in my tower defence game, I have BeamTower and ProjectileTower for lasers and cannons. I think this makes more sense than BeamTower and entt::not<BeamTower>.

[Alan-FGR]

@Kerndog73 makes sense but sometimes components aren't exclusive, for example you might want to loop the entities with 'Transform' but with no 'Parent' component. Making a 'UnparentedTransform' sounds meh but maybe even for perf reasons that's the way to go.

[skypjack]

@Alan-FGR

Just another viable solution. If you don't mind to make allocations here, remember that views are ranges.
You can also use std::set_difference probably. The sole problem I see is that it works on sorted ranges, so you have to sort B through the registry so that it respects the order of A (this operation is much, much faster then sorting a pool, see the benchmarks).

[zettdaymond]

Hello!

I try to write simple engine with entt and faced with some problems.

Imagine, that we have components:

• Position
• Rotation
• Hierarchy

struct Hierarchy
{
    uint32_t parent; //parent entity
};

and entt::DefaultRegistry scene.

We have System1, that changes Position component of entity e2:

System1::update() :
    scene.get<Position>(e2) = {0,1,0};
   ....

And System2, that makes entity e1 parent of entity e2 and changes Position component of e1. Then System2 performs raycast to make a decision about its next behaviour:

System2::update() :
    scene.assign<Hierarchy>(e2) = {e1};
    scene.get<Position>(e1) = {0,1,0};
    if( raycast() ) { ... }
    ....

1. To raycast properly we need to update all of our Position and Rotation components with respect to parent transform. Then we need to sync our scene transform with physics engine, and only after all that perform the raycast.
2. Additionally, we want to update and sync only changed components, and not loop over all:

raycast() :
    for Position, Rotation  in updated:
        updateTransform()
        syncWithPhysics()
    performRaycast()

Last problem makes me think about using an event system to store changed components within a list, something like

scene.get<Position>(e1) = {0,1,0};
notify<PositionChanged>(e1);

looks ugly, and very error prone.

First problem might be solved by introducing dependencies between systems:

class TransformSystem {...}
class SyncTransformsWithPhysicsSystem {...}

class Raycaster
{
    Raycaster(tuple<...> systems) 
    {
        systems.get<TransformSystem>(systems).update();
        systems.get<SyncTransformsWithPhysicsSystem>(systems).update();
    }

    void raycast(){...}
}

Maybe someone has a better ideas? @skypjack, what do you think?

[ArnCarveris]

@zettdaymond To do any Hierarchy stuff you need sort components. There is draft

[ArnCarveris]

@zettdaymond System in EnTT is just view, wrap any functionality to an function. Then you crate list of std::function<void(entt::DefaultRegistry&)>, add systems by priority and invoke each in run loop. There is project OWMAN with EnTT integration.

[skypjack]

@zettdaymond I think you should rather join the gitter channel for this kind of questions. There are programmers much skilled there.

Btw, you can mark changed components with a dirty flag (either a real flag somewhere or a dedicated component, it depends on your design). No need to trigger events for that, just split things in more than one system whether required.
Regarding raycast, I didn't get what problem you're facing exactly. You know the steps to do, why you can't just do them in the order you mentioned? Probably I'm missing something of your design you didn't mentioned.

[zettdaymond]

@ArnCarveris Looks interesting, thank you for sharing this.

[zettdaymond]

@skypjack Thanks for the answer!

1. I think about kind of dirty flags / dirty tags. But this introduces the question : "Who responsible to unset this flags?"

Imagine that we have 3 systems which update in main loop:

System1 -> System2 -> System3 | -> System1 -> System2 -> System3 | -> ...

System1: may use and change position of e1, and set dirty flag
System2: the same.
System3: the same.

But who responsable to unset dirty flag?

The answer may be: "The system, who has set it previously in last iteration."

Let's see:

Iteration1:
    System1 : uses and changes c1 of e1.
    System2 : uses and changes c1 of e1.
    System3 : uses and changes c1 of e1.
Iteration2:
    System1 : uses c1 of e1.
    System2 : uses and changes c1 of e1.
    System3 : uses c1 of e1.

If System1 unsets flag, then System2 will not know about System3 changes.

• This problem might be solved by using 'unique dirty flags' for each system, for example : attaching PositionChangedByPhysics tags and so on.
• Second solution, it is sort of event system - each system will notify others about components changes.

It seems that raycasting depends on behaviour of other systems and we get nested loops over components:

EnemySystem::update() {
    registry.view<Enemy, Transform, RigidBody>().each([](auto& entity, auto& enemy, auto& transform, auto& rigidBody) {
        
        if(raycast(...)) // <- 2 nested registry.view().
        {...}

    });
}

It might be a problem if we want to execute EnemySystem's view() in parallel. And i am not sure that keep such dependencies is a good idea , because we want to keep Systems as independent, as posible.

Maybe i am overcomplicating? Sorry for bad english.

[skypjack]

@zettdaymond Is there any chance you join us on gitter (see badge on the README file)?
There is much to write on the topic and a direct chat would be faster to get the point.

Let me know. Otherwise I'll reply here as soon as I find some spare time to do that. :-)

[dakom]

Only problem with gitter is now a year or two later I am not sure where the conversation ended ;)

[indianakernick]

@dakom You could look through the archives

[skypjack]

Or start a new conversation. 😄

[BartSiwek]

@vblanco20-1 I am attempting something similar to what you did in the ECS Huge Battle - that is have a contiguous array that contains all the transforms sorted by depth. I also have read your post where you mention "To do that, I perform a parallel for that iterates over the data of every entity, and stores the calculated matrix into the correct scene tree position".

What is unclear to me is how do you know the right position given you don't know how many different levels of depth you are dealing with or how many entities are on each level?

If you use any dynamically sized container, its not possible to do that update in parallel since any change in size would be a race on the container internals that deal with memory. So this would have to be done singe threaded.
If you use a container with preallocated space how do you know how much of it do you need without an additional pass over the data to calculate how many transform are on each level and how many levels there are?