Some questions concerning entt's implementation of ecs

[Sedomanai]

I loved reading your ECS Back & Forth series. I think I grew as a programmer (at least in theory), and understand your module better so thank you very much. I really appreciate your work.

This generated some additional questions however, specifically about entt api.

q_1. Is there anything in an entity that enforces it to be confined to its creator registry? I realized to my surprise that addition and retrieval is possible for an entity created from another regitry. I suppose it was to be expected... but I realized this was unwanted behavior when I destroyed the entity and could still access/retrieve that component that I enlisted in the other registry.

q_2. Related to q_1, but how did you implement destroying elements? I understand versioning and recycling now, but everything I read suggests you still have to iterate all the pools somehow, at the very least to set null_entity/tombstone for the relevant sparse index (even if we try to preserve pointers i.e. not pop the actual elements). Is there a way to not iterate the entire pools for this?

q_3. Part 8 on type_id made me excited since you just casually revealed a way to generate platform independent rtti, and that in compile time too. Believe me I was wondering myself (for years, I'll be honest) why such a thing was not part of the standard api. I tried it myself and really found that it's possible to generate a unique hash per type, order-independent and without enlisting anything manually.
However, combined with q_2, this made me curious. The runtime way of incrementing counters ensured that the type_id is continuous. This means the pools could also be ordered as an array, making indexing o(1) and iterating pools trivial. I wonder how you'd do this with a hash value however. I know this may seem like the equivalent of asking how hashes work, but I wanted to know whether you do things differently. Technically, doesn't this make component access amortized o(1) instead of pure o(1)? I mean sure the keys probably won't clash but... (also if it doesn't hurt to answer, what hash function do you use and what hash space size do you target?)

q_4. This is related to q_3. I was meaning to ask this one for a long time actually... but I didn't know how to word it until now. Due to entt requiring template type as key to access component/pools, there's no simple way in my knowledge to read/write component from other langauges, other than with indirection such as a bunch of get/setters for each component. For obvious reasons, I don't really prefer that method; I'm hoping that I can store entity keys and component hashes in another language and pinvoke directly from it to read/write a component, without having to write boilerplate code for each of them.
You've convinced me with part8 (q_3) that this is possible in theory. However, I want to know if there's a way to access a specific pool with a hash value directly. I think technically any entt operation would be directly available to another language this way, especially low-level ones (except adding to non-existing pools I suppose?)

I hope my questions made sense... I want to let you know I'm still terrible at parsing c++ (due to the black magic of tmp), this means I can't read the source code to check if my concerns/queries are valid. So I have to resort to manuals and uh.. just a bit of handholding. Sorry it got too long-winding at the end.

[skypjack]

I think I grew as a programmer

This is the best compliment you can do to a fellow developer. 🙂

q_1

No, there is not. EnTT fully supports a registry-less model too. The new version also introduces a storage entity for the purpose.

q_2

There are many ways but it really depends on the use case and the necessity. The default implementation just iterates them.

An alternative approach (that you can implement on top of EnTT) is to use a storage mixin to track the components assigned to an entity, then only touch their pools.
I did it in a game (which was shipped recently). The implementation was slightly different actually. It only tracks additions to save more cpu cycles and support frequent add/remove operations. A couple of false positives on 100 pools when you have more than 1000 of them are pretty much irrelevant anyway.
Otherwise, you can exploit the bump function of a storage, mark an entity as destroyed in the entity storage only, then spawn all storage cleanup in parallel at the end of the tick and have a massive deletion of all pending elements.
And so on. We can chat for hours about possible solutions to the problem. The fact is: different applications have different requirements (and most of them are just fine with a naive solution).

As everything else in EnTT, you have hooks to get the best out of it based on your requirements.
It rarely tries to offer a one-fits-all solutions that doesn't really fit anything, as it happens with many libraries out there. 🤷

q_3

If you liked it, you are probably interested in the rtti layer of EnTT. I love it and use it a lot actually.
As for the counter based solution, that's great but it has the problem that it breaks across boundaries. Using hashes is more reliable instead and works literally in all cases.
Of course, it's amortized rather than pure. However, I recently switched to the map based solution for the reasons above. To squeeze everything out of it, I've also imlemented the dense_map for the purpose. 😅
Under the hood, it uses fnv-1a but keeps in mind that the hash is computed at compile time, so I don't really do anything at runtime for that. When you ask for a pool, it's all about looking up an element in an array the size of which is a power of two. Your key is the (compile-time) hashed name of the pool mapped with a fast_mod technique on this vector.

q_4

You have the storage(id) function that returns a pointer to a base (opaque) storage. With it, you can get the component (as a void * ofc, since the type is unknown), emplace or erase it and even copy it across pools if you like. It's pretty easy to work with runtime pools actually but feel free to reach out if you have doubts/problems.

I hope my questions made sense... I want to let you know I'm still terrible at parsing c++ (due to the black magic of tmp), this means I can't read the source code to check if my concerns/queries are valid

No worries, you'll get better as the time passes as it happened for all others. 🙂

[Sedomanai]

Thank you. Just to clarify, I don't mind the performance or safety impact of any of these, I just wanted to know how they were implemented. I'm only beginning to learn about mixins so bear with me; if you mean using creation/destruction signals and mapping each entity with its component hashes, I think it's doable, I'll try it if it ever becomes a bottleneck. I also actually tag each destroyed entity and destroy them all at once at the end of the tick, would this do about as much you explained? I don't think I need to bump them because they're still considered "alive" until the end of the tick.

I was actually shifting through the docs and delving a bit deeper into the module since the last day (and even peering the source). I'm comfortable working with the storage now, but I didn't know it had another constructer that takes a type hash. I love the rtti layer! I'm not sure I know everything of it but type_id and type_info is more than enough for me, I can already think of so many usecases. Other things like finding/iterating all the components of an entity (without mixins) and mapping by type that crosses boundaries are all possible now.

So many doors were opened since the past couple of days studying sparse set ecs and parsing this module... I almost thought I had to roll my own (partly why I asked those questions) but only because I didn't investigate enough to use entt properly. Really should've asked and done this sooner, you already thought of everything and it looks like I don't have to reinvent the wheel.

[skypjack]

Just to clarify, I don't mind the performance or safety impact of any of these, I just wanted to know how they were implemented.

Yeah, I tried to give you some hints on different solutions with different tradeoffs for the same reason. 👍

if you mean using creation/destruction signals and mapping each entity with its component hashes

No, well, kind of. I mean mixins as in C++ mixin. It's a C++ idiom. If you look at the sigh_mixin class, that's what I'm talking about. 🙂
Briefly (very briefly), it's a sort of extension of a storage class that is configured at compile-time. The registry, the views and all other classes in EnTT don't care about it. Therefore, you can add whatever you want on top of a storage class.
Signals are implemented like this. The whole signal support is a mixin that you can turn off for all types or just a few ones with a single line of code.

I also actually tag each destroyed entity and destroy them all at once at the end of the tick, would this do about as much you explained?

Yeah, in this case, use the range-destroy function. It's optimized (A LOT) when you use built-in iterators.

you already thought of everything

Well, yeah, you're hitting the same walls I faced in the past apparently. 🙂

[Sedomanai]

Oh okay. I'll be sure to look into mixins and the sigh_mixin class as best as I can. If it's a customizable extension it's not something I can afford to miss. Hopefully understanding it will expand my general repertoire too.

Well, yeah, you're hitting the same walls I faced in the past apparently. 🙂

Apparently 😄 I never depended on a c++ module this extensively before, already. I just didn't know how to properly read it until now. I was wondering if I wanted to create one on my own for the learning experience but also because I was intimated before to pry into the specifics of entt to get more out of it. But I'm glad I started to do that now.

Anyway thank you so much for the wonderful library. And your writings too! I'll be sure to come back if I have more questions