Support Deserialising Array3 Properties Into Arrays

[MarkKoz]

A data table stores properties of several types. One of those types is Array3. This type is distinct from the Array type and is serialised quite differently. An Array3 is actually send as a data table, with each element of the array being a property of the data table. These properties are given a name from "000" to "1023" based on their order in the corresponding array.

For example, m_hostageRescueX is an Array3 field of the class C_CS_PlayerResource. Serialisation results in a DT_CSPlayerResource data table which has a property m_hostageRescueX. That property is also a data table and contains properties like "000", "001", and so on.

Currently, the demo parser handles Array3 by flattening all the properties into the parent data table's properties. Following the above example, the result would be that DT_CSPlayerResource directly contains properties m_hostageRescueX.000, m_hostageRescueX.001, and so on. This is clumsy because there's no way to easily iterate over the properties for a specific array nor to know how many properties that array has.

Deserialising it into an array instead would be great, but it doesn't seem so simple to accomplish. First, how can the parser know it's dealing with an Array3? Currently, all it knows is that it has a data table. A possible avenue for detecting this would be checking if SendProp.m_pArrayProp is not null. Both Array and Array3 set this, but the former's m_Type is DPT_Array while the latter's is DPT_DataTable. However, m_pArrayProp is not something that's in the CSVCMsg_SendTable protobuf message.

The Source Engine has class definitions on both the server and client side. Each side has the DLL for the other, so each side has access to the class definitions in the other. The class definitions are registered into a global registry as data tables, and their fields are registered as properties of those data tables. When receiving a property, the engine can use this registry to know into which class to deserialise that property. This mechanism is likely how it can distinguish Array3 types. However, I have not figured out how exactly it goes about deserialising Array3.

Second, how can the parser actually get all the data into an array at once? The current API design requires an event handler to be attached to a property, because deserialising the values of the properties happens after the data table's metadata (such as the list of properties) is parsed. Deserialising Array is relatively simple since all its data in the stream is contiguous, but is this also the case for Array3 properties?

[MarkKoz]

I think I figured out how the Source Engine deserialises these. It does also have a property flattening mechanism that I used to kind-of understand, but no longer remember. Anyway, I believe the array properties also end up as properties of the parent data table. However, when the Array3 property was registered, each property of the resulting data table had its offset set relative to the offset of the Array3 property i.e. the actual array field in the class.

Therefore, the decoder knows the offset that corresponds to the position each property should have in the array. Building the array is as simple as writing to those offsets for each flattened property.

These offsets are not transmitted over the network packets. Rather, it knows them because, as mentioned above, it uses the class definitions.

Our demo parser doesn't have anything like this. Instead, it uses an event system to simply notify when values are received for properties. Even if the data was deserialised into an array, it would mean the event would have to just send the full array when any of its values change. That may be okay since it can just pass by reference.

---

Ultimately, I think the system of having classes with attributes registered to data table props would be nicer as an API over events, though more to manage internally. There isn't an inherent performance difference, since all entity updates are handled and have events dispatched even if there are no handlers attached. There could be differences when it comes to implementation details e.g. overhead of dispatching events vs. reflection for attributes used to register properties (or some faster approach that doesn't rely on reflection). Another difference would be that the stats collector doesn't have to constantly update its own state when entities update if the collector only cares about the final state of a property e.g. at the end of each round.

Classes would be more structured and organised. It'd be simpler than dealing with a bunch of events. The stats collector would essentially become a thing that just snapshots the current (partial) state of entity classes by copying the fields of the classes.