💡 cpp (raw) backend implementation

[chaosddp]

Summary

A cpp backend implementation.

Motivation

With self write backend implementation, we can add more features we need later, not limited by underlying modules.

Description

Basic:

We can have a variant class to hold all data type that we supported, and consider it as an attribute slot, then a frame that used to hold all these attributes and their information (like slot number, node index it belongs to). And there is not Node physically, just a concept.

Advance:

attributes that not in snapshots

we can just split the memory into 3 parts: frame (attributes will be in snapshot), frame (attributes not in snapshot), snapshots.

NOTE: this feature would be not that easy for numpy backend, as we are using structured-array.

dynamic nodes

we can have 2 implementation about this:

1. user provides a buffer size, we allocate memory with this buffer size, and dynamic nodes (attributes) cannot more than this chunk size.
2. reallocate whole memory for each dynamic node request, this is more flexible, but need more memory size to reallocate.

batch operation

we can use openmp to support batch operation, it would be not that hard to implement, as each operation would access different memory address, do not need a lock.

Steps:

This implementation would be separated into following phases.

1. basic features that same as current numpy backend
2. complete other basic features base on current backends (numpy and cpp)
3. extend features based on this implementation, and merge back to numpy backend if possible

References

No

Alternatives

No

Additional context

No