You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current version of TxFlow DAG works, but was implemented using inefficient data structures in an inefficient way. The following optimizations could improve its speed and memory usage by several times or even an order of magnitude.
Replace u64 with more efficient and appropriate types. Current TxFlow DAG structs Message, Group, GroupApproval, GroupsPerEpoch, GroupApprovalPerEpoch, use u64 in most cases, because it was easy to write fast, this should be replaced. Note that most of these fields are only for internal usage and not going to be send across the network (except maybe epoch which is the part of the message) so it might make sense to use u8 for memory reasons, but it might not be faster CPU-wise with than u32.
Message, Group, GroupApproval, GroupsPerEpoch, GroupApprovalPerEpoch are currently returning entire datastructures in their methods, which of course creates unnecessary allocation and deallocation. Instead they should be returning iterators wherever it is possible;
Replace HashMap/HashSet usages, because we don't really need their flexibility of adding/removing elements at any time. Most of our mutations of these containers are limited to the construction time and therefore they can be replaced with simpler but more cost-efficient containers. Also some of our containers have limited number of elements so we might be able to use some bitmasks.
Keeping track only of the relevant epochs. The tracking containers inside Message struct, like approved_epochs, approved_complete_epochs, etc. keep track of all past messages. It makes sense to not keep track of very old messages. This issue might become irrelevant once we implement pruning.
The current version of TxFlow DAG works, but was implemented using inefficient data structures in an inefficient way. The following optimizations could improve its speed and memory usage by several times or even an order of magnitude.
Replace
u64
with more efficient and appropriate types. Current TxFlow DAG structsMessage
,Group
,GroupApproval
,GroupsPerEpoch
,GroupApprovalPerEpoch
, useu64
in most cases, because it was easy to write fast, this should be replaced. Note that most of these fields are only for internal usage and not going to be send across the network (except maybeepoch
which is the part of the message) so it might make sense to useu8
for memory reasons, but it might not be faster CPU-wise with thanu32
.Message
,Group
,GroupApproval
,GroupsPerEpoch
,GroupApprovalPerEpoch
are currently returning entire datastructures in their methods, which of course creates unnecessary allocation and deallocation. Instead they should be returning iterators wherever it is possible;Replace
HashMap/HashSet
usages, because we don't really need their flexibility of adding/removing elements at any time. Most of our mutations of these containers are limited to the construction time and therefore they can be replaced with simpler but more cost-efficient containers. Also some of our containers have limited number of elements so we might be able to use some bitmasks.Keeping track only of the relevant epochs. The tracking containers inside
Message
struct, likeapproved_epochs
,approved_complete_epochs
, etc. keep track of all past messages. It makes sense to not keep track of very old messages. This issue might become irrelevant once we implement pruning.Pruning. See separate issue: TxFlow DAG pruning #116
The text was updated successfully, but these errors were encountered: