Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TxFlow DAG performance optimization #117

Closed
5 tasks
MaksymZavershynskyi opened this issue Nov 30, 2018 · 1 comment
Closed
5 tasks

TxFlow DAG performance optimization #117

MaksymZavershynskyi opened this issue Nov 30, 2018 · 1 comment
Assignees
Labels
C-enhancement Category: An issue proposing an enhancement or a PR with one.
Milestone

Comments

@MaksymZavershynskyi
Copy link
Contributor

The current version of TxFlow DAG works, but was implemented using inefficient data structures in an inefficient way. The following optimizations could improve its speed and memory usage by several times or even an order of magnitude.

  • Replace u64 with more efficient and appropriate types. Current TxFlow DAG structs Message, Group, GroupApproval, GroupsPerEpoch, GroupApprovalPerEpoch, use u64 in most cases, because it was easy to write fast, this should be replaced. Note that most of these fields are only for internal usage and not going to be send across the network (except maybe epoch which is the part of the message) so it might make sense to use u8 for memory reasons, but it might not be faster CPU-wise with than u32.

  • Message, Group, GroupApproval, GroupsPerEpoch, GroupApprovalPerEpoch are currently returning entire datastructures in their methods, which of course creates unnecessary allocation and deallocation. Instead they should be returning iterators wherever it is possible;

  • Replace HashMap/HashSet usages, because we don't really need their flexibility of adding/removing elements at any time. Most of our mutations of these containers are limited to the construction time and therefore they can be replaced with simpler but more cost-efficient containers. Also some of our containers have limited number of elements so we might be able to use some bitmasks.

  • Keeping track only of the relevant epochs. The tracking containers inside Message struct, like approved_epochs, approved_complete_epochs, etc. keep track of all past messages. It makes sense to not keep track of very old messages. This issue might become irrelevant once we implement pruning.

  • Pruning. See separate issue: TxFlow DAG pruning #116

@MaksymZavershynskyi MaksymZavershynskyi added the C-enhancement Category: An issue proposing an enhancement or a PR with one. label Nov 30, 2018
@MaksymZavershynskyi MaksymZavershynskyi added this to the MVB milestone Nov 30, 2018
@MaksymZavershynskyi MaksymZavershynskyi self-assigned this Nov 30, 2018
@MaksymZavershynskyi
Copy link
Contributor Author

TxFlow is getting removed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Category: An issue proposing an enhancement or a PR with one.
Projects
None yet
Development

No branches or pull requests

1 participant