You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the current implementation, once TxFlow message is constructed and added to DAG it is never modified, which allows safety and efficiency. However, each TxFlow message carries heavy "aggregators" -- containers that aggregate information from the parents of the message -- which we use to compute properties of the message in near-constant time without traversing the DAG, see approved_* and computed_* fields in message/mod.rs. Since these containers grow with the number of past messages the current memory usage of the DAG grows quadratically. We can solve it by pruning the aggregators of the very old messages in the DAG. It requires several changes:
The aggregators should be wrapped in RefCell. This will allow to have a mutable access to them even for the messages that are deep down in DAG (note, these messages are linked to their children through immutable references, so we cannot avoid RefCel).
The aggregators should be prunable and restorable. Meaning the containers should have function prune or some such that erases its content. If later this message is used again (e.g. a new message comes in that refers to it as parent) then we should be able to "restore" the content by recomputing it from the parents.
Implement "pruner" a struct that will monitor the usage of the DAG and prune rarely used messages.
Note, we might not need to implement this for MVB, because TxFlow is getting restarted for each BeaconChain block.
The text was updated successfully, but these errors were encountered:
In the current implementation, once TxFlow message is constructed and added to DAG it is never modified, which allows safety and efficiency. However, each TxFlow message carries heavy "aggregators" -- containers that aggregate information from the parents of the message -- which we use to compute properties of the message in near-constant time without traversing the DAG, see
approved_*
andcomputed_*
fields inmessage/mod.rs
. Since these containers grow with the number of past messages the current memory usage of the DAG grows quadratically. We can solve it by pruning the aggregators of the very old messages in the DAG. It requires several changes:The aggregators should be wrapped in
RefCell
. This will allow to have a mutable access to them even for the messages that are deep down in DAG (note, these messages are linked to their children through immutable references, so we cannot avoidRefCel
).The aggregators should be prunable and restorable. Meaning the containers should have function
prune
or some such that erases its content. If later this message is used again (e.g. a new message comes in that refers to it as parent) then we should be able to "restore" the content by recomputing it from the parents.Implement "pruner" a struct that will monitor the usage of the DAG and prune rarely used messages.
Note, we might not need to implement this for MVB, because TxFlow is getting restarted for each BeaconChain block.
The text was updated successfully, but these errors were encountered: