-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unbounded HistoryTreeNode
field
#112
Comments
Exporting external group conversation@kevinlewi @afterdusk SELECT a.epoch
FROM HistoryNodeStateTable a
WHERE a.label_len = <label_len> AND a.label_val = <label_val> (In hindsight, a join to combine this with the query to HistoryTreeNodeTable wouldn’t be feasible, so two queries are needed.) |
My 2 cents. I've actually been thinking about this before as a place to optimize. For a lookup proof & a publish, we only need the most recent state of a What if we change |
…jority of operations only require the birth and current epochs. (Only audit & key history require the full list of epochs a node was mutated in) Resolves #112
Yes, @Jasleen1 to confirm. However, I think the intermediate states are needed for the append-only / key history consistency proofs. And we still need the information about which epochs a node actually gets changed in for these proofs. Otherwise, we could end up having to search for all epochs between the birth and current epochs for changes -- which may not be very efficient... Edit: left a comment here (https://github.com/novifinancial/akd/pull/113/files/9367f0ebc7dde560dfaf47491b1b32a5bb2eff51#diff-be3234553a30392a01a72481f1676c8338a439bc2a15a8a9f00f7923d1ef9353) which further elaborates on this |
Yeah @kevinlewi so in MySQL it's relatively straightforward with SELECT `epoch`
FROM `states`
WHERE
`label_len` = :len
AND `label_val` = :val
AND `epoch` <= :epoch
ORDER BY `epoch` DESC
LIMIT 1 which is searching over an indexed column space with "retrieve the first epoch which is <=
|
In MySQL, this guarantees that the query is going to return a single |
* Remove the epochs vector from the history tree node since the vast majority of operations only require the birth and current epochs. (Only audit & key history require the full list of epochs a node was mutated in) Resolves #112 * Add mysql limit to limit select as small as possible * Version bump to 0.3.6 Co-authored-by: Sean Lawlor <seanlawlor@fb.com>
HistoryTreeNode has an epochs field that is used to identify the epochs which an update took place.
As it stands now, the field can grow to an unbounded size. This will pose an issue when the corresponding DB record hits either soft or hard limits imposed by the DB schema or architecture.
ESTIMATED LIMITS
Epochs are u64 = 8 bytes.
SOLUTION?
The associated epochs for a HistoryTreeNode can be obtained directly from the HistoryNodeState table given the node label, since each update corresponds to a HistoryNodeState.
Accumulating the data from separate tables will incur some computation time on the database server and we will need to profile and assess the impact. However, I think this is necessary to avoid the unbounded record size issue. Are there any other potential solutions I've missed?
The text was updated successfully, but these errors were encountered: