Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encode activesets with delta #5282

Open
dshulyak opened this issue Nov 20, 2023 · 1 comment
Open

encode activesets with delta #5282

dshulyak opened this issue Nov 20, 2023 · 1 comment

Comments

@dshulyak
Copy link
Contributor

dshulyak commented Nov 20, 2023

in recent epoch there are many activesets that are slightly different. the root cause of this difference is suboptimal reconciliation
of known atxs before the start of the epoch. which should be addressed on its own.

however activesets can be encoded more efficiently as well. instead of generating new active sets smesher can lookup
previously generated activeset and extend/remove items from it based on local view.

  • builder selected an activeset and ready to publish reference ballot
  • pick activeset with the most weight from activesets received in this epoch
  • compute added/removed hashes and encode with reference to highest activeset
  • broadcast delta-activeset, if peer is missing a reference it should request it, but in most cases it should already have it as it was created earlier
  • use new protocol that relies on streaming.

note that peer can request data using previous (non-delta encoding) or a new delta-encoding for backward compatibility.
if peer requested non-delta encoding we should re-encode it after loading from database. eventually we will be able to drop non-delta encoded activesets, but we will need to maintain backward compatibility for several epochs.

both this and #5306 are aimed to tackle the same problem, but the latter one is more important, and we should do that first, maybe we won't even need to implement this one.

spacemesh-bors bot pushed a commit that referenced this issue Nov 22, 2023
)

related: #5282

this should reduce variability in encoded active sets.
previous grader was using hare preround delay, which is 25s and not sufficient for atx to propagate across the network.
dshulyak added a commit that referenced this issue Nov 24, 2023
)

related: #5282

this should reduce variability in encoded active sets.
previous grader was using hare preround delay, which is 25s and not sufficient for atx to propagate across the network.
@dshulyak
Copy link
Contributor Author

use streams #5278

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 📋 Backlog
Development

No branches or pull requests

1 participant