Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add quick-dao-startup mode #266

Closed
chimp1984 opened this issue Oct 20, 2020 · 0 comments
Closed

Add quick-dao-startup mode #266

chimp1984 opened this issue Oct 20, 2020 · 0 comments
Labels
an:idea https://github.com/bisq-network/proposals/issues/182#issuecomment-596599174 re:processes was:stalled

Comments

@chimp1984
Copy link

chimp1984 commented Oct 20, 2020

This is a Bisq Network proposal. Please familiarize yourself with the submission and review process.

If a user has not started Bisq for a while the DAO sync can be quite heavy and as it runs on the UI thread it can slow down UI quite a bit on low end machines. Some users do not use the DAO at all (if not paying trade fee in BSQ). I think we should try to find a solution to lower the burden for those users.

One option would be to check when the app was last time used and if it exceeds a certain period (e.g. 5 days) the initial BSQ blocks request is replaced by a new request where the seed is providing the full dao state data. That would be the mutable data structures + the missing (immutable) blocks. The mutable data is about 7.5 MB [1] and the missing blocks are about a few 100kb. The data sent are thus much larger as when only requesting the blocks but the user do not need to run validation over 100s or 1000s of blocks, so its a trade-off between traffic and performance. The user should be able to disable that automatic feature in the settings (or define the duration when it gets triggered).

Of course those nodes do not contribute to secure the validation of BSQ and they put higher trust into the seed nodes as it is now. But any invalid data would be detected by the in-app dao state hash monitor and most more active users would run the full validation anyway.

As 7.5. MB is quite a bit heavy we have to look into solutions to reduce the size for the mutable data. The bigger ones are maps with tx output id as key. We can use a similar model as with sending the hashes of the data we already have to be excluded from responses from seed nodes at initial data requests. The txo ids are still quite large to send as excluded keys to the seeds (about 3 MB) but we can use a 20 byte hash from the txo ids ans use the 20 bytes as exlude key. SpentInfoMap has 47492 entries atm, so with 20 bytes it would be about 1 MB for the request. The response would still contain the full map entry (120 bytes per entry) but it can be expected that it will not be that many missing items.

To reduce even further the 1 MB request size we could use the approach now used for trade statistics to split up the data in version-tagged historical data.

We could also consider to redesgin the data model to break it up in individual db files and use the existing getData requests to deliver those data as well.

[1]
Size in bytes if the data inside daoState. Blocks are the main part with 77 MB followed by getSpentInfoMap with 6.5 MB.

Oct-19 21:25:56.071 INFO b.c.dao.state.model.DaoState: getBlocksList 77570139 
Oct-19 21:25:56.073 INFO b.c.dao.state.model.DaoState: getCyclesList 936 
Oct-19 21:25:56.076 INFO b.c.dao.state.model.DaoState: getUnspentTxOutputMap 668023 
Oct-19 21:34:05.452 INFO b.c.dao.state.model.DaoState: getSpentInfoMap keys 3134618 
Oct-19 21:34:05.454 INFO b.c.dao.state.model.DaoState: getSpentInfoMap valued 3328132 
Oct-19 21:25:56.097 INFO b.c.dao.state.model.DaoState: getSpentInfoMap 6455270 
Oct-19 21:25:56.098 INFO b.c.dao.state.model.DaoState: getIssuanceMap 81178 
Oct-19 21:25:56.099 INFO b.c.dao.state.model.DaoState: getConfiscatedLockupTxListList 0 
Oct-19 21:25:56.099 INFO b.c.dao.state.model.DaoState: getParamChangeListList 925 
Oct-19 21:25:56.101 INFO b.c.dao.state.model.DaoState: getEvaluatedProposalListList 98088 
Oct-19 21:25:56.110 INFO b.c.dao.state.model.DaoState: getDecryptedBallotsWithMeritsListList 1183309 
@MwithM MwithM added an:idea https://github.com/bisq-network/proposals/issues/182#issuecomment-596599174 re:processes labels Oct 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
an:idea https://github.com/bisq-network/proposals/issues/182#issuecomment-596599174 re:processes was:stalled
Projects
None yet
Development

No branches or pull requests

3 participants