Support resource files tagged by version to reduce initial data requests #4519
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of trying to find faulty keys, we stick to the mechanics we have in place. This way, we have less false positives. However, we enforce a certain Bisq version format, as parsing and business logic relies on that now.
Turns out, scanning resources does not work reliable enough. Thus, an array of Strings denoting historical Bisq versions it is. An other way to put it is it is an array denoting which data stores are there in the resources.
If we use the diff sync between seed nodes we create a race condition where when a seednode gets updated to the new system, it does not sync up properly with the other seed nodes. And that would be fatal. So for the time being, when a seednode asks for data, it uses the "old" big requests with all object keys. Should not be a problem for now since they have enough bandwidth.
CI does have troubles with tests which do file operations. Thus, these tests have been disabled. They have been useful during development and are useful for testing locally, though.
This test addresses the migration scenario where a user does not upgrade on the first possible occation (to the first Bisq version that has the new database structure in place) but does so later.
…m/chimp1984/bisq into chimp-reduce_initial_request_size
AppendOnlyDataStoreService does not have a map but aggregates the data form its services.
We use a set at the filter so it is not deterministic which item get truncated.
If we get logs from windows users the missing line breaks makes it harder to read.
…-versioned resource file anymore. Instead use readStore(); to create the live store. Add more logs
…esources. Improve logs
chimp1984
changed the title
|
@sqrrm This PR is ready for review and testing now. |
Closed
# Conflicts: # core/src/main/java/bisq/core/trade/statistics/TradeStatistics2Store.java # p2p/src/main/java/bisq/network/p2p/network/Connection.java
|
Replaced by #4586 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is based on #4233 from @freimair
It applies the feature only to tradeStatistics to reduce risks in case there are issues.