New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support resource files tagged by version to reduce initial data requests #4519
Support resource files tagged by version to reduce initial data requests #4519
Commits on Sep 12, 2020
-
Configuration menu - View commit details
-
Copy full SHA for cb9f0f0 - Browse repository at this point
Copy the full SHA cb9f0f0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3296db3 - Browse repository at this point
Copy the full SHA 3296db3View commit details -
Configuration menu - View commit details
-
Copy full SHA for e7831d6 - Browse repository at this point
Copy the full SHA e7831d6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 581286e - Browse repository at this point
Copy the full SHA 581286eView commit details -
Configuration menu - View commit details
-
Copy full SHA for fc7a43d - Browse repository at this point
Copy the full SHA fc7a43dView commit details -
Configuration menu - View commit details
-
Copy full SHA for d111d8c - Browse repository at this point
Copy the full SHA d111d8cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6eda9a9 - Browse repository at this point
Copy the full SHA 6eda9a9View commit details -
Configuration menu - View commit details
-
Copy full SHA for ff53f51 - Browse repository at this point
Copy the full SHA ff53f51View commit details -
Configuration menu - View commit details
-
Copy full SHA for 49731f9 - Browse repository at this point
Copy the full SHA 49731f9View commit details -
Configuration menu - View commit details
-
Copy full SHA for e123a0b - Browse repository at this point
Copy the full SHA e123a0bView commit details -
Configuration menu - View commit details
-
Copy full SHA for da5d41b - Browse repository at this point
Copy the full SHA da5d41bView commit details -
Configuration menu - View commit details
-
Copy full SHA for c7351a0 - Browse repository at this point
Copy the full SHA c7351a0View commit details -
Configuration menu - View commit details
-
Copy full SHA for e2e80d7 - Browse repository at this point
Copy the full SHA e2e80d7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 10c1ecc - Browse repository at this point
Copy the full SHA 10c1eccView commit details -
Configuration menu - View commit details
-
Copy full SHA for f6cea15 - Browse repository at this point
Copy the full SHA f6cea15View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0d80123 - Browse repository at this point
Copy the full SHA 0d80123View commit details -
Enforce version format and filter faulty ones
Instead of trying to find faulty keys, we stick to the mechanics we have in place. This way, we have less false positives. However, we enforce a certain Bisq version format, as parsing and business logic relies on that now.
Configuration menu - View commit details
-
Copy full SHA for aad57c3 - Browse repository at this point
Copy the full SHA aad57c3View commit details -
Configuration menu - View commit details
-
Copy full SHA for faca8e4 - Browse repository at this point
Copy the full SHA faca8e4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 72600b1 - Browse repository at this point
Copy the full SHA 72600b1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3ad395b - Browse repository at this point
Copy the full SHA 3ad395bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2589137 - Browse repository at this point
Copy the full SHA 2589137View commit details -
Configuration menu - View commit details
-
Copy full SHA for 34abad0 - Browse repository at this point
Copy the full SHA 34abad0View commit details -
Turns out, scanning resources does not work reliable enough. Thus, an array of Strings denoting historical Bisq versions it is. An other way to put it is it is an array denoting which data stores are there in the resources.
Configuration menu - View commit details
-
Copy full SHA for 62a77e3 - Browse repository at this point
Copy the full SHA 62a77e3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 39fcca5 - Browse repository at this point
Copy the full SHA 39fcca5View commit details -
Use full sync between seednodes
If we use the diff sync between seed nodes we create a race condition where when a seednode gets updated to the new system, it does not sync up properly with the other seed nodes. And that would be fatal. So for the time being, when a seednode asks for data, it uses the "old" big requests with all object keys. Should not be a problem for now since they have enough bandwidth.
Configuration menu - View commit details
-
Copy full SHA for a3628ff - Browse repository at this point
Copy the full SHA a3628ffView commit details -
Configuration menu - View commit details
-
Copy full SHA for a374249 - Browse repository at this point
Copy the full SHA a374249View commit details -
CI does have troubles with tests which do file operations. Thus, these tests have been disabled. They have been useful during development and are useful for testing locally, though.
Configuration menu - View commit details
-
Copy full SHA for 8e55f64 - Browse repository at this point
Copy the full SHA 8e55f64View commit details -
Configuration menu - View commit details
-
Copy full SHA for e927e6d - Browse repository at this point
Copy the full SHA e927e6dView commit details -
This test addresses the migration scenario where a user does not upgrade on the first possible occation (to the first Bisq version that has the new database structure in place) but does so later.
Configuration menu - View commit details
-
Copy full SHA for 2fb2232 - Browse repository at this point
Copy the full SHA 2fb2232View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5f8b7cf - Browse repository at this point
Copy the full SHA 5f8b7cfView commit details -
Configuration menu - View commit details
-
Copy full SHA for f8320fe - Browse repository at this point
Copy the full SHA f8320feView commit details -
Configuration menu - View commit details
-
Copy full SHA for 017bbfd - Browse repository at this point
Copy the full SHA 017bbfdView commit details -
Configuration menu - View commit details
-
Copy full SHA for d2a6655 - Browse repository at this point
Copy the full SHA d2a6655View commit details -
Fixed a potential issue when loading stores
Previously, we only checked if the newest store is there. If it is not, we copy and reload everything. If it is there, we are satisfied. Now, we check for each and every store we should have and copy and load them individually.
Configuration menu - View commit details
-
Copy full SHA for 469c374 - Browse repository at this point
Copy the full SHA 469c374View commit details -
Configuration menu - View commit details
-
Copy full SHA for 249e185 - Browse repository at this point
Copy the full SHA 249e185View commit details -
Configuration menu - View commit details
-
Copy full SHA for 06ec223 - Browse repository at this point
Copy the full SHA 06ec223View commit details -
Configuration menu - View commit details
-
Copy full SHA for eabadc0 - Browse repository at this point
Copy the full SHA eabadc0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 755fc03 - Browse repository at this point
Copy the full SHA 755fc03View commit details -
Configuration menu - View commit details
-
Copy full SHA for d218f2e - Browse repository at this point
Copy the full SHA d218f2eView commit details -
Configuration menu - View commit details
-
Copy full SHA for c34bfee - Browse repository at this point
Copy the full SHA c34bfeeView commit details -
Configuration menu - View commit details
-
Copy full SHA for d6f195d - Browse repository at this point
Copy the full SHA d6f195dView commit details -
Configuration menu - View commit details
-
Copy full SHA for d81bdab - Browse repository at this point
Copy the full SHA d81bdabView commit details -
Configuration menu - View commit details
-
Copy full SHA for abbfedc - Browse repository at this point
Copy the full SHA abbfedcView commit details -
Removed deprecated test mechanisms
Up until now, the tests needed to create a version history set for testing. These changes had to be made to a final static variable during runtime. Now, enough versions are in the history set anyways, so we could remove this part of the testing framework.
Configuration menu - View commit details
-
Copy full SHA for 5f94406 - Browse repository at this point
Copy the full SHA 5f94406View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0eccec7 - Browse repository at this point
Copy the full SHA 0eccec7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 270f1e7 - Browse repository at this point
Copy the full SHA 270f1e7View commit details -
Configuration menu - View commit details
-
Copy full SHA for c8c63ef - Browse repository at this point
Copy the full SHA c8c63efView commit details -
Configuration menu - View commit details
-
Copy full SHA for 76cc37e - Browse repository at this point
Copy the full SHA 76cc37eView commit details -
I thought I could get by using a fixed version string so we can have tests involing future data stores...
Configuration menu - View commit details
-
Copy full SHA for b4aae51 - Browse repository at this point
Copy the full SHA b4aae51View commit details
Commits on Sep 13, 2020
-
Merge branch 'master_upstream' into chimp-reduce_initial_request_size
# Conflicts: # core/src/main/java/bisq/core/support/dispute/agent/MultipleHolderNameDetection.java # core/src/main/java/bisq/core/trade/txproof/xmr/XmrTxProofService.java # p2p/src/main/resources/AccountAgeWitnessStore_BTC_MAINNET # p2p/src/main/resources/SignedWitnessStore_BTC_MAINNET # p2p/src/main/resources/TradeStatistics2Store_BTC_MAINNET
Configuration menu - View commit details
-
Copy full SHA for f83c147 - Browse repository at this point
Copy the full SHA f83c147View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5f8e779 - Browse repository at this point
Copy the full SHA 5f8e779View commit details -
Use readStore(); instead of store = readStore(getFileName());
storage.initAndGetPersisted and initAndGetPersistedWithFileName are not needed. Refactorings: - Extract variables for better readability - Rename variables - Split copyAndSplit to getStoreFromResource and pruneStore which describes better what is happening - Use getStore from base class, remove local readStore method
Configuration menu - View commit details
-
Copy full SHA for 3100888 - Browse repository at this point
Copy the full SHA 3100888View commit details -
Configuration menu - View commit details
-
Copy full SHA for 591c2fb - Browse repository at this point
Copy the full SHA 591c2fbView commit details -
Add mapOfHistoricalStores field
Avoid that we have to iterate all historical stores at each getMap call and create that map once when the historical data are created. Reduce visibility Make fields final (readFromResources is not called twice) Rename variable for better readability
Configuration menu - View commit details
-
Copy full SHA for d5c6644 - Browse repository at this point
Copy the full SHA d5c6644View commit details -
Revert critical data stores to current version.
We should not risk to break those critical data and it does not provide that much benefit as with the trade statistics. Lets test it first with trade stats and if all goes well extend in a later release.
Configuration menu - View commit details
-
Copy full SHA for 3dac4c4 - Browse repository at this point
Copy the full SHA 3dac4c4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2cef411 - Browse repository at this point
Copy the full SHA 2cef411View commit details -
Remove methods in MapStoreService and do instead instance of check in…
… AppendOnlyDataStoreService to call the new method only where it makes sense. Remove the hack of adding a "magic" key for communicating the version. Use a new field in the request instead. Remove the extra handling of seedNode as the impl. did not make any difference as the Version passed in `excludedKeys = getKeySetInBytes(this.appendOnlyDataStoreService.getMap("since " + Version.VERSION));` was same as in `if (!finalFilter.equals(Version.VERSION)) ` thus never doing anything there and the result maps is the same as in the seed node branch. Was not clear anyway what the motivation of a diff. handling is there? That seed nodes request always all data without filtering? Why?
Configuration menu - View commit details
-
Copy full SHA for 8458a88 - Browse repository at this point
Copy the full SHA 8458a88View commit details -
Add current version to version list. Makes it more clear that those a…
…re all historical immutable data. Add ignoreHistoricalData param to getMap Remove unused variables/params
Configuration menu - View commit details
-
Copy full SHA for 2ece512 - Browse repository at this point
Copy the full SHA 2ece512View commit details -
Configuration menu - View commit details
-
Copy full SHA for 965de7f - Browse repository at this point
Copy the full SHA 965de7fView commit details -
Rename SplitStore to PersistableNetworkPayload
The store has nothing to do with the splitting of live data and historical data. It only provides a getMap method with the PersistableNetworkPayload type Use that for all stores with such a type of map.
Configuration menu - View commit details
-
Copy full SHA for cfefae2 - Browse repository at this point
Copy the full SHA cfefae2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 28987f5 - Browse repository at this point
Copy the full SHA 28987f5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6cf054c - Browse repository at this point
Copy the full SHA 6cf054cView commit details -
Configuration menu - View commit details
-
Copy full SHA for aadd2e4 - Browse repository at this point
Copy the full SHA aadd2e4View commit details -
Fix param in test. Add Ignore to the 2 test classes handling the new …
…historical data store feature. @freimair Feel free to adjust the test to the new code.
Configuration menu - View commit details
-
Copy full SHA for 7a5ba0b - Browse repository at this point
Copy the full SHA 7a5ba0bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 224eaab - Browse repository at this point
Copy the full SHA 224eaabView commit details -
Revert changes at tests from prev version
Some tests are still failing. Need to look into it another day...
Configuration menu - View commit details
-
Copy full SHA for 98877b4 - Browse repository at this point
Copy the full SHA 98877b4View commit details
Commits on Sep 15, 2020
-
Merge branch 'master_upstream' into chimp-reduce_initial_request_size
# Conflicts: # build.gradle # desktop/package/linux/Dockerfile # desktop/package/linux/package.sh # desktop/package/linux/release.sh # desktop/package/macosx/create_app.sh # desktop/package/macosx/finalize.sh # desktop/package/windows/package.bat # desktop/package/windows/release.bat # relay/src/main/resources/version.txt
Configuration menu - View commit details
-
Copy full SHA for 32cf32c - Browse repository at this point
Copy the full SHA 32cf32cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8f2e4ca - Browse repository at this point
Copy the full SHA 8f2e4caView commit details
Commits on Sep 16, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 22fc97d - Browse repository at this point
Copy the full SHA 22fc97dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9c873a6 - Browse repository at this point
Copy the full SHA 9c873a6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3252a50 - Browse repository at this point
Copy the full SHA 3252a50View commit details -
Configuration menu - View commit details
-
Copy full SHA for a3ea54a - Browse repository at this point
Copy the full SHA a3ea54aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2b61563 - Browse repository at this point
Copy the full SHA 2b61563View commit details -
Configuration menu - View commit details
-
Copy full SHA for 996364e - Browse repository at this point
Copy the full SHA 996364eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4115c0c - Browse repository at this point
Copy the full SHA 4115c0cView commit details
Commits on Sep 17, 2020
-
Configuration menu - View commit details
-
Copy full SHA for ad95019 - Browse repository at this point
Copy the full SHA ad95019View commit details
Commits on Sep 19, 2020
-
Configuration menu - View commit details
-
Copy full SHA for e680c4d - Browse repository at this point
Copy the full SHA e680c4dView commit details
Commits on Sep 27, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 80bf8bf - Browse repository at this point
Copy the full SHA 80bf8bfView commit details -
Merge branch 'chimp-reduce_initial_request_size' of https://github.co…
…m/chimp1984/bisq into chimp-reduce_initial_request_size
Configuration menu - View commit details
-
Copy full SHA for 3a4784e - Browse repository at this point
Copy the full SHA 3a4784eView commit details -
Configuration menu - View commit details
-
Copy full SHA for c482a1b - Browse repository at this point
Copy the full SHA c482a1bView commit details
Commits on Sep 28, 2020
-
Fix mock class to reflect real class better.
AppendOnlyDataStoreService does not have a map but aggregates the data form its services.
Configuration menu - View commit details
-
Copy full SHA for ba0dc1b - Browse repository at this point
Copy the full SHA ba0dc1bView commit details -
We use a set at the filter so it is not deterministic which item get truncated.
Configuration menu - View commit details
-
Copy full SHA for f132a35 - Browse repository at this point
Copy the full SHA f132a35View commit details -
Remove todo, add system line break
If we get logs from windows users the missing line breaks makes it harder to read.
Configuration menu - View commit details
-
Copy full SHA for 6eb1efb - Browse repository at this point
Copy the full SHA 6eb1efbView commit details -
Configuration menu - View commit details
-
Copy full SHA for ee725bc - Browse repository at this point
Copy the full SHA ee725bcView commit details
Commits on Sep 29, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 82c5b5e - Browse repository at this point
Copy the full SHA 82c5b5eView commit details -
Do not call super.readFromResources(postFix); as we do not have a not…
…-versioned resource file anymore. Instead use readStore(); to create the live store. Add more logs
Configuration menu - View commit details
-
Copy full SHA for 171be24 - Browse repository at this point
Copy the full SHA 171be24View commit details -
Configuration menu - View commit details
-
Copy full SHA for bd38097 - Browse repository at this point
Copy the full SHA bd38097View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0feece6 - Browse repository at this point
Copy the full SHA 0feece6View commit details -
Only call pruneStore if we have created a new historical store from r…
…esources. Improve logs
Configuration menu - View commit details
-
Copy full SHA for e402c8a - Browse repository at this point
Copy the full SHA e402c8aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 620dfca - Browse repository at this point
Copy the full SHA 620dfcaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 90da37f - Browse repository at this point
Copy the full SHA 90da37fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9240eec - Browse repository at this point
Copy the full SHA 9240eecView commit details -
Configuration menu - View commit details
-
Copy full SHA for ab82286 - Browse repository at this point
Copy the full SHA ab82286View commit details
Commits on Oct 1, 2020
-
Merge branch 'master_upstream' into wip-merge
# Conflicts: # core/src/main/java/bisq/core/trade/statistics/TradeStatistics2Store.java # p2p/src/main/java/bisq/network/p2p/network/Connection.java
Configuration menu - View commit details
-
Copy full SHA for f2cc5a6 - Browse repository at this point
Copy the full SHA f2cc5a6View commit details