Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Trie Node Retrieve and Save #1569

Merged
merged 8 commits into from
Sep 6, 2021
Merged

Improve Trie Node Retrieve and Save #1569

merged 8 commits into from
Sep 6, 2021

Conversation

Vovchyk
Copy link
Contributor

@Vovchyk Vovchyk commented Jun 30, 2021

Description

It removes a trie node set, add an internal boolean flag to each trie node to knows if the node was already saved, improve the retrieval of the node (no get hash calculation is needed for adding the retrieved node to the removed set), improve save trie without visiting unloaded child nodes

Originated from #1526

Motivation and Context

It solves #1509 and #1510

How Has This Been Tested?

Additionally to the usual code tests, the execution of blocks from a testnet node was executed:

java -cp rskj-core-3.0.0-triefix-all.jar -Dlogback.configurationFile= -Dblockchain.flushNumberOfBlocks=1000 -Dcache.states.max-elements=1000000 co.rsk.cli.tools.ExecuteBlocks 1672391 1673139 --testnet
There are new logs, to obtain some additional information from TrieStoreImpl and DataSourceWithCache. The above command was executed with

<logger name="triestore" level="TRACE"/>
<logger name="datasourcewithcache" level="TRACE"/>
<logger name="execute" level="TRACE"/>
<logger name="blockvalidator" level="TRACE"/>
<logger name="blocksyncservice" level="TRACE"/>
<logger name="blockexecutor" level="TRACE"/>

Only for testing purpose.

Some additional logs obtained:

2021-05-07-16:52:33.317 TRACE [triestore] Start saving trie root.
2021-05-07-16:52:33.318 TRACE [triestore] Start saving trie, level : 0
2021-05-07-16:52:57.923 TRACE [triestore] End saving trie, level : 0, already saved.
2021-05-07-16:52:57.924 TRACE [triestore] End saving trie root. No. Retrieves: 396. No. Saves: 0. No. No Saves: 1
2021-05-07-16:52:57.924 TRACE [triestore] End process block. No. Retrieves: 968. No. Saves: 0. No. No Saves: 1
2021-05-07-16:52:57.925 TRACE [datasourcewithcache] Activity: No. Gets: 969. No. Puts: 0. No. Gets from Store: 967

With this additional logs we have more information about the effectivity of having a cache (and the Trie getHash issue #1511 was discovered using these logs)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • Tests for the changes have been added (for bug fixes / features)
  • Requires Activation Code (Hard Fork)
  • Other information:

@@ -55,9 +60,17 @@ public TrieStoreImpl(KeyValueDataSource store) {
*/
@Override
public void save(Trie trie) {
noRetrieves = 0;
noSaves = 0;
noNoSaves = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find the no prefix confusing, noNoSaves sounds like a double negation. Consider changing for numberNoSaves?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefixed those fields with numOf

@nagarev nagarev self-requested a review September 3, 2021 20:01
nagarev
nagarev previously approved these changes Sep 3, 2021
Copy link
Contributor

@nagarev nagarev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sonarcloud
Copy link

sonarcloud bot commented Sep 6, 2021

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 4 Code Smells

78.7% 78.7% Coverage
0.0% 0.0% Duplication

@Vovchyk
Copy link
Contributor Author

Vovchyk commented Sep 6, 2021

pipeline: run

@nagarev nagarev self-requested a review September 6, 2021 13:02
Copy link
Contributor

@nagarev nagarev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Vovchyk Vovchyk merged commit 194f0bb into master Sep 6, 2021
@Vovchyk Vovchyk deleted the ajl-triefix branch September 6, 2021 13:25
@aeidelman aeidelman added this to the Iris v3.1.0 milestone Oct 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants