Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create Entity Map Warmer For Performance #6223

Merged
merged 14 commits into from Apr 28, 2023
Merged

Create Entity Map Warmer For Performance #6223

merged 14 commits into from Apr 28, 2023
+899 −3

Conversation

mhess-swl
Copy link
Member

Closes #5868

@mhess-swl
Create new cache warm threads property
330d5f8 
Signed-off-by: Matt Hess <matt.hess@swirldslabs.com>
@mhess-swl
Create entity map warmer to pre-warm objects for cache
c4cbbb8 
Signed-off-by: Matt Hess <matt.hess@swirldslabs.com>
@mhess-swl mhess-swl added CI:UnitTests Performance Issues related to performance concerns. labels Apr 21, 2023
@mhess-swl mhess-swl added this to the v0.38 milestone Apr 21, 2023
@mhess-swl mhess-swl self-assigned this Apr 21, 2023
@mhess-swl
Remove extra empty line
9729902 
Signed-off-by: Matt Hess <matt.hess@swirldslabs.com>
@mhess-swl mhess-swl marked this pull request as ready for review April 21, 2023 20:00
@mhess-swl mhess-swl requested review from a team as code owners April 21, 2023 20:00
@github-actions
Copy link

github-actions bot commented Apr 21, 2023 •
edited

Node: Unit Test Results

  1 320 files    1 320 suites   1h 16m 46s ⏱️
97 224 tests 97 217 ✔️ 7 💤 0 ❌
98 856 runs  98 849 ✔️ 7 💤 0 ❌

Results for commit 47e27d0.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Apr 21, 2023 •
edited

Node: Integration Test Results

    3 files      3 suites   14m 56s ⏱️
151 tests 151 ✔️ 0 💤 0 ❌
152 runs  152 ✔️ 0 💤 0 ❌

Results for commit 47e27d0.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Apr 21, 2023 •
edited

Node: E2E Test Results

    1 files      1 suites   17m 6s ⏱️
309 tests 309 ✔️ 0 💤 0 ❌
327 runs  327 ✔️ 0 💤 0 ❌

Results for commit 47e27d0.

♻️ This comment has been updated with latest results.

netopyr
netopyr previously approved these changes Apr 24, 2023
View reviewed changes
Copy link
Contributor

@netopyr netopyr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in regard to hedera-app-spi and hedera-app

@mhess-swl
Remove unneeded round-labeled runnable for designated task purges
3eb4882 
Signed-off-by: Matt Hess <matt.hess@swirldslabs.com>
OlegMazurov
OlegMazurov previously approved these changes Apr 26, 2023
View reviewed changes
tinker-michaelj
tinker-michaelj previously approved these changes Apr 26, 2023
View reviewed changes
Copy link
Collaborator

@tinker-michaelj tinker-michaelj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

I have some doubts about warming the "root node" of the sender's ownedNfts linked list; but if superfluous warm() calls have a small penalty, probably not a blocker.

@mhess-swl
Incorporate other feedback
2b04114 
Signed-off-by: Matt Hess <matt.hess@swirldslabs.com>
@mhess-swl
Fix broken test
4179d69 
Signed-off-by: Matt Hess <matt.hess@swirldslabs.com>
OlegMazurov
OlegMazurov previously approved these changes Apr 27, 2023
View reviewed changes
tinker-michaelj
tinker-michaelj previously approved these changes Apr 27, 2023
View reviewed changes
Copy link
Collaborator

@tinker-michaelj tinker-michaelj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mhess-swl
Remove static initializer and fix object graph roots test
b59e492 
Signed-off-by: Matt Hess <matt.hess@swirldslabs.com>
@codecov
Copy link

codecov bot commented Apr 27, 2023 •
edited

Codecov Report

Patch coverage: 85.71% and project coverage change: +23.23 🎉

Comparison is base (ec0701e) 67.85% compared to head (47e27d0) 91.09%.

Additional details and impacted files 
@@              Coverage Diff               @@
##             develop    #6223       +/-   ##
==============================================
+ Coverage      67.85%   91.09%   +23.23%     
+ Complexity     22059    17086     -4973     
==============================================
  Files           1985     1295      -690     
  Lines         134976    49092    -85884     
  Branches        7591     4926     -2665     
==============================================
- Hits           91585    44718    -46867     
+ Misses         41914     3435    -38479     
+ Partials        1477      939      -538
Impacted Files Coverage Δ
...p/state/merkle/adapters/VirtualMapLikeAdapter.java 93.10% <0.00%> (-6.90%) ⬇️
...a/node/app/service/mono/cache/EntityMapWarmer.java 86.08% <86.08%> (ø)
...om/hedera/node/app/service/mono/ServicesState.java 82.45% <100.00%> (+0.05%) ⬆️
...e/mono/context/properties/BootstrapProperties.java 97.50% <100.00%> (+26.96%) ⬆️
...no/context/properties/GlobalDynamicProperties.java 98.90% <100.00%> (+16.84%) ⬆️
...pp/service/mono/state/adapters/VirtualMapLike.java 100.00% <100.00%> (ø)
...ervice/mono/state/virtual/EntityNumVirtualKey.java 100.00% <100.00%> (ø)

... and 923 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@mhess-swl
Remove unused instance variable
a27f151 
Signed-off-by: Matt Hess <matt.hess@swirldslabs.com>
@mhess-swl
Merge branch 'develop' into 05868_entity-map-warmer
47e27d0 
Signed-off-by: Matt Hess <matt.hess@swirldslabs.com>

# Conflicts:
#	hedera-node/hedera-app/src/test/resources/bootstrap.properties
#	hedera-node/hedera-mono-service/src/main/java/com/hedera/node/app/service/mono/context/properties/GlobalDynamicProperties.java
#	hedera-node/hedera-mono-service/src/main/resources/bootstrap.properties
#	hedera-node/hedera-mono-service/src/test/java/com/hedera/node/app/service/mono/context/properties/BootstrapPropertiesTest.java
#	hedera-node/hedera-mono-service/src/test/java/com/hedera/node/app/service/mono/context/properties/GlobalDynamicPropertiesTest.java
#	hedera-node/hedera-mono-service/src/test/resources/bootstrap.properties
#	hedera-node/hedera-mono-service/src/test/resources/bootstrap/standard.properties
@sonarcloud
Copy link

sonarcloud bot commented Apr 27, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

89.5% 89.5% Coverage
0.0% 0.0% Duplication

tinker-michaelj
tinker-michaelj approved these changes Apr 28, 2023
View reviewed changes
Copy link
Collaborator

@tinker-michaelj tinker-michaelj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @mhess-swl!

@mhess-swl mhess-swl merged commit 18c4000 into develop Apr 28, 2023
11 of 13 checks passed
@mhess-swl mhess-swl deleted the 05868_entity-map-warmer branch April 28, 2023 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@OlegMazurov OlegMazurov OlegMazurov approved these changes

@tinker-michaelj tinker-michaelj tinker-michaelj approved these changes

@kimbor kimbor Awaiting requested review from kimbor kimbor is a code owner automatically assigned from hashgraph/hedera-services

@isavov isavov Awaiting requested review from isavov isavov is a code owner automatically assigned from hashgraph/release-engineering

@nathanklick nathanklick Awaiting requested review from nathanklick nathanklick is a code owner automatically assigned from hashgraph/release-engineering-managers

@timo0 timo0 Awaiting requested review from timo0 timo0 was automatically assigned from hashgraph/platform-base

@jsync-swirlds jsync-swirlds Awaiting requested review from jsync-swirlds

@netopyr netopyr Awaiting requested review from netopyr

Assignees

@mhess-swl mhess-swl

Labels
Performance Issues related to performance concerns.
Projects
None yet
Milestone
v0.38
Development

Successfully merging this pull request may close these issues.

Port caching changes from the sig-reqs-vmap-warm branch to develop
5 participants
@mhess-swl @netopyr @OlegMazurov @tinker-michaelj @jsync-swirlds