feat: Add small cache to execution traces #10517
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fridrik01
force-pushed
the
10504-cache-execution-traces
branch
from
bc83c4f
to
3738433
Compare
raulk
requested changes
Mar 22, 2023
|
Updated the PR, waiting on my lotus to sync so I can test this |
arajasek
reviewed
Mar 23, 2023
5 tasks
This PR adds a small cache to calls to ExecutionTrace which helps improve performance for node operators like exchanges and block explorers. If items is in cache calls to this function will be 2-3x faster. Fixes: #10504
fridrik01
force-pushed
the
10504-cache-execution-traces
branch
from
877074a
to
2e45f6f
Compare
|
@raulk could you take another look at this PR so I can get it landed? |
raulk
approved these changes
Mar 28, 2023
| @@ -39,6 +40,8 @@ import ( | |||
| const LookbackNoLimit = api.LookbackNoLimit | |||
| const ReceiptAmtBitwidth = 3 | |||
|
|
|||
| const execTraceCacheSize = 16 | |||
There was a problem hiding this comment.
We probably want to make this configurable because SPs may want to disable it while exchanges may want to crank it up? We also are going with intuition for this value, so having ability to change it without a new build would help.
There was a problem hiding this comment.
Will make a followup PR which makes this tuneable via env variable so we can get this landed
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Issues
#10504
Proposed Changes
This PR adds a small cache to calls to
ExecutionTracewhich helps improve performance for node operators like exchanges and block explorers.If items is in cache calls to this function will be 2-3x faster.
Additional Info
I ran a local test for 30minutes where I called
lotus state compute-state --jsonin a loop from two threads.NOTE: Stress testing this over an hour resulted in Lotus process residual memory taking almost 30GB, I am kind concerned that it grew this large and GC didn't kick in, so will continue to stress test this for longer.
EDIT: Ran longer stress test, memory stayed flat so should be good
Checklist
Before you mark the PR ready for review, please make sure that:
<PR type>: <area>: <change being made>fix: mempool: Introduce a cache for valid signaturesPR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, testarea, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps