Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup/remove effect of memory hint to write buffer and adjust some db parameters #6767

Merged
merged 13 commits into from Mar 9, 2024

Conversation

asdacap
Copy link
Contributor

@asdacap asdacap commented Feb 23, 2024

Partially addresses #4040

  • Remove the effect of memory hint to the write buffer size of the databases. Memory hint still effect size of rowcache and shared block cache.
  • This is because write buffer size is quite significant to the database compaction as it effect the size of l0 file.
  • Also, @benaadams seems to have better performance with lower memory hint. It could be due to the write buffer instead of the block cache. It could be the bloom filter of the write buffer. It could be the size of the write buffer. Its hard to know for sure when both of them are binded to the same memory hint number.
  • Additionally, expose some more config which can have an effect when upgrading to new rocksdb.
  • Also expose the AdviseRandomOnOpen option. It looks like this does not have an effect on windows. So disabling it on linux and halfpath might improve performance which explain why @benaadams have much better performance. This definitely will cause hashdb to slowdown.
  • Retuned BlockNumbers and CodeDb for low block size and have some row cache.
  • StateDb's level multiplier and base size not changed as increasing it to have 3 level causes slightly slower sync. Will change in halfpath/flatdb as they naturally have lower write amplification rate.

Types of changes

What types of changes does your code introduce?

  • Refactoring

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Notes on testing

  • Very slightly reduced time during code lookup.

@benaadams
Copy link
Member

Also, @benaadams seems to have better performance with lower memory hint.

Was better during block processing generally, but terrible during OldReceipts blockprocessing

@benaadams
Copy link
Member

Also, @benaadams seems to have better performance with lower memory hint.

Was better during block processing generally, but terrible during OldReceipts blockprocessing

I tried to share Env between rocksdb databases to see if that resolved it; but just caused access violations so don't think I was doing it right 🤣

@asdacap asdacap marked this pull request as ready for review February 23, 2024 12:12
@asdacap asdacap requested a review from rubo as a code owner February 23, 2024 12:12
@asdacap
Copy link
Contributor Author

asdacap commented Mar 4, 2024

Reducing block cache size definitely improve block processing time. IOPs measured from process increased, but IOPs measured from SSD decreased, meaning it make better use of OS cache. Using direct IO have the opposite effect as direct IO bypass os cache.

@asdacap asdacap changed the title Cleanup/remove effect of memory hint Cleanup/remove effect of memory hint to write buffer and adjust some db parameters Mar 4, 2024
@asdacap
Copy link
Contributor Author

asdacap commented Mar 4, 2024

It is still unclear why its not the case with the default block cache. It could be that by hitting the SSD more, the OS can make better decision on what to cache.

@asdacap asdacap merged commit 1391b73 into master Mar 9, 2024
67 checks passed
@asdacap asdacap deleted the cleanup/remove-effect-of-memory-hint branch March 9, 2024 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants