Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try to use posix_fadvise with CBufferedFile #14485

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

@luke-jr
Copy link
Member

@luke-jr luke-jr commented Oct 15, 2018

Resurrecting #12491, since it seemed fine and desirable...

@meshcollider
Copy link
Member

@meshcollider meshcollider commented Oct 15, 2018

Concept ACK, would be good to hear from @eklitzke why the original was closed though

src/util.h Outdated
//! Close a file and return the result of fclose(). On POSIX systems that
//! support it, advise the kernel to remove the file contents from the page
//! cache (which can help on memory-constrained systems).
int CloseAndDiscard(FILE *file);
Copy link
Member

@laanwj laanwj Oct 18, 2018

"discard" sounds to me as if writes to the file will be discarded, too
maybe CloseAndUncache?

@laanwj
Copy link
Member

@laanwj laanwj commented Oct 18, 2018

Concept ACK though I'd really like to see benchmarks for changes like this, to know if this has any impact in practice worth making the risk/complication of the change.

@jgarzik
Copy link
Contributor

@jgarzik jgarzik commented Oct 18, 2018

Probably worthing having -- rough concept ACK -- however:

  • The double-lseek will be slower than fstat() on some networked or detachable filesystems.

  • If close-and-uncache is used on just-written-to files, this action is a no-op unless a sync (fdatasync etc.) preceded it.

  • Definitely worth benchmarking on multiple filesystems to see the positive or negative value of this change.

@luke-jr
Copy link
Member Author

@luke-jr luke-jr commented Feb 11, 2019

Rebased and addressed @laanwj's function rename request.

src/util/system.cpp Outdated Show resolved Hide resolved
@DrahtBot
Copy link
Contributor

@DrahtBot DrahtBot commented Mar 23, 2019

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #20839 (fuzz: Avoid extraneous copy of input data, using Span<> by MarcoFalke)
  • #20464 (refactor: Treat CDataStream bytes as uint8_t by MarcoFalke)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@DrahtBot
Copy link
Contributor

@DrahtBot DrahtBot commented Sep 30, 2019

There hasn't been much activity lately and the patch still needs rebase, so I am closing this for now. Please let me know when you want to continue working on this, so the pull request can be re-opened.

@luke-jr
Copy link
Member Author

@luke-jr luke-jr commented Oct 13, 2019

Rebased

Copy link
Contributor

@vasild vasild left a comment

My gut feel tells me that this is good, but experience tells me that it needs some performance testing to demonstrate that it really brings a boost. Sometimes such changes have surprisingly adverse effects or no effects and end up being just code clutter.

src/util/system.cpp Outdated Show resolved Hide resolved
src/util/system.cpp Outdated Show resolved Hide resolved
src/util/system.cpp Show resolved Hide resolved
src/util/system.cpp Show resolved Hide resolved
@luke-jr
Copy link
Member Author

@luke-jr luke-jr commented May 7, 2020

Rebased & comments addressed

This primarily affects blocks when bitcoin is launched with -reindex, as
that causes the block files to be loaded as CBufferedFile objects one at
a time as the reindex progresses.

Co-Authored-By: Luke Dashjr <luke-jr+git@utopios.org>
@DrahtBot
Copy link
Contributor

@DrahtBot DrahtBot commented Feb 1, 2021

🐙 This pull request conflicts with the target branch and needs rebase.

Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".

@practicalswift
Copy link
Contributor

@practicalswift practicalswift commented Feb 1, 2021

Adding to the three previous requests for benchmarks:

Do we have any benchmarks showing the practical gains in swiftness? :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

9 participants