Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explain that mempool memory is added to -dbcache #13676

Merged
merged 1 commit into from Feb 21, 2019

Conversation

Projects
None yet
10 participants
@Sjors
Copy link
Member

commented Jul 16, 2018

Since -maxmempool is 450 MB by default it's quite possible for a user to accidentally OOM a low
memory device if they increase -dbcache beyond the default.

schermafbeelding 2018-09-06 om 17 02 40

@fanquake fanquake added the Docs label Jul 16, 2018

src/init.cpp Outdated
@@ -363,7 +363,7 @@ void SetupServerArgs()
gArgs.AddArg("-conf=<file>", strprintf("Specify configuration file. Relative paths will be prefixed by datadir location. (default: %s)", BITCOIN_CONF_FILENAME), false, OptionsCategory::OPTIONS);
gArgs.AddArg("-datadir=<dir>", "Specify data directory", false, OptionsCategory::OPTIONS);
gArgs.AddArg("-dbbatchsize", strprintf("Maximum database write batch size in bytes (default: %u)", nDefaultDbBatchSize), true, OptionsCategory::OPTIONS);
gArgs.AddArg("-dbcache=<n>", strprintf("Set database cache size in megabytes (%d to %d, default: %d)", nMinDbCache, nMaxDbCache, nDefaultDbCache), false, OptionsCategory::OPTIONS);
gArgs.AddArg("-dbcache=<n>", strprintf("Set database cache size in megabytes. During initial sync memory from the mempool is added (see -maxmempool). (%d to %d, default: %d)", nMinDbCache, nMaxDbCache, nDefaultDbCache), false, OptionsCategory::OPTIONS);

This comment has been minimized.

@Sjors Sjors force-pushed the Sjors:2018/07/dbcache-ibd-mempool branch Jul 17, 2018

@Sjors

This comment has been minimized.

Copy link
Member Author

commented Jul 17, 2018

Tweaked the message to use "initial block download" instead of "initial sync" and use <n> megabytes convention. Also keeping (4 to ..., default: 450) directly after the first sentence, instead of at the very end (I think that's more readable).

@skeees

This comment has been minimized.

Copy link
Member

commented Aug 1, 2018

Would the same OOM issue also be encountered during periods of time when mempools are full and not just in IBD? Maybe just clarify that the process can use up to dbcache + maxmempool space?

@jonasschnelli

This comment has been minimized.

Copy link
Member

commented Aug 2, 2018

Agree with @skeees.
Users need to be aware that MAXMEMPOOLLIMIT+DBCACHE is what they should count for, regardless of IBD/In-sync.

But the – in this PR – updated -dbcache message seems to make it more clear.

utACK 2f631f0

@DrahtBot

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2018

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

@leishman

This comment has been minimized.

Copy link
Contributor

commented Sep 5, 2018

I personally don't think the message is clear enough to prevent users from over-allocating. Perhaps something along the lines of: "During initial block download, the amount of memory used may be up to maxmempoollimit + dbcache". ? Or something like this?

@sipa

This comment has been minimized.

Copy link
Member

commented Sep 5, 2018

That's not just the case during IBD.

At any point in time, the total memory is limited by the headers in memory, network buffers, validation caches, UTXO set cache, mempool, plus some overhead.

When the mempool limit is not reached, the UTXO set cache is allowed to grow and use the mempool space as well, but this shouldn't affect users' expectations of memory usage. It'd also independent of IBD or not.

@Sjors Sjors force-pushed the Sjors:2018/07/dbcache-ibd-mempool branch to 6cf3cf6 Sep 6, 2018

@Sjors

This comment has been minimized.

Copy link
Member Author

commented Sep 6, 2018

Ok, I tried a new wording:

  -dbcache=<n>
       Maximum database cache size <n> megabytes (4 to 16384, default: 450). In
       addition, unused mempool memory is shared for this cache (see
       -maxmempool).

@Sjors Sjors changed the title Explain that mempool memory is added to -dbcache during IBD Explain that mempool memory is added to -dbcache Sep 6, 2018

@DrahtBot DrahtBot removed the Needs rebase label Sep 6, 2018

@Sjors Sjors force-pushed the Sjors:2018/07/dbcache-ibd-mempool branch from 6cf3cf6 to 7cb1a14 Jan 31, 2019

@Sjors

This comment has been minimized.

Copy link
Member Author

commented Jan 31, 2019

Rebased after #15163, using MiB now.

@DrahtBot DrahtBot removed the Needs rebase label Jan 31, 2019

@laanwj laanwj merged commit 7cb1a14 into bitcoin:master Feb 21, 2019

1 of 2 checks passed

continuous-integration/appveyor/pr AppVeyor build failed
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request Feb 21, 2019

Merge #13676: Explain that mempool memory is added to -dbcache
7cb1a14 Explain that unused mempool memory is added to -dbcache (Sjors Provoost)

Pull request description:

  Since `-maxmempool` is 450 MB by default it's quite possible for a user to accidentally OOM a low
  memory device if they increase `-dbcache` beyond the default.

  <img width="563" alt="schermafbeelding 2018-09-06 om 17 02 40" src="https://user-images.githubusercontent.com/10217/45166219-c9c4f700-b1f6-11e8-9ee5-14b8b3a9830b.png">

Tree-SHA512: 44c7419d0b06c14aee5d2c02a41e5da488bcb40a5f65ba24554a45707b222f1e4b03d42486dfef9336d917ac2990eef2b1aec287a75b3ef1ccca0e88ac86a0c0

@Sjors Sjors deleted the Sjors:2018/07/dbcache-ibd-mempool branch Feb 21, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.