Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explain that mempool memory is added to -dbcache #13676

Merged
merged 1 commit into from Feb 21, 2019

Conversation

@Sjors
Copy link
Member

@Sjors Sjors commented Jul 16, 2018

Since -maxmempool is 450 MB by default it's quite possible for a user to accidentally OOM a low
memory device if they increase -dbcache beyond the default.

schermafbeelding 2018-09-06 om 17 02 40

@fanquake fanquake added the Docs label Jul 16, 2018
src/init.cpp Outdated
@@ -363,7 +363,7 @@ void SetupServerArgs()
gArgs.AddArg("-conf=<file>", strprintf("Specify configuration file. Relative paths will be prefixed by datadir location. (default: %s)", BITCOIN_CONF_FILENAME), false, OptionsCategory::OPTIONS);
gArgs.AddArg("-datadir=<dir>", "Specify data directory", false, OptionsCategory::OPTIONS);
gArgs.AddArg("-dbbatchsize", strprintf("Maximum database write batch size in bytes (default: %u)", nDefaultDbBatchSize), true, OptionsCategory::OPTIONS);
gArgs.AddArg("-dbcache=<n>", strprintf("Set database cache size in megabytes (%d to %d, default: %d)", nMinDbCache, nMaxDbCache, nDefaultDbCache), false, OptionsCategory::OPTIONS);
gArgs.AddArg("-dbcache=<n>", strprintf("Set database cache size in megabytes. During initial sync memory from the mempool is added (see -maxmempool). (%d to %d, default: %d)", nMinDbCache, nMaxDbCache, nDefaultDbCache), false, OptionsCategory::OPTIONS);
Copy link
Member

@promag promag Jul 16, 2018

@Sjors Sjors force-pushed the 2018/07/dbcache-ibd-mempool branch from a9b242c to 2f631f0 Jul 17, 2018
@Sjors
Copy link
Member Author

@Sjors Sjors commented Jul 17, 2018

Tweaked the message to use "initial block download" instead of "initial sync" and use <n> megabytes convention. Also keeping (4 to ..., default: 450) directly after the first sentence, instead of at the very end (I think that's more readable).

@skeees
Copy link
Contributor

@skeees skeees commented Aug 1, 2018

Would the same OOM issue also be encountered during periods of time when mempools are full and not just in IBD? Maybe just clarify that the process can use up to dbcache + maxmempool space?

@jonasschnelli
Copy link
Contributor

@jonasschnelli jonasschnelli commented Aug 2, 2018

Agree with @skeees.
Users need to be aware that MAXMEMPOOLLIMIT+DBCACHE is what they should count for, regardless of IBD/In-sync.

But the – in this PR – updated -dbcache message seems to make it more clear.

utACK 2f631f0

@DrahtBot
Copy link
Contributor

@DrahtBot DrahtBot commented Aug 29, 2018

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

@leishman
Copy link
Contributor

@leishman leishman commented Sep 5, 2018

I personally don't think the message is clear enough to prevent users from over-allocating. Perhaps something along the lines of: "During initial block download, the amount of memory used may be up to maxmempoollimit + dbcache". ? Or something like this?

@sipa
Copy link
Member

@sipa sipa commented Sep 5, 2018

That's not just the case during IBD.

At any point in time, the total memory is limited by the headers in memory, network buffers, validation caches, UTXO set cache, mempool, plus some overhead.

When the mempool limit is not reached, the UTXO set cache is allowed to grow and use the mempool space as well, but this shouldn't affect users' expectations of memory usage. It'd also independent of IBD or not.

@Sjors Sjors force-pushed the 2018/07/dbcache-ibd-mempool branch from 2f631f0 to 6cf3cf6 Sep 6, 2018
@Sjors
Copy link
Member Author

@Sjors Sjors commented Sep 6, 2018

Ok, I tried a new wording:

  -dbcache=<n>
       Maximum database cache size <n> megabytes (4 to 16384, default: 450). In
       addition, unused mempool memory is shared for this cache (see
       -maxmempool).

@Sjors Sjors changed the title Explain that mempool memory is added to -dbcache during IBD Explain that mempool memory is added to -dbcache Sep 6, 2018
@Sjors Sjors force-pushed the 2018/07/dbcache-ibd-mempool branch from 6cf3cf6 to 7cb1a14 Jan 31, 2019
@Sjors
Copy link
Member Author

@Sjors Sjors commented Jan 31, 2019

Rebased after #15163, using MiB now.

@laanwj laanwj merged commit 7cb1a14 into bitcoin:master Feb 21, 2019
1 of 2 checks passed
laanwj added a commit that referenced this issue Feb 21, 2019
7cb1a14 Explain that unused mempool memory is added to -dbcache (Sjors Provoost)

Pull request description:

  Since `-maxmempool` is 450 MB by default it's quite possible for a user to accidentally OOM a low
  memory device if they increase `-dbcache` beyond the default.

  <img width="563" alt="schermafbeelding 2018-09-06 om 17 02 40" src="https://user-images.githubusercontent.com/10217/45166219-c9c4f700-b1f6-11e8-9ee5-14b8b3a9830b.png">

Tree-SHA512: 44c7419d0b06c14aee5d2c02a41e5da488bcb40a5f65ba24554a45707b222f1e4b03d42486dfef9336d917ac2990eef2b1aec287a75b3ef1ccca0e88ac86a0c0
@Sjors Sjors deleted the 2018/07/dbcache-ibd-mempool branch Feb 21, 2019
vijaydasmp added a commit to vijaydasmp/dash that referenced this issue Sep 5, 2021
7cb1a14 Explain that unused mempool memory is added to -dbcache (Sjors Provoost)

Pull request description:

  Since `-maxmempool` is 450 MB by default it's quite possible for a user to accidentally OOM a low
  memory device if they increase `-dbcache` beyond the default.

  <img width="563" alt="schermafbeelding 2018-09-06 om 17 02 40" src="https://user-images.githubusercontent.com/10217/45166219-c9c4f700-b1f6-11e8-9ee5-14b8b3a9830b.png">

Tree-SHA512: 44c7419d0b06c14aee5d2c02a41e5da488bcb40a5f65ba24554a45707b222f1e4b03d42486dfef9336d917ac2990eef2b1aec287a75b3ef1ccca0e88ac86a0c0
vijaydasmp added a commit to vijaydasmp/dash that referenced this issue Sep 6, 2021
7cb1a14 Explain that unused mempool memory is added to -dbcache (Sjors Provoost)

Pull request description:

  Since `-maxmempool` is 450 MB by default it's quite possible for a user to accidentally OOM a low
  memory device if they increase `-dbcache` beyond the default.

  <img width="563" alt="schermafbeelding 2018-09-06 om 17 02 40" src="https://user-images.githubusercontent.com/10217/45166219-c9c4f700-b1f6-11e8-9ee5-14b8b3a9830b.png">

Tree-SHA512: 44c7419d0b06c14aee5d2c02a41e5da488bcb40a5f65ba24554a45707b222f1e4b03d42486dfef9336d917ac2990eef2b1aec287a75b3ef1ccca0e88ac86a0c0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

10 participants