Pruning does not respect specified disk space under bigger blocks

[ftrader]

I raise this issue because there is something about the pruning feature that requires fixing.

The current implementation of it is based on an assumption that blocks are <= 1MB , and under this assumption it manages the disk space used "satisfactorily", shall we say.

If blocks were to become 2MB big, it could use up twice the space that the user specified.
If blocks were to become even bigger -- you see the problem.
This can obviously lead to disk space problems on nodes where space has been budgeted expressly for the Bitcoin client, and should not be exceeded.

There is no ideal fix for this, due to the construction / design of the pruning function. It currently keeps a minimum number of blocks.

If blocks get bigger, either it will have to keep less blocks as a minimum, or it will have to exceed the pruning size. Either way, it should alert the user about this condition, so that they could decide whether to increase the pruned size, or whether they are ok with keeping less bigger blocks around under their specified size constraints.

So to fix this, I plan to make the 'minimum number of blocks to keep' a configuration item, based on the current default of 288 blocks (see code in comment below).
If the pruned set (using configured minimum) exceeds the configured disk space (prune=<size> parameter) then the user should be alerted. If the disk space runs out the client should shut down safely in any case - this needs to be verified though - it might not be the case yet.

I appreciate any suggestions / input on improving upon this further.

[ftrader]

Relevant code section in main.h which describes the current assumptions / limitations:

/** Number of MiB of block files that we're trying to stay below. */        
extern uint64_t nPruneTarget;                                               
/** Block files containing a block-height within MIN_BLOCKS_TO_KEEP of chainActive.Tip() will not be pruned. */
static const unsigned int MIN_BLOCKS_TO_KEEP = 288;                         
                                                                            
static const signed int DEFAULT_CHECKBLOCKS = MIN_BLOCKS_TO_KEEP;           
static const unsigned int DEFAULT_CHECKLEVEL = 3;                           
                                                                            
// Require that user allocate at least 550MB for block & undo files (blk???.dat and rev???.dat)
// At 1MB per block, 288 blocks = 288MB.                                    
// Add 15% for Undo data = 331MB                                            
// Add 20% for Orphan block rate = 397MB                                    
// We want the low water mark after pruning to be at least 397 MB and since we prune in
// full block file chunks, we need the high water mark which triggers the prune to be
// one 128MB block file + added 15% undo data = 147MB greater for a total of 545MB
// Setting the target to > than 550MB will make it likely we can respect the target.
static const uint64_t MIN_DISK_SPACE_FOR_BLOCK_FILES = 550 * 1024 * 1024;

This quite good final comment shows the derivation of the 550MB documented minimum pruning size.
Of course, this goes out the window under the >1MB blocks assumption.