Storage rewrite

[agazso]

There are several well-known issues that are related to the storage internals. The plan is that instead of addressing them one-by-one there would be a project to refactor the internals in a way that would also eliminate the known issues.

The list of known issues that is planned to be fixed with the rewrite:

• Running out of disk space Running out of disk space bee#1258
• UnreserveBatch incorrectly unpins chunks UnreserveBatch incorrectly unpins chunks bee#3037
• Sharky leaks without a shard limit sharky leaks without a shard limit bee#2992
• Localstore write in-place inconsistency bug localstore write-in-place inconsistency bug bee#3039
• localstore and storage radius change discrepancy localstore and storage radius change discrepancy  #46
• leftover chunks in the reserve after all of the batches have expired nodes still have chunks in the counted reserves after the testnet was wiped clean by the expiration of all the batches #49

There are a few other ones which are not directly a storage issue, however when rewriting the storage it would be good to take them into consideration as well:

• Reduce light node disk space requirement Reduce light node disk space requirement bee#2990
• One-to-many relationship between chunks and postage stamps One-to-many relationship between chunks and postage stamps bee#2850
• sharky shard defragmentation tooling feature request: localstore/sharky does not give back disk space after chunks are removed. #50

[agazso]

This is tracked in ethersphere/bee#3171