LTS : Implement more efficient space reclamation after truncation.

[sachin-j-joshi]

Problem description
Currently the truncation is implemented at the granularity of chunks. Only the chunks that are completely below truncate offset are are removed. For the chunks that are partially contain valid data (i.e truncate offset in the middle of chunk) such chunks are not deleted. The head portion of the chunk is still occupying storage.

This is a problem if the chunks end up being really big.
In this case we will not actually reclaim any space at all.

Problem location
ChunkedSegmentStorage::truncate

Suggestions for an improvement

One obvious solution is to create a small chunk, copy valid portion of old chunk to it and then delete the old chunk.
However we need to be careful not to unnecessarily move data again and again. Table and attribute segments are often appended to and truncated at front - moving data in such cases every time we truncate will result in poor performance.
Ideal the space reclamation is done as background job.
Develop and implement some basic heuristics to trigger (or not trigger ) such background on specific segments.