Storage: Enforce size based retention for topic

[sehz]

The current retention policy only supports the time-based property. This may result in a incorrect estimate of the total data size. We should add max data size properties as part of retention.

The topic should have new property: 'max_size' to indicate maximum data size will be allowed to accumulate.

[ExternalReality]

Hi @sehz, I'd like to take a look at this issue. At first glace it looks to be a good first issue that is beyond just a simple CLI fix. Would it be a good idea to work this one as a new contributor?

[sehz]

Sounds good

[galibey]

@sehz Do we really want to have max size per topic or per partition will suffice? Per topic size calculation will require communication between nodes whereas per partition can be done locally.

[sehz]

Per partition make sense but specified at topic level