utils: Move cstore column and frame implementation to utils

[Lazin]

This PR simply moving the code from cloud_storage/segment_meta_cstore.h to the utils/delta_for.h so it could be reused more easily.

The tests/benchmarks are also separated and moved to the new location in utils.

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v23.3.x
• v23.2.x
• v23.1.x

Release Notes

• none

[dotnwat]

i wonder if this duplication of max_frame_size points at a slightly different decomposition of the types being possible?

for example, would it make logical sense for the deltafor_column to expose its configured max size as a constexpr field and then the column_impl would use that?

i'm assuming that it would be problematic for the two max sizes to be different?

[Lazin]

I don't think it's problematic because the value is pushed from top to bottom during instantiation. The deltafor_column instantiates deltafor_column_impl which actually uses the value. The stuff in the column_store doesn't really care about the value.

[andrwng]

Since the actual impl of append() is in the _impl class, I don't think making a constexpr field will change much, since it still needs to get passed to the _impl. I dont feel strongly about it at all, but perhaps an alternative is something like:

• deltafor column: owns the _frames, is also an appender and a seeker, and instantiates them to point at _frames
• frame appender: references the frames, appends to the column, templatized on max_frame_size, uses it to roll new frames, identical impl regardless of algo
• frame seeker: references the frames, templatized on algo, seeks based on which algo is templatized

Though I'm not convinced further decomposition adds much beyond maybe it clarifies the roll of the frame size, and solidifies the difference the algos' different seek implementations.

[Lazin]

I didn't want to change the logic much. This PR only moves things around. Changes like this can be done in a followup.

[vbotbuildovich]

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43332#018c93a5-36b0-4534-a3e1-848d5a395331

[dotnwat]

what changed?

[Lazin]

what changed?

I forgot to git add the delt_for_bench.cc in first push.

[andrwng]

Just a couple nits and some clarifying questions, but mostly lgtm

[andrwng]

nit: comment here needs an update too

[Lazin]

fixed

[andrwng]

column for monotonic sequences.

Curious, what guarantee or failure modes exist with regards to monotonicity of appends? If the caller doesn't append in order, what breaks? If it's unexpected usage, it's probably worth adding a warning in the description, maybe even an assert?

[Lazin]

I think that currently it will trigger an assertion

[andrwng]

nit: since we're trying to make this a bit more generic, perhaps it's worth calling out the expected limitations that non-segment-cstore users may need to consider, like the fact that pred_search will return the first frame that matches the predicate.

[andrwng]

Since the actual impl of append() is in the _impl class, I don't think making a constexpr field will change much, since it still needs to get passed to the _impl. I dont feel strongly about it at all, but perhaps an alternative is something like:

• deltafor column: owns the _frames, is also an appender and a seeker, and instantiates them to point at _frames
• frame appender: references the frames, appends to the column, templatized on max_frame_size, uses it to roll new frames, identical impl regardless of algo
• frame seeker: references the frames, templatized on algo, seeks based on which algo is templatized

Though I'm not convinced further decomposition adds much beyond maybe it clarifies the roll of the frame size, and solidifies the difference the algos' different seek implementations.