feat(tiering): Simple OpManager

[dranikpg]

OpManager is a higher level abstraction over DiskStorage that provides:

• Tracking reads and comibing reads for the same offset
• Tracking stashes and cancelling them
• Enqueuing deletes after reads
• Running the resolve loop after reading an entry

[adiholden]

I know this is still a draft. I just want to make sure I understand correctly, you current implementation does not takes into account that we want to group multiple small objects into a single page blob right?
f.e in here we will need to check which page this offset belongs page = offset & ~4095ULL
and then in the actions we will need to fill copy the string from the read result in offset - page with the relevant length right?

[dranikpg]

Yes, currently it doesn't, I will insert it right here in the future

[dranikpg]

Actually I might need to redesign it a little so it fits with small key accumulators

[dranikpg]

The plan is as follows:

the components managing small keys and gluing together pages will also use the OpManager for safe IO. Because they don't have any direct keys, they will refer to their operations with numeric identifiers.

Please review the design, I'll add a simple test in the meantime.

[romange]

btw, please make sure the whole build works on MacOS and linux specific parts are ignored on macos. See what I did with current tiered code

[romange]

please rebase as well.

[dranikpg]

Rebased, added one more test for delete after reads

[romange]

you loose the value offset, so how do you read the value from the page?
seems like a bug that requires a test

[dranikpg]

In Read() I pass it twice

PrepareRead(segment).ForId(id, segment)

and the first modified segment isn't stored anywhere

ProcessRead will calculate it correctly

[romange]

wow, it's confusing by reading the code in the function.
suggestion: call the argument aligned_segment, dcheck on it and pass the aligned segment in Read.

Another thing - it will fail for large lengths like 4095 because you never align length. Unless you also handle it somewhere 😆

[dranikpg]

Added a test for the case you pointed out - no bugs found 🙂

[dranikpg]

In Read() I pass it twice

PrepareRead(segment).ForId(id, segment)

and the first modified segment isn't stored anywhere

ProcessRead will calculate it correctly

[romange]

I gave you comments that are not addressed yet