Write testcases for message cache

[hubcio]

In iggy-server messages cache works in per partition manner. When partition object is created, and cache is enabled, it creates internal SmartCacheobject, which is essentially wrapper around VecDeque<Arc<Messages>>. In other words, this is variable length ring buffer.

Messages in cache have to be coherent and sequential in terms of offset.

Cache has global (for whole server executable, for all partitions) usage counter, of which maximum value (in bytes) is interpreted from config. It's accessible via CacheMemoryTracker. When message is received by server, it always ends up in cache. If cache is full, eviction happens.

At the server startup, it loads messages from disk (if any data is present) to cache of each partition proportionally:

In terms of execution, it looks something like:

Same formula is used at runtime, when cache is evicted. (proportionally from each partition)
During cache eviction memory usage looks something like:

Purpose of this task is to write a testcase, which would check if cache works fine.

Suggested OK scenario:

• initialize System using TestSetup (see integration/tests/streaming/topic_messages.rs) with enabled cache
• append some messages
• see if cache has these messages
• append some more, above cache limit
• see if cache has evicted part of messages and cache size is below limit

Suggested negative scenario:

• initialize SmartCache with some bad messages (incoherent cache, non sequential messages offsets)
• initialize System using TestSetup and above SmartCache
• append some messages
• see if cache has emptied because cache coherency check failed

These are just general suggestions.

[vdnhi]

I can help with this issue. @hubcio Please help to provide more detail.

[hubcio]

@vdnhi I updated issue with description.