Add RLI access cache to support efficient lookup

[danny0405]

Task Description

What needs to be done:

• Add the basic cache layer abstraction
• Introduce hotspot entries cache: cache A
• Introduce inflight entries cache(for uncommitted checkpoint): cache B
• Introce smart eviction strategy for cache A & B
• support point queries with optional mini-batch access(one access with a set of keys)
• add tests

Why this task is needed:

ms level point query access is required for streaming, otherwise it's very prone to trigger backpressure then volatility of the pipeline.

Task Type

Code improvement/refactoring

Related Issues

Parent feature issue: (if applicable )
#17647