[Enhancement] Optimize LiteConsumerLagCalculator.getLagCountTopK by deferring expensive getStoreTimestamp to topK results only

[f1amingo]

Before Creating the Enhancement Request

• I have confirmed that this should be classified as an enhancement rather than a bug/feature.

Summary

LiteConsumerLagCalculator.getLagCountTopK currently calls getStoreTimestamp() (disk I/O) for every offset entry during full-table traversal, creates a LiteLagInfo object for each entry regardless of whether it enters the topK heap, and uses String.split() for key parsing. These cause unnecessary overhead when the offset table is large.

Motivation

In production with thousands of lite topics per group, the current implementation issues N getStoreTimestamp calls (each hitting the message store), allocates N intermediate objects, and performs N string splits — but only K (topK, typically small) results are actually needed. This wastes CPU and I/O, especially under periodic metric collection.

Describe the Solution You'd Like

1. Defer getStoreTimestamp to topK results only: Accumulate consumerOffset in LiteLagInfo during traversal, then compute timestamps for the final topK entries after heap filtering — reducing N expensive calls to K.
2. Lazy object creation: Only instantiate LiteLagInfo when the entry qualifies for heap insertion (size < topK || diff > peek), avoiding N-K wasted allocations.
3. Replace split() with indexOf(): Parse topicAtGroup key via indexOf + substring, avoiding array allocation; check lite prefix before any string extraction.
4. Replace AtomicLong with long[]: The traversal is single-threaded; atomic overhead is unnecessary.

Describe Alternatives You've Considered

• Caching store timestamps in memory: adds complexity and staleness risk without proportional benefit.
• Parallel timestamp lookup: over-engineering for a metric query that is already fast after deferral.

Additional Context

Files changed: LiteConsumerLagCalculator.java, LiteLagInfo.java.

[oss-taishan-ai]

Issue Evaluation

Category: type/enhancement | Status: Evaluated

Feasibility: Feasible — the proposed optimizations are well-scoped and target clear hotspots in the getLagCountTopK hot path.

Scope: LiteConsumerLagCalculator.java, LiteLagInfo.java (metrics module only, no cross-module impact)

Compatibility: No breaking changes — internal optimization, no API or protocol changes.

Assessment:

The four proposed optimizations are all valid and well-justified:

1. Defer getStoreTimestamp to topK results only — The most impactful change. getStoreTimestamp calls getMessageStoreTimeStamp() which hits the message store. Reducing N calls to K (where K is typically small, e.g., 3-10) is a significant I/O reduction for groups with thousands of lite topics.

2. Lazy object creation — Avoids N-K unnecessary LiteLagInfo allocations that are immediately discarded by the heap.

3. Replace split() with indexOf() — Eliminates array allocation per entry. The indexOf + substring approach is a well-known Java optimization for this pattern.

4. Replace AtomicLong with long[] — Correct since offsetTableForEachByGroup is single-threaded. Removes unnecessary CAS overhead.

PR #10424 implements all four optimizations correctly.

---

Automated evaluation by github-manager-bot