feat(ingestion): improve control plane UI for large deployments

[jose-sequeira]

Problem

The control plane UI doesn't scale to our larger deployments. Some topics have 1000 partitions, so the lagging partitions table becomes one giant unnavigable list, and the consumer groups table is heading the same way. The consumer debug pod list mixes pods from every ingestion lane into one flat table. And the per-pod debug view (batches, commits) is squeezed into a fixed-height iframe with nested scrollbars instead of the page expanding to fit the data.

Changes

• Add a reusable client-side search + pagination widget (listView) to the lag section. The consumer groups table filters by group or topic (25 per page), the partitions table filters by partition number (100 per page). The search input survives refreshes, and lag heat bars stay scaled against the full result set so paging doesn't recalibrate the colors.
• Group consumer debug pods by namespace, one section per ingestion lane, instead of a flat table with a namespace column.
• Auto-size the pod debug iframe to its content via a ResizeObserver (same origin through the control plane's proxy), so the outer page scrolls naturally instead of nesting scrollbars.
• In the embedded debug page, let the batches and commits views grow with their content instead of clamping to 100vh (which inside an iframe is the iframe's height, hence the squeeze). The live feed keeps a bounded 600px tail, decoupled from vh so it can't feed back into the iframe auto-sizing.

How did you test this code?

This is browser-only vanilla JS with no test harness, so I verified it by extracting the inline script and running node --check for syntax, plus a DOM-stub unit test exercising listView end to end (paging through 1000 partitions, substring search, empty states, page clamping when the row set shrinks, filter reset). I was not able to run the UI against a live control plane in this session.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

Docs update

Internal debug tooling, no user-facing docs affected.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Authored with Claude Code. José described the three pain points (huge partition lists, lanes mixed together, the batches view fitting its container instead of the container expanding) and I (Claude) implemented them. Notable decisions: heat bars scale against the full dataset rather than the visible page; the iframe auto-size measures body.scrollHeight rather than documentElement.scrollHeight because the latter is floored at the iframe's own height and would let the frame grow but never shrink; the live feed got a fixed pixel height instead of growing unbounded, since a vh-based clamp inside an auto-sizing iframe is a feedback loop.

[greptile-apps]

_{Reviews (1): Last reviewed commit: "feat(ingestion): improve control plane U..." | Re-trigger Greptile}

[trunk-io]

     

View Full Report ↗︎ ⋅ Docs

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-07-15 12:19 UTC	Run
prod-us	✅ Deployed	2026-07-15 12:43 UTC	Run
prod-eu	✅ Deployed	2026-07-15 12:45 UTC	Run