Skip to content

perf(state): lean entity model + kwok scale lab - #2

Merged
ErfanY merged 2 commits into
mainfrom
perf/lean-entity-model
Jun 24, 2026
Merged

perf(state): lean entity model + kwok scale lab#2
ErfanY merged 2 commits into
mainfrom
perf/lean-entity-model

Conversation

@ErfanY

@ErfanY ErfanY commented Jun 24, 2026

Copy link
Copy Markdown
Owner

First slice of the stable-release refactor (readonly-first, 20+ clusters / 10k pods/cluster). Measurement-driven: a kwok scale lab + headless bench, then the biggest memory win.

What's here

Phase 0 — scale lab & measurement

  • scripts/scale/ (kwok): 20 nodes, 20 namespaces, 10,000 pods, 20-context kubeconfig, churn generator. Tunable via KRUST_* env.
  • Hidden --bench mode (src/ui/app/bench.rs) boots the real data plane and times project/render/pulse hot paths + RSS.

Phase 1.1 — lean entity model

  • Drop full object JSON per row (ResourceEntity.raw); replace with a compact Extracted payload computed once at ingest (cluster/extract.rs).
  • Full object fetched on demand for detail/describe/decode/edit via new ResourceProvider::get_object, single-slot detail_cache.
  • Pulse aggregate + log fan-in owner mapping read extracted fields instead of walking raw JSON.

Measured (kwok 10k pods, 1 context, release build)

Metric Before After
RSS 482 MB 85.5 MB (5.6× smaller)
pulse aggregate 7.62 ms 0.36 ms (21× faster)
projection / render unchanged (Phase 2.1)
initial sync unchanged — per-event to_value (Phase 1.2)

Validation

fmt · check · test (71 passed / 3 ignored) · release build — all green.

Notes

  • Net −255 lines of product code despite adding the feature.
  • Design: docs/design/phase-1.1-lean-entity-model.md; tracker: ROADMAP.md.
  • Not targeting a release; merging to main triggers release.yml, so merge when ready to cut a version.

🤖 Generated with Claude Code

ErfanY and others added 2 commits June 24, 2026 11:00
Local high-scale harness (scripts/scale/) using kwok to simulate large
clusters with near-zero compute: fake nodes report huge capacity and fake
pods reach Running instantly.

- up/down: create/delete the kwok cluster (docker runtime)
- load: 20 nodes, 20 namespaces, 10,000 pods (+ rs/svc/cm/secrets), tunable
  via KRUST_* env vars (total = NAMESPACES * DEPLOYS_PER_NS * REPLICAS)
- contexts: generate a 20-context kubeconfig (distinct default namespaces)
  to exercise multi-context / warm-context paths against one apiserver
- churn: rolling deployment restarts at a configurable rate
- gitignore the generated kubeconfig (embeds kwok cluster certs)

Used to capture the stable-release performance baselines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stop retaining the full object JSON for every cached entity. At 10k pods in
one context the store held ~482 MB (~48 KB/pod) because ResourceEntity.raw
kept the entire serialized object.

- Replace ResourceEntity.raw with a compact Extracted payload (node name,
  container names, ownerReferences, pod resource req/lim, node capacity),
  computed once at ingest in cluster/extract.rs and dropped.
- Fetch the full object on demand for detail/describe/decode/edit via the new
  ResourceProvider::get_object, cached single-slot (detail_cache).
- Pulse aggregate and log fan-in owner mapping now read extracted fields
  instead of walking raw JSON for every pod.

Also adds a hidden headless benchmark mode (--bench, src/ui/app/bench.rs) that
boots the real data plane against a cluster and times the project/render/pulse
hot paths + RSS, plus the stable-release ROADMAP and the 1.1 design note.

Measured on the kwok 10k-pod lab (1 context, release build):
  RSS              482 MB -> 85.5 MB  (5.6x smaller)
  pulse aggregate  7.62 ms -> 0.36 ms (21x faster)
projection/render unchanged (Phase 2.1); initial sync unchanged — per-event
to_value remains, addressed in Phase 1.2.

Validation: fmt, check, test (71 passed / 3 ignored), release build all green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ErfanY
ErfanY merged commit 4517541 into main Jun 24, 2026
1 check passed
@ErfanY
ErfanY deleted the perf/lean-entity-model branch June 24, 2026 09:11
ErfanY added a commit that referenced this pull request Jun 25, 2026
In the logs pane, `P` (shift+p) toggles between the current and the previous
(terminated) container instance's logs — `kubectl logs -p`, for debugging
crashlooping containers. Closes gap #2 from the k9s parity audit (4.7).

- One-shot fetch: previous logs don't follow and don't auto-reconnect (the
  container is gone), so the reconnect loop is skipped while previous is active.
- Refetches on toggle (clears the buffer); a fresh log view always starts on the
  current instance.
- Status line shows `instance:current|previous`.

The PodLogRequest.previous plumbing already existed; this wires the UI to it.

Test: toggle flips the flag, resets the buffer, and the title reflects the
instance.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant