feat(positron): serving glass box — the beat-WASTE control loop on screen (#141 slice 1) - #2076
Merged
Merged
Conversation
… ahead of decode (#282) Joel's steer: predict which experts will be hot BEFORE the inference needs them. Measured on the RUN-1 fixture, the strongest signal is not a prediction at all — prefill routing IS the decode hot set, extracted early: the prefill union covers 66% of every decode token's experts (vs 58% for full decode recency), and the prefill TAIL alone covers 47% at a third of the size. Stability selectors lose badly ("in all prefill batches" = 1.4%) — recency-of-context dominates even within one prompt, the same law RUN-1/RUN-2 measured across prompts. The cache-dynamics problem is the big prefill batches (8637 experts vs ~4416 slots) churning out decode-relevant experts before decode starts — so the moment to act is the boundary. - segment::PrefillBoundaryDetector — classifies the first decode-sized segment (size < 0.8× min seen) from segment sizes alone; fires once; decode-only serves never fire. Pinned against the fixture's exact sizes. - Driver publishes the plan IMMEDIATELY at the boundary (bandit is already seeded by the prefill batches) instead of waiting rewrite_every decode tokens — the warm-start window opens at decode token 1. Fixture replay verified live: boundary fires between the 3854 prefill tail and the first 1472 decode token; published plan carries 256 tier-0 pins + default_tier for the cold tail. 16/16 tests, clippy clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…reen (#141 slice 1) Joel's greenlight ("Take it. We are sota now"): the pager campaign's live telemetry becomes a positron widget on the proven metrics-gauge spine — one new view kind, one projector, one Lit panel, no new transport. Rust: - ServingViewState (kind="serving") in continuum-positron: header folded from the daemon's ServingSnapshot (model/ready/lanes/ctx/degraded — the process's own truth via current_serving(), zero new plumbing), pager series reusing MetricSeriesView (hit % / tok/s / fetch MB/s), bandit arm beliefs, and event cards for the discrete control-loop moments. - positron_serving_source: 2s emitter cloned from the metrics-gauge shape; tails the GGML_MOE_CAPTURE_FILE JSONL the serving binary already appends ("Positron tails it" — the only live feed today), offset-tail with raw-length truncation detection (consumed-offset alone misses a new serve that outgrows it — pinned by test), torn-tail guard, PagerCaptureEvent's serde(default)s decoding both the raw C++ perf feed and the future Rust decision feed. Event cards: serve-start, decay-switch, residency-shift (>1/8 jump). Arms labeled from the leaf crate's DECAY_ARMS ladder — data-driven, never invented. - Honest absence throughout: no capture env → header-only serving-health widget; no snapshot → awaiting line; never a fabricated gauge. TypeScript (the SysPanel spine end to end): - ServingPanelView in patterns; SERVING_KIND seam; servingWidget() projection + WorkspaceLive.serving + left-rail stack slot; <serving-panel> light-DOM Lit renderer (header line, renderGaugeBody sparklines, arm chips with reward bars, newest-first event cards); registry + host subscribe + ChatWidget property; TUI parity case; 4 view types vendored through the SDK barrels. Validation: 3/3 new projector tests + view kind test green; crate suites green; vendor-views in sync; typecheck clean except the pre-existing renderChat.spec process error; lint adds ZERO errors (baseline 61 pre-existing, my files clean); client test suites 36/36. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Joel's greenlight: the pager campaign's telemetry becomes a live positron widget — the self-improving serving stack visible in the rail, run by run.
What renders
ServingSnapshotviacurrent_serving()— zero new plumbing).GGML_MOE_CAPTURE_FILEper-token JSONL the serving binary already appends (the comment literally says "Positron tails it" — this is that consumer).DECAY_ARMSladder.How (the proven spine, no new transport)
ServingViewState (kind="serving")in continuum-positron →positron_serving_sourceemitter (2s, cloned from the metrics-gauge shape, offset-tail with raw-length truncation detection + torn-tail guard) → StateEnvelope →SERVING_KINDseam →servingWidget()projection →<serving-panel>(light-DOM Lit) + TUI parity case. 4 view types vendored through the SDK barrels.Honest-absence contract throughout: no capture feed → header-only serving-health widget; no snapshot → awaiting line; never a fabricated gauge.
Validation
cargo check -p continuum-core --features metal,accelerateclean; vendor-views in sync.renderChat.spec.tsprocess error. Client tests 36/36.Part of #141 (first slice) riding #282. Follow-ups named, not built: WatchPagerCaptureSink for the in-process ServingExpertPager feed (lands with the controller wiring), and
positron_arena_sourceto light the client-complete arena leaderboard as the ratchet-trends companion.🤖 Generated with Claude Code
https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo