From a6270d33d88a4452b39b705ddfeccc2e55c420ef Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Tue, 28 Apr 2026 01:17:50 +0900
Subject: [PATCH 01/31] docs: propose KeyViz adapter / namespace labels
 (follow-up)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A user observed that on a default single-Raft-group deployment the
KeyViz heatmap shows one row covering all five adapters
(DynamoDB / Redis / S3 / SQS / RawKV) — the smallest unit of
attribution today is a Raft route, and one route covers
[empty, empty) by default. Splitting routes via SplitRange or
multi-group startup flags works as an operational workaround but
the most common shape (single-group) is opaque.

This proposal adds an independent label dimension to the sampler
so a single route can split into per-adapter sub-rows in the
heatmap, with zero hot-path penalty when labels are not configured.

Recommended approach (4.1): per-Observe label string; slot key
widens from routeID to (routeID, label). One extra map lookup per
Observe; legacy callers pass empty label and behaviour is
unchanged.

Forward-compatible wire-format extension: optional `label` on
KeyVizRow / MatrixRow; old SPAs ignore it, new SPAs render
"route:N / <label>" when present.

Five PRs of follow-up work outlined in 7. Implementation does
not start until this doc is reviewed and accepted.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 269 ++++++++++++++++++
 1 file changed, 269 insertions(+)
 create mode 100644 docs/design/2026_04_28_proposed_keyviz_adapter_labels.md

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
new file mode 100644
index 00000000..bae38382
--- /dev/null
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -0,0 +1,269 @@
+---
+status: proposed
+phase: keyviz / follow-up
+parent_design: docs/admin_ui_key_visualizer_design.md
+date: 2026-04-28
+---
+
+# KeyViz adapter / namespace labels
+
+## 1. Background
+
+Phase 2-A through 2-C ship a fully functional KeyViz heatmap, but
+the smallest unit of attribution is a **Raft route** — a contiguous
+key range owned by one group. When multiple adapters share a route
+(the default single-group config has every adapter writing into a
+single `[-∞, +∞)` route), the heatmap shows one row with the
+combined traffic and the operator cannot tell whether the spike
+came from DynamoDB, Redis, S3, SQS, or RawKV.
+
+A user observed this in production: 6-node cluster, fan-out
+returning a single row at `route:1` with `Total = 378` writes,
+covering all five adapters indistinguishably.
+
+The hotspot-shard-split design and the multi-group startup flags
+(`--raftRedisMap` / `--raftDynamoMap` / `--raftS3Map` /
+`--raftSqsMap`) already give operators a way to split traffic
+across distinct Raft groups so per-route attribution becomes
+per-adapter attribution. But that is an **operational** workaround:
+single-group deployments — the most common shape for first-time
+operators and small clusters — still get the all-traffic-in-one-row
+view.
+
+This proposal adds an **independent label dimension** to the
+sampler so a single Raft group can still surface per-adapter
+breakdown in the heatmap.
+
+## 2. Goals and non-goals
+
+### 2.1 Goals
+
+- Attribution **inside a single route**: a row that today reads
+  `route:1, total=378` should optionally split into sub-rows like
+  `route:1 / dynamo`, `route:1 / s3`, `route:1 / redis`, …
+- Zero hot-path penalty when labels are **not** configured: a
+  default deployment continues to behave exactly as today.
+- Adapter-side wiring is the natural place to set labels (every
+  adapter already has the dispatch entry into
+  `ShardedCoordinator.Observe…` — see `kv/sharded_coordinator.go`).
+  No global key-prefix table on the operator side.
+- Cluster fan-out merges per-(route, label) cells, not just
+  per-route — operators see the same per-adapter breakdown across
+  the whole cluster.
+
+### 2.2 Non-goals (deferred)
+
+- **Free-form custom labels** (per-table, per-bucket, per-queue).
+  This proposal stops at one level: the adapter family. A
+  follow-up can add a second dimension (e.g. `dynamo / users`,
+  `dynamo / orders`) once the wire format settles.
+- **Persistence**. Labels are in-memory like the rest of the
+  Phase 2 sampler.
+- **Per-key-byte attribution**. Sub-route classification is done
+  at the dispatch entry (the adapter knows its identity); we do
+  not attempt to classify by inspecting the key bytes.
+- **Backwards-incompatible wire-format changes**. The label is
+  added as an optional field; old SPAs against new servers keep
+  working.
+
+## 3. Surface
+
+The user-visible delta is one extra label on every heatmap row.
+Default config emits the empty label (legacy behaviour); when
+adapters set labels, the heatmap splits a single route into one
+row per (route, label) pair:
+
+```
+Today (single group, no labels):
+  Row 0  route:1  Total 378
+
+After (adapters tag their traffic):
+  Row 0  route:1 / dynamo   Total 142
+  Row 1  route:1 / redis    Total 200
+  Row 2  route:1 / sqs      Total 18
+  Row 3  route:1 / s3       Total 11
+  Row 4  route:1 / rawkv    Total  7
+```
+
+## 4. Options for label propagation
+
+Four ways to get the label from the adapter to the sampler. Pick
+one based on hot-path cost vs. plumbing weight.
+
+### 4.1 (Recommended) Per-Observe label string
+
+Extend the sampler signature:
+
+```go
+Observe(routeID uint64, op Op, keyLen, valueLen int, label string)
+```
+
+Empty `label` → existing behaviour, single row per route. Adapters
+set their own constant string (`"dynamo"`, `"redis"`, …) at the
+dispatch site they already own.
+
+Cost on hot path: one extra map lookup per Observe — slot is now
+keyed by `(routeID, label)` instead of just `routeID`. The map
+key is a struct of `{uint64, string}`; the `label` is a small
+interned constant (`"dynamo"`), so allocation should be zero.
+
+Storage: each (route, label) gets its own `routeSlot`. With 5
+adapters and a 1024-route budget, the worst case is 5120 slots
+per node — still well below the existing `MaxTrackedRoutes` cap
+when operators raise it for label use.
+
+### 4.2 Per-adapter sampler instance
+
+Wire one `*MemSampler` per adapter, each with a fixed label. The
+admin handler queries every sampler and concatenates the results.
+
+- Pro: zero hot-path code change in the existing sampler.
+- Con: every adapter gets its own ring buffer, history, and
+  retention machinery. Memory is N× higher and per-route
+  metadata duplicates across samplers.
+
+### 4.3 Per-key-prefix taxonomy (operator-configured)
+
+Static `{prefix → label}` map registered at startup. Sampler
+classifies each key at Observe time by prefix-matching.
+
+- Pro: no adapter wiring; works with any caller that goes through
+  `ShardedCoordinator`.
+- Con: prefix-match per Observe is a hot-path cost, and the
+  taxonomy is a new operator-facing config the design has been
+  careful to avoid.
+
+### 4.4 Hash the adapter into the route catalog
+
+Make `distribution.Route` carry an adapter label and route by
+adapter at the catalog layer. The sampler stays single-keyed.
+
+- Pro: solves attribution at the catalog level, where it actually
+  belongs.
+- Con: the catalog is the wrong place for this — adapters share
+  Raft groups by design, and forcing `Route` to carry adapter
+  identity bakes a different separation into the route topology.
+  Reverses the multi-group startup-flag story.
+
+**Recommendation: Option 4.1.** Lowest plumbing weight, smallest
+hot-path delta (one map lookup), and the label originates where
+it is most naturally available (the adapter's dispatch entry).
+The `routeSlot` map shape changes from `map[uint64]*routeSlot` to
+`map[slotKey]*routeSlot` with `slotKey = {uint64, string}`.
+
+## 5. Wire format extension
+
+`MatrixRow` (Go) and `KeyVizRow` (proto + JSON):
+
+```diff
+ type MatrixRow struct {
+   RouteID      uint64
++  Label        string  // optional; empty for legacy unlabelled traffic
+   Start, End   []byte
+   …
+ }
+```
+
+The SPA reads the label and renders rows as
+`route:<id> / <label>` when label is non-empty; legacy rows
+(empty label) render as `route:<id>` exactly like today.
+
+`bucket_id` formatting in the wire shape stays as-is; the label
+is a separate field. This keeps the `route_ids` / `aggregate`
+semantics from §5 of the parent design unchanged.
+
+Forward compatibility: an old SPA against a new server ignores
+the unknown `label` field and shows just the route as it always
+did. A new SPA against an old server sees no `label` field and
+falls back to the legacy formatting. Both directions are
+non-breaking.
+
+## 6. Aggregator merge changes
+
+The fan-out aggregator's per-cell merge key gains the label:
+
+- Phase 2-C (current): `(bucketID, raftGroupID, leaderTerm,
+  windowStart)` per design `2026_04_27_proposed_keyviz_cluster_fanout.md`
+  §4.
+- With labels: `(bucketID, label, raftGroupID, leaderTerm,
+  windowStart)`.
+
+Reads still sum, writes still max-with-conflict; nothing about
+the merge rules changes other than the key-tuple width.
+
+## 7. Implementation plan
+
+| PR | Scope |
+|---|---|
+| **PR-A** | Land this design doc. |
+| **PR-B** | Sampler API extension: `Observe(... label string)`, `RegisterRoute` no-op (label is per-Observe, not per-route). `MatrixRow.Label`. Update the existing tests to pass empty labels. |
+| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Default constants live in one file (`adapter/keyviz_labels.go`) so a future audit lists every label in one place. |
+| **PR-D** | Wire-format extension: proto + JSON `label` field; SPA renders `route:N / <label>` when present. |
+| **PR-E** | Aggregator merge key gains `label`. Mirrors PR-3 of the Phase 2-C+ track. |
+
+PRs B and C are independent of the wire format; the heatmap will
+keep showing one row per route until D ships, but the per-label
+counts are already accumulating in the sampler so the wire
+extension is "switch on the field".
+
+## 8. Five-lens checklist
+
+1. **Data loss** — n/a; per-Observe label is metadata. The
+   existing "no counts lost across flush" invariant
+   (`keyviz/sampler_test.go`) extends straightforwardly with the
+   label dimension; the `(routeID, label)` slot is still atomic-
+   add updated like the current `routeID` slot.
+2. **Concurrency / distributed** — slot-key change is contained
+   in the routesMu COW path; the hot-path Load + map lookup keeps
+   the same shape (one lookup, one atomic add). Burst test
+   updates: parametrise on (route, label) instead of just route.
+3. **Performance** — One extra map lookup per Observe via the
+   wider key. `BenchmarkObserveParallel` already pins the hot-path
+   cost; the new bench should land within run-to-run variance of
+   the current 4 ns/op. If it doesn't, the design is wrong and we
+   fall back to Option 4.2.
+4. **Data consistency** — Cluster fan-out merge gains a tuple
+   field; the dedup invariant (per-cell, per-(route, label,
+   group, term, window)) still holds. Old SPA against new server
+   sees the label-collapsed view; new SPA against old server
+   sees the legacy view; both are coherent.
+5. **Test coverage** — New test categories:
+   - `Observe(label="dynamo")` and `Observe(label="redis")`
+     against the same routeID produce two distinct rows in
+     `Snapshot`.
+   - Empty `label` matches no other rows (legacy behaviour pinned).
+   - Burst test: many goroutines hitting the same route with
+     different labels — exact-counting invariant must hold per
+     (route, label).
+   - Aggregator merge: same route, two labels, two nodes —
+     each label dedupes correctly without bleeding into the
+     other.
+
+## 9. Open questions
+
+1. **Should the label be hierarchical** (`dynamo / users`) from
+   day one, or restricted to a single segment now and extended
+   later? Proposal: single segment now (cheapest sampler change),
+   extend with a `/`-delimited convention later if adapters want
+   sub-tenant attribution.
+2. **Label allocation discipline** — who owns the canonical label
+   set? Proposal: `adapter/keyviz_labels.go` exports
+   `LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`,
+   `LabelRawKV` constants. Adapters refer to these; nothing
+   stops a future adapter from inventing its own label, but the
+   review burden of adding to the central file catches accidental
+   variants like `"DynamoDB"` vs `"dynamo"`.
+3. **Should the aggregator collapse same-route different-label
+   rows for operators who don't want the breakdown?** Proposal:
+   no — the SPA already lets operators pick which row to
+   examine; the wire form should always carry the breakdown so
+   the data is queryable.
+
+## 10. Out of scope (explicit deferrals)
+
+- Per-table / per-bucket / per-queue / per-Redis-DB sub-labels.
+- Operator-configurable label taxonomy.
+- Persistence of labelled rows (Phase 3 covers persistence
+  generally; labels ride along once the persistence path lands).
+- Adapter-aware splitting of routes (`SplitRange` triggered by
+  adapter-label hotspots) — that is a Phase 3+ idea.

From 3c14f29fdf9ee5bb06e45c51f48857db363be4b3 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Tue, 28 Apr 2026 01:42:07 +0900
Subject: [PATCH 02/31] docs: address PR #694 round-1 (Claude bot + Gemini)

Eight items from round-1 review on the adapter-labels proposal:

- 4.1.1 (new) slot lifecycle: pin the lockless invariant by
  having RegisterRoute pre-create one slot per known label at
  registration time. Add routeSlot.Label and document
  reclaimRetiredSlot dedupe on (RouteID, Label). Reject the two
  alternatives (silent fall-back and hot-path mutex) explicitly.
- 5 composite bucket_id: when Label is non-empty,
  bucket_id = "route:N:label" so it stays globally unique. Without
  this, pivotKeyVizColumns (keyed by RouteID alone) silently
  collapses same-route different-label rows back into one row.
  The pivot-key widening in internal/admin/keyviz_handler.go is
  now an explicit PR-D+E scope item.
- 5 separator: route-vs-label uses `:` (not `/`) so a future
  hierarchical label using `/` does not produce parser
  ambiguity. (Gemini minor.)
- 6 aggregator: clarify that the bucketID widening encodes the
  new label dimension; the merge tuple width does not change.
- 6.5 (new) virtual buckets: explicit deferral that virtual
  buckets always emit Label="" - per-label breakdown is only
  available below MaxTrackedRoutes.
- 7 PR-D and PR-E ship together: between them, the aggregator
  would mis-merge labelled rows (BucketID alone as merge key).
  Combine into a single PR-D+E. PR-B/C remain independent.
- 7 PR-C operator note: refuse to start with a warning when
  len(routes) x len(labels) > MaxTrackedRoutes - otherwise hot
  routes silently fold into the virtual bucket.
- 9 Q2: canonical constants live in keyviz/labels.go, not
  adapter/keyviz_labels.go - sampler must not import back to
  adapter (cycle).
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 154 +++++++++++++++---
 1 file changed, 128 insertions(+), 26 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index bae38382..8aa206bb 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -112,6 +112,52 @@ adapters and a 1024-route budget, the worst case is 5120 slots
 per node — still well below the existing `MaxTrackedRoutes` cap
 when operators raise it for label use.
 
+#### 4.1.1 Slot lifecycle and the lockless invariant
+
+The current `Observe` is `Load → map-lookup → atomic.Add` with no
+fallback: a miss is silently dropped. This is intentional —
+`RegisterRoute` pre-creates every slot before traffic arrives, so
+the hot path never needs to allocate. We keep that invariant by
+making `RegisterRoute` **pre-create one slot per known label** at
+registration time. The label set is the canonical
+`keyviz/labels.go` constants (§9 Q2); register-time slot
+count is `len(routes) × len(labels) + 1` (the +1 is the empty-
+label legacy slot, kept so callers that pass `label=""` still hit
+a slot). New labels added after a route is registered are not
+auto-bound; an operator deploying a new adapter must restart the
+sampler (matching the current "ApplySplit / ApplyMerge in a
+future PR" semantics for the static route catalog).
+
+`routeSlot` gains a `Label string` field so `Flush` /
+`appendDrainedRow` can read the label off the slot rather than
+threading it through a separate channel:
+
+```go
+type routeSlot struct {
+    metaMu  sync.RWMutex
+    RouteID uint64
+    Label   string         // new — empty for the legacy unlabelled slot
+    GroupID uint64         // Phase 2-C+, see PR-3a
+    Start, End []byte
+    ...
+}
+```
+
+`reclaimRetiredSlot` (`keyviz/sampler.go:710`) must dedupe on
+`(RouteID, Label)`, not `RouteID` alone — a slot retired as
+`(routeID, "dynamo")` and re-registered as `(routeID, "redis")`
+is **not** a reclaim candidate; they are different slots.
+
+Two alternatives we explicitly reject:
+
+- **First-seen miss-fall-back to empty label**: surprising silent
+  data-loss for the first Observe per (route, label) pair. Hard
+  to debug.
+- **First-seen creates a slot under mutex inside `Observe`**:
+  breaks the zero-mutex hot-path contract. Non-starter.
+
+(Reviewer notes from PR #694: Claude bot critical, Gemini medium.)
+
 ### 4.2 Per-adapter sampler instance
 
 Wire one `*MemSampler` per adapter, each with a fixed label. The
@@ -168,15 +214,44 @@ The SPA reads the label and renders rows as
 `route:<id> / <label>` when label is non-empty; legacy rows
 (empty label) render as `route:<id>` exactly like today.
 
-`bucket_id` formatting in the wire shape stays as-is; the label
-is a separate field. This keeps the `route_ids` / `aggregate`
-semantics from §5 of the parent design unchanged.
-
-Forward compatibility: an old SPA against a new server ignores
-the unknown `label` field and shows just the route as it always
-did. A new SPA against an old server sees no `label` field and
-falls back to the legacy formatting. Both directions are
-non-breaking.
+**`bucket_id` is composite when `Label` is non-empty**:
+
+- Legacy (empty label): `bucket_id = "route:<id>"` — unchanged
+  from the parent design.
+- Labelled: `bucket_id = "route:<id>:<label>"` (e.g.
+  `"route:1:dynamo"`).
+
+Composite `bucket_id` keeps the field globally unique across
+labelled rows. Without it, `pivotKeyVizColumns` (currently keyed
+by `RouteID` alone in `internal/admin/keyviz_handler.go`) would
+collapse `(routeID=1, label="dynamo")` and `(routeID=1,
+label="redis")` into the same map entry — silently re-merging
+the rows the feature is trying to split. The same uniqueness
+property is what `applyKeyVizRowBudget` and `sortKeyVizRowsByStart`
+need for deterministic tiebreak (both sort on `BucketID` last).
+
+`pivotKeyVizColumns` widens its pivot key from `uint64` (RouteID)
+to a struct of `(uint64, string)` to mirror the sampler's slot
+key — same shape, same reasoning as §4.1.1.
+
+We use `:` (not `/`) as the route-vs-label separator so a future
+hierarchical label (§9 Q1) can use `/` without ambiguity.
+`route:1:dynamo` is unambiguous; if labels later become
+`dynamo/users` the composite becomes `route:1:dynamo/users` and
+parsers split on the *first* `:` after `route`.
+
+`route_ids` / `aggregate` / virtual-bucket semantics from §5 of
+the parent design are unchanged — labels are an orthogonal axis
+to the route-coarsening machinery (and §6.5 below pins the
+virtual-bucket interaction).
+
+Forward compatibility: an old SPA against a new server sees a
+new `bucket_id` shape (`route:1:dynamo`) and renders it
+literally — operators get a less-pretty label but no
+correctness bug. A new SPA against an old server sees no `label`
+field and `bucket_id = "route:1"`, falling back to the legacy
+formatting. Both directions are non-breaking. (Reviewer notes
+from PR #694: Claude bot critical, Gemini high.)
 
 ## 6. Aggregator merge changes
 
@@ -185,26 +260,46 @@ The fan-out aggregator's per-cell merge key gains the label:
 - Phase 2-C (current): `(bucketID, raftGroupID, leaderTerm,
   windowStart)` per design `2026_04_27_proposed_keyviz_cluster_fanout.md`
   §4.
-- With labels: `(bucketID, label, raftGroupID, leaderTerm,
-  windowStart)`.
+- With labels: same tuple — but `bucketID` itself now carries the
+  label via the §5 composite (`route:1:dynamo`). The merge key
+  width does **not** change; the new label dimension is
+  encoded into `bucketID` so the aggregator already separates
+  same-route different-label rows correctly.
 
 Reads still sum, writes still max-with-conflict; nothing about
-the merge rules changes other than the key-tuple width.
+the merge rules changes other than the wire shape of `bucketID`.
+This is what makes PR-D and PR-E shippable in either order
+(see §7) — the wire-format change in PR-D *is* the merge-key
+change.
+
+### 6.5 Virtual buckets and labels
+
+Routes that overflow `MaxTrackedRoutes` fold into a virtual
+bucket (parent design §5.3). Virtual buckets aggregate counters
+across every route they swallow, including across **labels** —
+attempting to per-label-key a virtual bucket would defeat the
+"single coarsened slot" property the bucket exists for.
+
+Resolution: **virtual buckets always emit `Label = ""`**.
+Operators who want per-adapter breakdown for their hot routes
+must keep them under `MaxTrackedRoutes` (§7 PR-C calls this out).
+Virtual-bucket rows in the heatmap render as
+`virtual:<lineageID>` (no label suffix), exactly like today.
 
 ## 7. Implementation plan
 
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: `Observe(... label string)`, `RegisterRoute` no-op (label is per-Observe, not per-route). `MatrixRow.Label`. Update the existing tests to pass empty labels. |
-| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Default constants live in one file (`adapter/keyviz_labels.go`) so a future audit lists every label in one place. |
-| **PR-D** | Wire-format extension: proto + JSON `label` field; SPA renders `route:N / <label>` when present. |
-| **PR-E** | Aggregator merge key gains `label`. Mirrors PR-3 of the Phase 2-C+ track. |
+| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, `RegisterRoute` pre-creates one slot per known label (§4.1.1). `MatrixRow.Label`. `reclaimRetiredSlot` dedupes on `(RouteID, Label)`. Update existing tests to pass empty labels. |
+| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Canonical constants live in `keyviz/labels.go` (NOT `adapter/keyviz_labels.go` — sampler-side imports must not climb back to the adapter package; sampler's `RegisterRoute` reads the canonical set at registration time). **Operator note**: with N labels, slot count grows from M routes to ~N×M; raise `--keyvizMaxTrackedRoutes` proportionally before deploying or hot routes will fold into the virtual bucket and lose per-label breakdown silently. PR-C should refuse to start with a warning when `len(routes) × len(labels) > MaxTrackedRoutes`. |
+| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) in the same PR as the SPA `route:N / label` rendering AND the `pivotKeyVizColumns` pivot-key widening from `RouteID` to `(RouteID, Label)` in `internal/admin/keyviz_handler.go`. Splitting these creates a window where `pivotKeyVizColumns` collapses labelled rows back into a single row in the JSON response — exactly the bug this feature is fixing. The fan-out aggregator picks up the new behaviour for free because the merge key uses `bucketID` (now composite). |
 
 PRs B and C are independent of the wire format; the heatmap will
-keep showing one row per route until D ships, but the per-label
+keep showing one row per route until D+E ship, but the per-label
 counts are already accumulating in the sampler so the wire
-extension is "switch on the field".
+extension is "switch on the field". (Reviewer notes from PR
+#694: Claude bot critical / moderate.)
 
 ## 8. Five-lens checklist
 
@@ -241,18 +336,25 @@ extension is "switch on the field".
 
 ## 9. Open questions
 
-1. **Should the label be hierarchical** (`dynamo / users`) from
+1. **Should the label be hierarchical** (`dynamo/users`) from
    day one, or restricted to a single segment now and extended
    later? Proposal: single segment now (cheapest sampler change),
    extend with a `/`-delimited convention later if adapters want
-   sub-tenant attribution.
+   sub-tenant attribution. The route-vs-label separator in
+   `bucket_id` is `:` (`route:1:dynamo`), not `/`, so a future
+   `dynamo/users` extension produces `route:1:dynamo/users`
+   without parser ambiguity (split on the *first* `:` after
+   `route`). (Gemini round-1 minor on PR #694.)
 2. **Label allocation discipline** — who owns the canonical label
-   set? Proposal: `adapter/keyviz_labels.go` exports
-   `LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`,
-   `LabelRawKV` constants. Adapters refer to these; nothing
-   stops a future adapter from inventing its own label, but the
-   review burden of adding to the central file catches accidental
-   variants like `"DynamoDB"` vs `"dynamo"`.
+   set? Proposal: `keyviz/labels.go` exports `LabelDynamo`,
+   `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV` constants.
+   Constants live in the `keyviz` package (not `adapter/`) so
+   the sampler's `RegisterRoute` can read the full set at
+   registration time without an import cycle back to `adapter`.
+   Adapters refer to these constants; nothing stops a future
+   adapter from inventing its own label, but the review burden
+   of adding to the central file catches accidental variants
+   like `"DynamoDB"` vs `"dynamo"`.
 3. **Should the aggregator collapse same-route different-label
    rows for operators who don't want the breakdown?** Proposal:
    no — the SPA already lets operators pick which row to

From ce8f95bfc4c34d6a682c8a2728bcd9c3523efbb0 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Tue, 28 Apr 2026 01:51:39 +0900
Subject: [PATCH 03/31] docs: address PR #694 round-2 (Claude bot)

Four follow-ups from round-2:

- 7 PR-C: replace "refuse to start" with a structured slog.Warn
  emitted from inside RegisterRoute at coarsening time. Routes
  are dynamic (the catalog watcher calls RegisterRoute when a
  SplitRange lands), so a startup-time check would miss every
  route added after process start. The right operator signal is
  a log line at the moment a coarsening event fires.
- 4.1.1: spell out the coarsened-route path. RegisterRoute that
  returns false (route folded into a virtual bucket) does NOT
  pre-create labeled slots; the virtualForRoute fallback in
  Observe handles all labels uniformly per 6.5.
- 4.1.1: tighten the new-label rollout note. "Restart the
  sampler" was misleading because the canonical set lives at
  compile time; "deploy a new binary" is what operators actually
  need.
- 8 lens-5: add two missing test cases - slot pre-creation
  count (1 route -> N_labels+1 slots) and reclaimRetiredSlot
  (RouteID, Label) dedupe (retire dynamo + re-register redis
  must NOT reclaim). Plus the coarsened-route + label invariant
  (label="" on virtual buckets) for completeness.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 46 +++++++++++++++----
 1 file changed, 37 insertions(+), 9 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 8aa206bb..01439557 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -119,14 +119,26 @@ fallback: a miss is silently dropped. This is intentional —
 `RegisterRoute` pre-creates every slot before traffic arrives, so
 the hot path never needs to allocate. We keep that invariant by
 making `RegisterRoute` **pre-create one slot per known label** at
-registration time. The label set is the canonical
-`keyviz/labels.go` constants (§9 Q2); register-time slot
-count is `len(routes) × len(labels) + 1` (the +1 is the empty-
-label legacy slot, kept so callers that pass `label=""` still hit
-a slot). New labels added after a route is registered are not
-auto-bound; an operator deploying a new adapter must restart the
-sampler (matching the current "ApplySplit / ApplyMerge in a
-future PR" semantics for the static route catalog).
+registration time, but only on the individual-tracking path. The
+label set is the canonical `keyviz/labels.go` constants (§9 Q2);
+register-time slot count is `len(routes) × len(labels) + 1` (the
++1 is the empty-label legacy slot, kept so callers that pass
+`label=""` still hit a slot).
+
+When `RegisterRoute` decides a route will be coarsened into a
+virtual bucket (i.e. it returns `false`), **no labeled slots are
+created** for that route — the `tbl.virtualForRoute[routeID]`
+fallback in `Observe` already handles all labels uniformly per
+§6.5 (virtual buckets always emit `Label = ""`). Pre-creating
+labeled slots that would never be hit would just burn allocator
+work.
+
+New labels added after a route is registered are not auto-bound;
+an operator deploying a new adapter must **deploy a new binary**
+that includes the new constant (a process restart alone does
+nothing — the canonical set lives at compile time). Matches the
+current "ApplySplit / ApplyMerge in a future PR" semantics for
+the static route catalog.
 
 `routeSlot` gains a `Label string` field so `Flush` /
 `appendDrainedRow` can read the label off the slot rather than
@@ -292,7 +304,7 @@ Virtual-bucket rows in the heatmap render as
 |---|---|
 | **PR-A** | Land this design doc. |
 | **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, `RegisterRoute` pre-creates one slot per known label (§4.1.1). `MatrixRow.Label`. `reclaimRetiredSlot` dedupes on `(RouteID, Label)`. Update existing tests to pass empty labels. |
-| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Canonical constants live in `keyviz/labels.go` (NOT `adapter/keyviz_labels.go` — sampler-side imports must not climb back to the adapter package; sampler's `RegisterRoute` reads the canonical set at registration time). **Operator note**: with N labels, slot count grows from M routes to ~N×M; raise `--keyvizMaxTrackedRoutes` proportionally before deploying or hot routes will fold into the virtual bucket and lose per-label breakdown silently. PR-C should refuse to start with a warning when `len(routes) × len(labels) > MaxTrackedRoutes`. |
+| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Canonical constants live in `keyviz/labels.go` (NOT `adapter/keyviz_labels.go` — sampler-side imports must not climb back to the adapter package; sampler's `RegisterRoute` reads the canonical set at registration time). **Operator note**: with N labels, slot count grows from M routes to ~N×M; raise `--keyvizMaxTrackedRoutes` proportionally or hot routes will fold into the virtual bucket and lose per-label breakdown. PR-C emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so the operator sees the symptom in the log when it actually fires. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. |
 | **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) in the same PR as the SPA `route:N / label` rendering AND the `pivotKeyVizColumns` pivot-key widening from `RouteID` to `(RouteID, Label)` in `internal/admin/keyviz_handler.go`. Splitting these creates a window where `pivotKeyVizColumns` collapses labelled rows back into a single row in the JSON response — exactly the bug this feature is fixing. The fan-out aggregator picks up the new behaviour for free because the merge key uses `bucketID` (now composite). |
 
 PRs B and C are independent of the wire format; the heatmap will
@@ -323,6 +335,19 @@ extension is "switch on the field". (Reviewer notes from PR
    sees the label-collapsed view; new SPA against old server
    sees the legacy view; both are coherent.
 5. **Test coverage** — New test categories:
+   - **Slot pre-creation count**: register 1 route, assert
+     `len(tbl.slots) == len(labels) + 1` (N labeled + 1 legacy
+     empty-label slot). Catches a PR-B regression that skips a
+     label constant in the pre-creation loop — without this the
+     missing slot only surfaces when that adapter fires traffic.
+   - **`reclaimRetiredSlot` (RouteID, Label) dedupe**: retire
+     `(routeID, "dynamo")`, re-register `(routeID, "dynamo")`,
+     assert reclaim succeeds; then retire `(routeID, "dynamo")`
+     again, re-register as `(routeID, "redis")`, assert reclaim
+     does **not** fire — the dynamo slot must stay in the
+     retired list to be drained by its own grace window. Catches
+     a PR-B regression where the implementor forgets to widen
+     the dedupe key from `RouteID` alone.
    - `Observe(label="dynamo")` and `Observe(label="redis")`
      against the same routeID produce two distinct rows in
      `Snapshot`.
@@ -333,6 +358,9 @@ extension is "switch on the field". (Reviewer notes from PR
    - Aggregator merge: same route, two labels, two nodes —
      each label dedupes correctly without bleeding into the
      other.
+   - Coarsened-route + label: a route folded into the virtual
+     bucket emits `Label = ""` regardless of which label the
+     `Observe` carries (§6.5).
 
 ## 9. Open questions
 

From ae43bfe096c368d5e9d54dc1d1e9da8e9f0efcc0 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Tue, 28 Apr 2026 09:27:42 +0900
Subject: [PATCH 04/31] docs: address PR #694 round-3 (Claude bot)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Five follow-ups from round-3, all doc-level:

- 4.1.1 (moderate): pin virtualForRoute as map[uint64]*routeSlot
  (NOT widened to slotKey). Coarsened-route Observe falls back to
  this map on a labeled-slot miss, so widening the key would
  silently drop all coarsened-route traffic. Added pseudocode
  showing the exact lookup chain.
- 5 (moderate): add the explicit JSON KeyVizRow diff alongside
  MatrixRow and the proto. Carry the label as a dedicated field
  AND keep bucket_id composite — the field is for SPA ergonomics
  (no client-side parsing), bucket_id is the globally-unique
  row identifier merge logic uses.
- 9 Q2 (minor): name the exported AllLabels slice that
  RegisterRoute iterates over. The PR-B implementor needs an
  exported handle, not just the per-adapter constants.
- 7 PR-B (minor): coordinator call sites in
  kv/sharded_coordinator.go also need the label="" update; they
  are the only non-test caller of Sampler.Observe.
- 7 PR-C (minor): rollout discipline - raise
  --keyvizMaxTrackedRoutes before deploying the PR-C binary so
  startup doesn't flood logs with re-registration warnings on an
  already-misconfigured node.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 67 ++++++++++++++++---
 1 file changed, 56 insertions(+), 11 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 01439557..f6506229 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -140,6 +140,24 @@ nothing — the canonical set lives at compile time). Matches the
 current "ApplySplit / ApplyMerge in a future PR" semantics for
 the static route catalog.
 
+**`virtualForRoute` stays `map[uint64]*routeSlot`** (no widening
+to `slotKey`). Virtual buckets aggregate across labels per §6.5,
+so the fallback lookup keys on `RouteID` alone — widening it to
+`slotKey` would make every coarsened-route `Observe` miss both
+the labeled lookup AND the virtual fallback, silently dropping
+all traffic for those routes. The new `Observe` lookup chain is:
+
+```go
+slot, ok := tbl.slots[slotKey{RouteID: routeID, Label: label}]
+if !ok {
+    slot, ok = tbl.virtualForRoute[routeID] // labelled buckets fall through here
+    if !ok {
+        return // unknown route — drop, same as today
+    }
+}
+// atomic.Add as today
+```
+
 `routeSlot` gains a `Label string` field so `Flush` /
 `appendDrainedRow` can read the label off the slot rather than
 threading it through a separate channel:
@@ -211,20 +229,43 @@ The `routeSlot` map shape changes from `map[uint64]*routeSlot` to
 
 ## 5. Wire format extension
 
-`MatrixRow` (Go) and `KeyVizRow` (proto + JSON):
+Three layers gain a label field. They are kept in sync at PR-D+E
+(both shipping together) so an aggregator running PR-D against an
+upstream still on PR-C never sees a wire shape mismatch:
 
 ```diff
+ // keyviz/sampler.go — internal type, available from PR-B
  type MatrixRow struct {
    RouteID      uint64
-+  Label        string  // optional; empty for legacy unlabelled traffic
++  Label        string  // empty for legacy unlabelled traffic
    Start, End   []byte
    …
  }
+
+ // proto/admin.proto — wire form, PR-D+E
+ message KeyVizRow {
+   string bucket_id = 1;
+   …
++  string label = 13; // empty for legacy unlabelled traffic
+ }
+
+ // internal/admin/keyviz_handler.go — JSON struct, PR-D+E
+ type KeyVizRow struct {
+   BucketID  string `json:"bucket_id"`
+   …
++  Label     string `json:"label,omitempty"`
+ }
 ```
 
-The SPA reads the label and renders rows as
-`route:<id> / <label>` when label is non-empty; legacy rows
-(empty label) render as `route:<id>` exactly like today.
+We carry the label as a **dedicated field** (not just embedded in
+`bucket_id`) so SPA consumers do not need to parse the composite
+to recover it. The SPA renders `route:<id> / <label>` directly
+from `KeyVizRow.Label` when non-empty; legacy rows (empty label)
+render as `route:<id>` exactly like today. Carrying both
+`bucket_id` and `label` is intentional redundancy: `bucket_id`
+remains the globally-unique row identifier (used by
+`pivotKeyVizColumns` and merge-side dedupe), while `label`
+serves the SPA's render-and-filter ergonomics.
 
 **`bucket_id` is composite when `Label` is non-empty**:
 
@@ -303,8 +344,8 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, `RegisterRoute` pre-creates one slot per known label (§4.1.1). `MatrixRow.Label`. `reclaimRetiredSlot` dedupes on `(RouteID, Label)`. Update existing tests to pass empty labels. |
-| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Canonical constants live in `keyviz/labels.go` (NOT `adapter/keyviz_labels.go` — sampler-side imports must not climb back to the adapter package; sampler's `RegisterRoute` reads the canonical set at registration time). **Operator note**: with N labels, slot count grows from M routes to ~N×M; raise `--keyvizMaxTrackedRoutes` proportionally or hot routes will fold into the virtual bucket and lose per-label breakdown. PR-C emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so the operator sees the symptom in the log when it actually fires. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. |
+| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, `RegisterRoute` pre-creates one slot per known label (§4.1.1). `MatrixRow.Label`. `reclaimRetiredSlot` dedupes on `(RouteID, Label)`. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""` — until PR-C lands the legacy unlabelled behaviour is preserved. |
+| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Canonical constants live in `keyviz/labels.go` (NOT `adapter/keyviz_labels.go` — sampler-side imports must not climb back to the adapter package; sampler's `RegisterRoute` reads the canonical set at registration time). **Operator note**: with N labels, slot count grows from M routes to ~N×M; raise `--keyvizMaxTrackedRoutes` proportionally or hot routes will fold into the virtual bucket and lose per-label breakdown. PR-C emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so the operator sees the symptom in the log when it actually fires. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. **Rollout discipline**: raise `--keyvizMaxTrackedRoutes` to at least `current_routes × len(labels)` *before* deploying the PR-C binary, otherwise an already-misconfigured node prints a wave of warnings on startup as the watcher re-registers every catalog route — loud but not incorrect, only painful for log retention. |
 | **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) in the same PR as the SPA `route:N / label` rendering AND the `pivotKeyVizColumns` pivot-key widening from `RouteID` to `(RouteID, Label)` in `internal/admin/keyviz_handler.go`. Splitting these creates a window where `pivotKeyVizColumns` collapses labelled rows back into a single row in the JSON response — exactly the bug this feature is fixing. The fan-out aggregator picks up the new behaviour for free because the merge key uses `bucketID` (now composite). |
 
 PRs B and C are independent of the wire format; the heatmap will
@@ -375,10 +416,14 @@ extension is "switch on the field". (Reviewer notes from PR
    `route`). (Gemini round-1 minor on PR #694.)
 2. **Label allocation discipline** — who owns the canonical label
    set? Proposal: `keyviz/labels.go` exports `LabelDynamo`,
-   `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV` constants.
-   Constants live in the `keyviz` package (not `adapter/`) so
-   the sampler's `RegisterRoute` can read the full set at
-   registration time without an import cycle back to `adapter`.
+   `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV` constants
+   alongside an `AllLabels []string` slice that ranges over them
+   in stable order. `RegisterRoute` iterates `AllLabels` to
+   pre-allocate the labeled slots — exposing the slice (rather
+   than reflecting over the file) keeps the iteration deterministic
+   and the test surface small. Constants live in the `keyviz`
+   package (not `adapter/`) so the sampler can read the full set
+   at registration time without an import cycle back to `adapter`.
    Adapters refer to these constants; nothing stops a future
    adapter from inventing its own label, but the review burden
    of adding to the central file catches accidental variants

From 2750f4c29cce6731a58919facc194ea1d800eaf4 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Tue, 28 Apr 2026 23:51:27 +0900
Subject: [PATCH 05/31] docs: address PR #694 round-4 (Claude bot + CodeRabbit)

Eight items from round-4 review:

CodeRabbit critical:
- 5 proto field collision: proto/admin.proto KeyVizRow already
  declares string label = 4 (currently unused). The earlier draft
  proposed adding string label = 13, which would have collided.
  Reuse the existing field 4; no schema migration needed.
- 5 hard constraint: canonical labels MUST NOT contain ':'.
  Without this, "redis:db0" silently breaks bucket_id parsing.
  Pinned by a TestAllLabelsAvoidSeparator regression guard listed
  in 8 lens-5.

Claude bot round-4 moderate:
- 4.1.1 RemoveRoute symmetric teardown: when slot key widens,
  delete(slots, routeID) is a compile error; PR-B must loop over
  AllLabels + {""} and retire all N+1 slots. Otherwise labeled
  siblings leak into the live table forever.
- 7 PR-D+E mergeRowInto: mergeKeyVizMatrices keys on composite
  bucket_id (correct) but mergeRowInto constructs new KeyVizRow
  entries from scratch and does NOT copy Label. Without an
  explicit dst.Label = row.Label, every cluster-merged row emits
  Label="", defeating Option B's dedicated-field design.

Claude bot round-4 minor:
- 4.1.1 RegisterRoute idempotency: cur.slots[routeID] becomes a
  type mismatch after key widening; document the legacy-slot
  presence check that PR-B must use.
- 4.1.1 pseudocode comment: "labeled buckets fall through here"
  is misleading since the fallback is for coarsened routes, not
  labeled traffic. Fixed to "coarsened route: label irrelevant,
  aggregate all traffic".

CodeRabbit minor:
- 3 code fence language: tagged the surface example as `text`
  (MD040).
- Spelling: normalized "labelled" -> "labeled" / "unlabelled" ->
  "unlabeled" across the doc (LanguageTool flagged the
  inconsistency).

Plus 2 new test cases in 8 lens-5: RemoveRoute symmetric
teardown, mergeRowInto Label copy, and the labels-avoid-':'
constants test.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 101 ++++++++++++++++--
 1 file changed, 91 insertions(+), 10 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index f6506229..1fab104d 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -73,7 +73,7 @@ Default config emits the empty label (legacy behaviour); when
 adapters set labels, the heatmap splits a single route into one
 row per (route, label) pair:
 
-```
+```text
 Today (single group, no labels):
   Row 0  route:1  Total 378
 
@@ -150,7 +150,7 @@ all traffic for those routes. The new `Observe` lookup chain is:
 ```go
 slot, ok := tbl.slots[slotKey{RouteID: routeID, Label: label}]
 if !ok {
-    slot, ok = tbl.virtualForRoute[routeID] // labelled buckets fall through here
+    slot, ok = tbl.virtualForRoute[routeID] // coarsened route: label irrelevant, aggregate all traffic
     if !ok {
         return // unknown route — drop, same as today
     }
@@ -166,7 +166,7 @@ threading it through a separate channel:
 type routeSlot struct {
     metaMu  sync.RWMutex
     RouteID uint64
-    Label   string         // new — empty for the legacy unlabelled slot
+    Label   string         // new — empty for the legacy unlabeled slot
     GroupID uint64         // Phase 2-C+, see PR-3a
     Start, End []byte
     ...
@@ -178,6 +178,49 @@ type routeSlot struct {
 `(routeID, "dynamo")` and re-registered as `(routeID, "redis")`
 is **not** a reclaim candidate; they are different slots.
 
+#### Symmetric teardown in `RemoveRoute`
+
+`RemoveRoute(routeID)` must be updated to remove **every**
+`slotKey{routeID, label}` for `label ∈ AllLabels ∪ {""}` — not
+just the legacy `(routeID, "")` entry. The current code at
+`keyviz/sampler.go:532`:
+
+```go
+delete(next.slots, routeID)
+s.retiredSlots = append(s.retiredSlots, retiredSlot{slot: individual, retiredAt: retiredAt})
+```
+
+is a `map[uint64]*routeSlot` delete; after key widening this
+becomes a compile error. The PR-B fix loops over `AllLabels`
+plus the empty legacy label and retires N+1 slots in one pass:
+
+```go
+for _, label := range allLabelsWithLegacy() { // AllLabels ∪ {""}
+    key := slotKey{RouteID: routeID, Label: label}
+    if slot := next.slots[key]; slot != nil {
+        delete(next.slots, key)
+        s.retiredSlots = append(s.retiredSlots, retiredSlot{slot: slot, retiredAt: retiredAt})
+    }
+}
+```
+
+If only the legacy slot is retired, every labeled
+`(routeID, "dynamo")` / `(routeID, "redis")` / … remains in the
+live `routeTable` receiving Observe traffic forever — orphaned
+slots accumulating until process restart. The §8 lens-5 test
+"`RemoveRoute` retires N+1 slots" pins this. (Claude bot moderate
+on PR #694 round-4.)
+
+#### `RegisterRoute` idempotency check after key widening
+
+The current idempotency guard is `if _, ok := cur.slots[routeID]`
+which becomes a type mismatch after the key widens to `slotKey`.
+PR-B replaces it with a check on the legacy slot:
+`if _, ok := cur.slots[slotKey{RouteID: routeID}]; ok { return true }`
+— the legacy slot's presence implies the labeled siblings were
+already created by an earlier `RegisterRoute` call (the
+pre-creation loop is atomic with respect to the routesMu lock).
+
 Two alternatives we explicitly reject:
 
 - **First-seen miss-fall-back to empty label**: surprising silent
@@ -237,16 +280,27 @@ upstream still on PR-C never sees a wire shape mismatch:
  // keyviz/sampler.go — internal type, available from PR-B
  type MatrixRow struct {
    RouteID      uint64
-+  Label        string  // empty for legacy unlabelled traffic
++  Label        string  // empty for legacy unlabeled traffic
    Start, End   []byte
    …
  }
 
- // proto/admin.proto — wire form, PR-D+E
+ // proto/admin.proto — wire form, PR-D+E.
+ //
+ // Reuse the **existing** field 4 (`string label`), which is
+ // already declared on KeyVizRow but currently unused — no
+ // schema migration, no field-number bump. The existing comment
+ // already reserves it for "future per-Observe label", which is
+ // exactly this proposal's payload. (CodeRabbit critical on PR
+ // #694 caught this — an earlier draft proposed adding a
+ // duplicate `string label = 13;` which would have collided.)
  message KeyVizRow {
    string bucket_id = 1;
+   bytes start = 2;
+   bytes end = 3;
+-  string label = 4;        // currently unused
++  string label = 4;        // PR-D+E: filled with the per-Observe label
    …
-+  string label = 13; // empty for legacy unlabelled traffic
  }
 
  // internal/admin/keyviz_handler.go — JSON struct, PR-D+E
@@ -275,7 +329,7 @@ serves the SPA's render-and-filter ergonomics.
   `"route:1:dynamo"`).
 
 Composite `bucket_id` keeps the field globally unique across
-labelled rows. Without it, `pivotKeyVizColumns` (currently keyed
+labeled rows. Without it, `pivotKeyVizColumns` (currently keyed
 by `RouteID` alone in `internal/admin/keyviz_handler.go`) would
 collapse `(routeID=1, label="dynamo")` and `(routeID=1,
 label="redis")` into the same map entry — silently re-merging
@@ -293,6 +347,16 @@ hierarchical label (§9 Q1) can use `/` without ambiguity.
 `dynamo/users` the composite becomes `route:1:dynamo/users` and
 parsers split on the *first* `:` after `route`.
 
+**Hard constraint on canonical labels**: a label MUST NOT contain
+`:`. The constants in `keyviz/labels.go` (§9 Q2) are the only
+source of label values, and we enforce this at the constants
+file with a `func TestAllLabelsAvoidSeparator(t *testing.T)` test
+that asserts `!strings.ContainsRune(l, ':')` for every member of
+`AllLabels`. Without this constraint a future label like
+`"redis:db0"` would silently break `bucket_id` parsing — the
+parser would split at the wrong `:`. (CodeRabbit major on PR
+#694.)
+
 `route_ids` / `aggregate` / virtual-bucket semantics from §5 of
 the parent design are unchanged — labels are an orthogonal axis
 to the route-coarsening machinery (and §6.5 below pins the
@@ -344,9 +408,9 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, `RegisterRoute` pre-creates one slot per known label (§4.1.1). `MatrixRow.Label`. `reclaimRetiredSlot` dedupes on `(RouteID, Label)`. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""` — until PR-C lands the legacy unlabelled behaviour is preserved. |
+| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, `RegisterRoute` pre-creates one slot per known label (§4.1.1). `MatrixRow.Label`. `reclaimRetiredSlot` dedupes on `(RouteID, Label)`. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""` — until PR-C lands the legacy unlabeled behaviour is preserved. |
 | **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Canonical constants live in `keyviz/labels.go` (NOT `adapter/keyviz_labels.go` — sampler-side imports must not climb back to the adapter package; sampler's `RegisterRoute` reads the canonical set at registration time). **Operator note**: with N labels, slot count grows from M routes to ~N×M; raise `--keyvizMaxTrackedRoutes` proportionally or hot routes will fold into the virtual bucket and lose per-label breakdown. PR-C emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so the operator sees the symptom in the log when it actually fires. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. **Rollout discipline**: raise `--keyvizMaxTrackedRoutes` to at least `current_routes × len(labels)` *before* deploying the PR-C binary, otherwise an already-misconfigured node prints a wave of warnings on startup as the watcher re-registers every catalog route — loud but not incorrect, only painful for log retention. |
-| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) in the same PR as the SPA `route:N / label` rendering AND the `pivotKeyVizColumns` pivot-key widening from `RouteID` to `(RouteID, Label)` in `internal/admin/keyviz_handler.go`. Splitting these creates a window where `pivotKeyVizColumns` collapses labelled rows back into a single row in the JSON response — exactly the bug this feature is fixing. The fan-out aggregator picks up the new behaviour for free because the merge key uses `bucketID` (now composite). |
+| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) in the same PR as the SPA `route:N / label` rendering AND the `pivotKeyVizColumns` pivot-key widening from `RouteID` to `(RouteID, Label)` in `internal/admin/keyviz_handler.go`. Splitting these creates a window where `pivotKeyVizColumns` collapses labeled rows back into a single row in the JSON response — exactly the bug this feature is fixing. The fan-out aggregator picks up most of the new behaviour for free because `mergeKeyVizMatrices` keys on composite `BucketID`, but **`mergeRowInto` (`internal/admin/keyviz_fanout.go:509`) constructs new `KeyVizRow` entries from scratch and does NOT currently copy `Label`** — PR-D+E must add the explicit `dst.Label = row.Label` copy, or every cluster-merged row emits `Label = ""` and the SPA falls back to parsing `bucket_id`, defeating Option B. (Claude bot moderate on PR #694 round-4.) |
 
 PRs B and C are independent of the wire format; the heatmap will
 keep showing one row per route until D+E ship, but the per-label
@@ -381,6 +445,23 @@ extension is "switch on the field". (Reviewer notes from PR
      empty-label slot). Catches a PR-B regression that skips a
      label constant in the pre-creation loop — without this the
      missing slot only surfaces when that adapter fires traffic.
+   - **`RemoveRoute` symmetric teardown**: register 1 route,
+     `RemoveRoute(routeID)`, assert `len(tbl.slots) == 0` and
+     `len(retiredSlots) == len(labels) + 1`. Catches a PR-B
+     regression where only the legacy slot is retired and the
+     labeled siblings leak into the live table.
+   - **`mergeRowInto` Label copy** (PR-D+E): two nodes, one with
+     `KeyVizRow{Label: "dynamo"}`, one with `KeyVizRow{Label:
+     "redis"}` for the same routeID. The merged response must
+     have **two rows** with `Label="dynamo"` and `Label="redis"`
+     respectively — not two rows with `Label=""`. Catches the
+     mergeRowInto regression where Label is omitted from the
+     `dst := &KeyVizRow{...}` construction.
+   - **Canonical labels avoid `:`** (PR-B regression guard):
+     range `AllLabels`, assert `!strings.ContainsRune(label,
+     ':')` for every constant. Pins the §5 hard constraint at
+     compile time; a future "redis:db0" would otherwise silently
+     break `bucket_id` parsing.
    - **`reclaimRetiredSlot` (RouteID, Label) dedupe**: retire
      `(routeID, "dynamo")`, re-register `(routeID, "dynamo")`,
      assert reclaim succeeds; then retire `(routeID, "dynamo")`
@@ -438,7 +519,7 @@ extension is "switch on the field". (Reviewer notes from PR
 
 - Per-table / per-bucket / per-queue / per-Redis-DB sub-labels.
 - Operator-configurable label taxonomy.
-- Persistence of labelled rows (Phase 3 covers persistence
+- Persistence of labeled rows (Phase 3 covers persistence
   generally; labels ride along once the persistence path lands).
 - Adapter-aware splitting of routes (`SplitRange` triggered by
   adapter-label hotspots) — that is a Phase 3+ idea.

From 3ae3417d6f837404ffe91ebf42f4aac2f5b33e8b Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Tue, 28 Apr 2026 23:59:12 +0900
Subject: [PATCH 06/31] docs: address PR #694 round-5 (Codex P1 + P2)

Two items from Codex round-5 review:

- P1 / line 125 / pre-allocation regression: the earlier draft
  said RegisterRoute pre-creates one slot per label at PR-B. That
  would shrink the effective route budget from M to ~M / (N+1)
  even though no adapter is yet calling with non-empty labels -
  PR-B is supposed to be a behavior-neutral refactor.
  Resolution: split the pre-allocation across PR-B and PR-C. PR-B
  only creates the legacy empty-label slot (per-route slot count
  unchanged at 1); PR-C extends RegisterRoute to pre-create the
  labeled siblings. PR-B is now strictly route-count-neutral on
  existing deployments. Pseudocode added showing both bodies.

- P2 / line 412 / rollout formula off by one per route: rollout
  guidance said current_routes x len(labels), but operators must
  size for the legacy empty-label slot too:
  current_routes x (len(labels) + 1). Operators sizing only by
  len(labels) hit coarsening immediately and lose labeled
  breakdown anyway. Updated the rollout note to use the +1 form.

Plus 7 PR-B / PR-C scope rows updated to reflect the split
pre-allocation strategy so PR-B implementors do not accidentally
implement the full pre-creation loop and ship a regression.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 41 ++++++++++++++++---
 1 file changed, 36 insertions(+), 5 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 1fab104d..e4f1f902 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -121,9 +121,40 @@ the hot path never needs to allocate. We keep that invariant by
 making `RegisterRoute` **pre-create one slot per known label** at
 registration time, but only on the individual-tracking path. The
 label set is the canonical `keyviz/labels.go` constants (§9 Q2);
-register-time slot count is `len(routes) × len(labels) + 1` (the
-+1 is the empty-label legacy slot, kept so callers that pass
-`label=""` still hit a slot).
+per-route slot count is `len(labels) + 1` (the +1 is the empty-
+label legacy slot, kept so callers that pass `label=""` still hit
+a slot). Total individual slots in the table is therefore
+`len(routes) × (len(labels) + 1)`.
+
+#### Pre-allocation is gated on PR-C, not PR-B
+
+PR-B alone changes the slot-key type from `uint64` to `slotKey`
+but still only pre-creates the **legacy empty-label slot**. The
+labeled siblings are created by PR-C, when adapters actually
+start passing non-empty labels. This split is deliberate and
+preserves PR-B as a behavior-neutral refactor:
+
+- After PR-B alone, slot count per route is **1** (just the
+  legacy slot, identical to today's `routeID`-only behavior).
+  A deployment running 1024 routes today still supports 1024
+  routes after PR-B — no early coarsening, no virtual-bucket
+  fold, no per-operator action required. (Codex P1 on round-5.)
+- After PR-C, slot count per route grows to `len(labels) + 1`.
+  Operators raise `--keyvizMaxTrackedRoutes` before deploying
+  PR-C per the §7 rollout note.
+
+Concretely PR-B's `RegisterRoute` body looks like:
+
+```go
+// PR-B body:
+slot := newSlot(routeID, "", start, end, groupID)
+next.slots[slotKey{RouteID: routeID, Label: ""}] = slot
+
+// PR-C extends to:
+for _, label := range allLabelsWithLegacy() { // AllLabels ∪ {""}
+    next.slots[slotKey{RouteID: routeID, Label: label}] = newSlot(routeID, label, start, end, groupID)
+}
+```
 
 When `RegisterRoute` decides a route will be coarsened into a
 virtual bucket (i.e. it returns `false`), **no labeled slots are
@@ -408,8 +439,8 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, `RegisterRoute` pre-creates one slot per known label (§4.1.1). `MatrixRow.Label`. `reclaimRetiredSlot` dedupes on `(RouteID, Label)`. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""` — until PR-C lands the legacy unlabeled behaviour is preserved. |
-| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Canonical constants live in `keyviz/labels.go` (NOT `adapter/keyviz_labels.go` — sampler-side imports must not climb back to the adapter package; sampler's `RegisterRoute` reads the canonical set at registration time). **Operator note**: with N labels, slot count grows from M routes to ~N×M; raise `--keyvizMaxTrackedRoutes` proportionally or hot routes will fold into the virtual bucket and lose per-label breakdown. PR-C emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so the operator sees the symptom in the log when it actually fires. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. **Rollout discipline**: raise `--keyvizMaxTrackedRoutes` to at least `current_routes × len(labels)` *before* deploying the PR-C binary, otherwise an already-misconfigured node prints a wave of warnings on startup as the watcher re-registers every catalog route — loud but not incorrect, only painful for log retention. |
+| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, slot-key type widens to `slotKey{uint64, string}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). `MatrixRow.Label` field present but always empty. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
+| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Extends `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot from PR-B. Canonical constants live in `keyviz/labels.go` (NOT `adapter/keyviz_labels.go` — sampler-side imports must not climb back to the adapter package; sampler's `RegisterRoute` reads the canonical set at registration time). **Operator note**: with N labels, slot count grows from M routes to ~N×M; raise `--keyvizMaxTrackedRoutes` proportionally or hot routes will fold into the virtual bucket and lose per-label breakdown. PR-C emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so the operator sees the symptom in the log when it actually fires. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. **Rollout discipline**: raise `--keyvizMaxTrackedRoutes` to at least `current_routes × (len(labels) + 1)` *before* deploying the PR-C binary — the `+ 1` accounts for the legacy empty-label slot that lives alongside the N labeled siblings on every individually-tracked route. Operators sizing only with `× len(labels)` (one short per route) hit coarsening immediately and lose labeled breakdown anyway. (Codex P2 on round-5.) Otherwise an already-misconfigured node prints a wave of warnings on startup as the watcher re-registers every catalog route — loud but not incorrect, only painful for log retention. |
 | **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) in the same PR as the SPA `route:N / label` rendering AND the `pivotKeyVizColumns` pivot-key widening from `RouteID` to `(RouteID, Label)` in `internal/admin/keyviz_handler.go`. Splitting these creates a window where `pivotKeyVizColumns` collapses labeled rows back into a single row in the JSON response — exactly the bug this feature is fixing. The fan-out aggregator picks up most of the new behaviour for free because `mergeKeyVizMatrices` keys on composite `BucketID`, but **`mergeRowInto` (`internal/admin/keyviz_fanout.go:509`) constructs new `KeyVizRow` entries from scratch and does NOT currently copy `Label`** — PR-D+E must add the explicit `dst.Label = row.Label` copy, or every cluster-merged row emits `Label = ""` and the SPA falls back to parsing `bucket_id`, defeating Option B. (Claude bot moderate on PR #694 round-4.) |
 
 PRs B and C are independent of the wire format; the heatmap will

From 3793366704a5d8bc627f6b43dc2c59ed3959f264 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 00:04:28 +0900
Subject: [PATCH 07/31] docs: address PR #694 round-5 (Claude bot moderate
 items)

Three moderate items from Claude bot's round-5 review (the long
one that scanned the actual single-node path code):

- 7 PR-D+E single-node-path completeness: the previous draft
  only called out mergeRowInto in keyviz_fanout.go but missed
  three companion changes in keyviz_handler.go that PR-D+E must
  make. Documented all four:
  (a) pivotKeyVizColumns rowsByID map AND order slice widen to
      composite BucketID string (or equivalent (uint64, string));
  (b) newKeyVizRowFrom copies mr.Label -> row.Label;
  (c) bucketIDFor returns composite "route:<id>:<label>" when
      Label is non-empty;
  (d) mergeRowInto adds dst.Label = row.Label.
  Splitting any subset leaves a window where labeled rows
  collapse silently.
- 4.1.1 MaxTrackedRoutes semantics: pick Option A (count
  routes, not slots). After key widening, len(next.slots) counts
  labeled slots, so the coarsening check must divide by
  len(AllLabels)+1. This preserves operator-visible meaning of
  the existing flag - 1024 still means 1024 routes, not ~170.
  Pseudocode added at the actual line site (sampler.go:416).
  Option B (redefine as slots) explicitly rejected on the
  behavior-preserving goal in 2.1.
- 7 PR-B AllLabels timing made explicit: PR-B creates
  keyviz/labels.go with the slotKey struct + an EMPTY AllLabels
  slice; the five adapter constants are added in PR-C alongside
  adapter wiring. PR-B's pre-creation loop is a no-op against
  the empty slice, so per-route slot count is identical to
  today (1). Closes the ambiguity an implementor might hit
  about "where do the constants land".
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 34 +++++++++++++++++--
 1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index e4f1f902..1e01dded 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -164,6 +164,36 @@ fallback in `Observe` already handles all labels uniformly per
 labeled slots that would never be hit would just burn allocator
 work.
 
+#### `MaxTrackedRoutes` semantics: count routes, not slots
+
+After the slot-key widens to `slotKey{uint64, string}`, the live
+table contains `len(AllLabels) + 1` slots per individually-tracked
+route — bare `len(next.slots)` no longer equals "number of
+routes." We pick **Option A (route-counting)**: the coarsening
+check at `keyviz/sampler.go:416` divides by `len(AllLabels) + 1`
+so `MaxTrackedRoutes` continues to count routes, not slots. This
+preserves the operator-visible meaning of the existing flag —
+`--keyvizMaxTrackedRoutes=1024` still means "1024 individual
+routes," not "1024 slots that may be ~170 routes." Total slot
+memory is `MaxTrackedRoutes × (len(AllLabels) + 1)`, which we
+document but never count against the cap.
+
+```go
+// keyviz/sampler.go:416 (PR-B)
+slotsPerRoute := len(keyviz.AllLabels) + 1 // 1 today (empty AllLabels), 6 after PR-C
+if len(next.slots) / slotsPerRoute < s.opts.MaxTrackedRoutes {
+    // ... pre-create slot(s) for this route
+}
+```
+
+The alternative (Option B: `MaxTrackedRoutes` redefined as
+"slots") would silently halve the route capacity at PR-C
+rollout — an operator setting 1024 today would suddenly track
+~170 routes after PR-C deploys with five labels, even though
+the flag value is unchanged. We reject this on the
+"behavior-preserving for legacy single-group deployments"
+goal in §2.1. (Claude bot moderate on PR #694 round-5.)
+
 New labels added after a route is registered are not auto-bound;
 an operator deploying a new adapter must **deploy a new binary**
 that includes the new constant (a process restart alone does
@@ -439,9 +469,9 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, slot-key type widens to `slotKey{uint64, string}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). `MatrixRow.Label` field present but always empty. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
+| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, slot-key type widens to `slotKey{uint64, string}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). PR-B creates `keyviz/labels.go` with the `slotKey` struct + an **empty `AllLabels` slice**; the five adapter constants (`LabelDynamo`, `LabelRedis`, …) are added in PR-C alongside the adapter wiring, so PR-B's pre-creation loop is a no-op against the empty slice and the existing single slot per route is created exactly as today. `MatrixRow.Label` field present but always empty. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
 | **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Extends `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot from PR-B. Canonical constants live in `keyviz/labels.go` (NOT `adapter/keyviz_labels.go` — sampler-side imports must not climb back to the adapter package; sampler's `RegisterRoute` reads the canonical set at registration time). **Operator note**: with N labels, slot count grows from M routes to ~N×M; raise `--keyvizMaxTrackedRoutes` proportionally or hot routes will fold into the virtual bucket and lose per-label breakdown. PR-C emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so the operator sees the symptom in the log when it actually fires. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. **Rollout discipline**: raise `--keyvizMaxTrackedRoutes` to at least `current_routes × (len(labels) + 1)` *before* deploying the PR-C binary — the `+ 1` accounts for the legacy empty-label slot that lives alongside the N labeled siblings on every individually-tracked route. Operators sizing only with `× len(labels)` (one short per route) hit coarsening immediately and lose labeled breakdown anyway. (Codex P2 on round-5.) Otherwise an already-misconfigured node prints a wave of warnings on startup as the watcher re-registers every catalog route — loud but not incorrect, only painful for log retention. |
-| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) in the same PR as the SPA `route:N / label` rendering AND the `pivotKeyVizColumns` pivot-key widening from `RouteID` to `(RouteID, Label)` in `internal/admin/keyviz_handler.go`. Splitting these creates a window where `pivotKeyVizColumns` collapses labeled rows back into a single row in the JSON response — exactly the bug this feature is fixing. The fan-out aggregator picks up most of the new behaviour for free because `mergeKeyVizMatrices` keys on composite `BucketID`, but **`mergeRowInto` (`internal/admin/keyviz_fanout.go:509`) constructs new `KeyVizRow` entries from scratch and does NOT currently copy `Label`** — PR-D+E must add the explicit `dst.Label = row.Label` copy, or every cluster-merged row emits `Label = ""` and the SPA falls back to parsing `bucket_id`, defeating Option B. (Claude bot moderate on PR #694 round-4.) |
+| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) plus the SPA `route:N / label` rendering AND the four single-node-path code changes the labeled wire format requires in `internal/admin/keyviz_handler.go`: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` (or equivalently `(uint64, string)`) — widening only the map without the `order` slice is a compile error and an obvious mismatch, but worth naming; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label`, mirroring the `mergeRowInto` change for the fan-out path; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>(d) `mergeRowInto` (`internal/admin/keyviz_fanout.go:509`) adds `dst.Label = row.Label` so the fan-out path doesn't drop the label on merge. Splitting any subset of (a)–(d) leaves a window where labeled rows collapse — they're a single PR. The fan-out merge key (`bucketID` composite) picks up the per-label separation automatically once `bucketIDFor` is updated. (Claude bot moderate on PR #694 round-4 + round-5.) |
 
 PRs B and C are independent of the wire format; the heatmap will
 keep showing one row per route until D+E ship, but the per-label

From e9769e175758fb94fb543f128d7545904c79e31b Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 00:10:12 +0900
Subject: [PATCH 08/31] docs: address PR #694 round-6 (Claude bot + Codex)

Six items from round-6:

- 7 PR-B keyviz/labels.go scope (Claude bot moderate +
  Codex P2 line 451): pull file creation forward to PR-B with the
  five constants declared but AllLabels DELIBERATELY EMPTY.
  Without this, RemoveRoute's allLabelsWithLegacy() teardown loop
  shown in 4.1.1 would not compile in PR-B.
- 7 PR-D+E first vs second level Label copy (Claude bot moderate
  carry-over): clarified that newKeyVizRowFrom is the first-level
  fix (single-node and cluster) and mergeRowInto is the
  second-level fix (cluster only). Both required; missing either
  leaves a flavour of deployment with empty labels.
- 7 PR-C MaxTrackedRoutes scaling (Codex P2 line 473): the
  rollout note told operators to raise the cap by
  x (len(labels)+1), but 4.1.1 redefines the coarsening check to
  divide by the same factor so the cap continues to count routes.
  Both statements cannot be true. Removed the scaling instruction;
  PR-C is now route-count-neutral on the operator side.
- 6 PR-D+E either-order claim (Codex P2 line 451): 6 said PR-D
  and PR-E are shippable in either order; 7 explicitly says
  ship together. Resolved in favour of 7 (the merge contradiction
  during a partial deployment is the original motivation for
  bundling).
- 8 lens-5 phase-gate test descriptions (Claude bot minor + Codex
  P2 line 508): "Slot pre-creation count" and "RemoveRoute
  symmetric teardown" tests now describe both the PR-B counterpart
  (empty AllLabels -> 1 slot) and the PR-C counterpart
  (populated AllLabels -> N+1 slots). PR-B implementor cannot
  accidentally ship the labeled-sibling pre-creation early.
- 5 "Labelled" survivor at line 359 (Claude bot minor): single
  remaining double-l from the round-4 spelling sweep, fixed.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 45 ++++++++++++-------
 1 file changed, 28 insertions(+), 17 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 1e01dded..33e0dfcb 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -386,7 +386,7 @@ serves the SPA's render-and-filter ergonomics.
 
 - Legacy (empty label): `bucket_id = "route:<id>"` — unchanged
   from the parent design.
-- Labelled: `bucket_id = "route:<id>:<label>"` (e.g.
+- Labeled: `bucket_id = "route:<id>:<label>"` (e.g.
   `"route:1:dynamo"`).
 
 Composite `bucket_id` keeps the field globally unique across
@@ -446,9 +446,12 @@ The fan-out aggregator's per-cell merge key gains the label:
 
 Reads still sum, writes still max-with-conflict; nothing about
 the merge rules changes other than the wire shape of `bucketID`.
-This is what makes PR-D and PR-E shippable in either order
-(see §7) — the wire-format change in PR-D *is* the merge-key
-change.
+The aggregator's merge logic does not need to change at all once
+`bucketID` is composite; that's why PR-D and PR-E are described
+as **a single PR-D+E** in §7 rather than two separate PRs. (An
+earlier draft of this section claimed they were "shippable in
+either order" — that contradicted §7's "ship together"
+requirement; resolved in favour of §7. Codex round-6.)
 
 ### 6.5 Virtual buckets and labels
 
@@ -469,9 +472,9 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, slot-key type widens to `slotKey{uint64, string}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). PR-B creates `keyviz/labels.go` with the `slotKey` struct + an **empty `AllLabels` slice**; the five adapter constants (`LabelDynamo`, `LabelRedis`, …) are added in PR-C alongside the adapter wiring, so PR-B's pre-creation loop is a no-op against the empty slice and the existing single slot per route is created exactly as today. `MatrixRow.Label` field present but always empty. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
-| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. Extends `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot from PR-B. Canonical constants live in `keyviz/labels.go` (NOT `adapter/keyviz_labels.go` — sampler-side imports must not climb back to the adapter package; sampler's `RegisterRoute` reads the canonical set at registration time). **Operator note**: with N labels, slot count grows from M routes to ~N×M; raise `--keyvizMaxTrackedRoutes` proportionally or hot routes will fold into the virtual bucket and lose per-label breakdown. PR-C emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so the operator sees the symptom in the log when it actually fires. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. **Rollout discipline**: raise `--keyvizMaxTrackedRoutes` to at least `current_routes × (len(labels) + 1)` *before* deploying the PR-C binary — the `+ 1` accounts for the legacy empty-label slot that lives alongside the N labeled siblings on every individually-tracked route. Operators sizing only with `× len(labels)` (one short per route) hit coarsening immediately and lose labeled breakdown anyway. (Codex P2 on round-5.) Otherwise an already-misconfigured node prints a wave of warnings on startup as the watcher re-registers every catalog route — loud but not incorrect, only painful for log retention. |
-| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) plus the SPA `route:N / label` rendering AND the four single-node-path code changes the labeled wire format requires in `internal/admin/keyviz_handler.go`: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` (or equivalently `(uint64, string)`) — widening only the map without the `order` slice is a compile error and an obvious mismatch, but worth naming; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label`, mirroring the `mergeRowInto` change for the fan-out path; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>(d) `mergeRowInto` (`internal/admin/keyviz_fanout.go:509`) adds `dst.Label = row.Label` so the fan-out path doesn't drop the label on merge. Splitting any subset of (a)–(d) leaves a window where labeled rows collapse — they're a single PR. The fan-out merge key (`bucketID` composite) picks up the per-label separation automatically once `bucketIDFor` is updated. (Claude bot moderate on PR #694 round-4 + round-5.) |
+| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, slot-key type widens to `slotKey{uint64, string}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `LabelDynamo`/`LabelRedis`/etc. **constants** (declared but unused — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []string`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present but always empty. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
+| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. PR-C **populates** `AllLabels` in `keyviz/labels.go` (the file itself was created in PR-B with an empty slice — see PR-B above) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extends `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** at PR-C deploy. (Earlier drafts told operators to scale the cap by `× (len(labels)+1)` — that was correct under Option B's slot-counting semantics but redundant under the chosen Option A. Codex round-6.) Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. PR-C also emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so a misconfigured cap surfaces in logs when it actually bites. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. |
+| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) plus the SPA `route:N / label` rendering AND the four single-node-path code changes the labeled wire format requires in `internal/admin/keyviz_handler.go`: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` (or equivalently `(uint64, string)`) — widening only the map without the `order` slice is a compile error and an obvious mismatch, but worth naming; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` — **first-level** Label copy, affects single-node and cluster deployments alike (every JSON response goes through this constructor); <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>(d) `mergeRowInto` (`internal/admin/keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this. Both copies are required: missing (b) leaves single-node responses with empty labels; missing (d) leaves cluster-merged responses with empty labels. Splitting any subset of (a)–(d) leaves a window where labeled rows collapse — they're a single PR. The fan-out merge key (`bucketID` composite) picks up the per-label separation automatically once `bucketIDFor` is updated. (Claude bot moderate on PR #694 round-4 + round-5.) |
 
 PRs B and C are independent of the wire format; the heatmap will
 keep showing one row per route until D+E ship, but the per-label
@@ -501,16 +504,24 @@ extension is "switch on the field". (Reviewer notes from PR
    sees the label-collapsed view; new SPA against old server
    sees the legacy view; both are coherent.
 5. **Test coverage** — New test categories:
-   - **Slot pre-creation count**: register 1 route, assert
-     `len(tbl.slots) == len(labels) + 1` (N labeled + 1 legacy
-     empty-label slot). Catches a PR-B regression that skips a
-     label constant in the pre-creation loop — without this the
-     missing slot only surfaces when that adapter fires traffic.
-   - **`RemoveRoute` symmetric teardown**: register 1 route,
-     `RemoveRoute(routeID)`, assert `len(tbl.slots) == 0` and
-     `len(retiredSlots) == len(labels) + 1`. Catches a PR-B
-     regression where only the legacy slot is retired and the
-     labeled siblings leak into the live table.
+   - **Slot pre-creation count (PR-C, post-`AllLabels`
+     population)**: register 1 route, assert
+     `len(tbl.slots) == len(AllLabels) + 1`. Catches a PR-C
+     regression that skips a label constant in the pre-creation
+     loop — without this the missing slot only surfaces when
+     that adapter fires traffic. **PR-B counterpart** (with
+     empty `AllLabels`): register 1 route, assert
+     `len(tbl.slots) == 1` (just the legacy slot); pins
+     PR-B's behavior-neutral refactor goal.
+   - **`RemoveRoute` symmetric teardown (PR-C)**: register 1
+     route, `RemoveRoute(routeID)`, assert `len(tbl.slots) == 0`
+     and `len(retiredSlots) == len(AllLabels) + 1`. Catches a
+     PR-C regression where only the legacy slot is retired and
+     the labeled siblings leak into the live table. **PR-B
+     counterpart** (empty `AllLabels`): register 1 route,
+     `RemoveRoute(routeID)`, assert
+     `len(retiredSlots) == 1` — the loop body executes once
+     for the empty-label slot only.
    - **`mergeRowInto` Label copy** (PR-D+E): two nodes, one with
      `KeyVizRow{Label: "dynamo"}`, one with `KeyVizRow{Label:
      "redis"}` for the same routeID. The merged response must

From ab0162c2574a08d1ce067f4068c4a32bb87141bb Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 00:18:34 +0900
Subject: [PATCH 09/31] docs: address PR #694 round-7 (Claude bot minor + Codex
 P2)

Four follow-ups from round-7:

- 4.1.1 PR-C pseudocode adds reclaimRetiredSlot per (RouteID,
  Label): the prior block showed slot creation but skipped the
  reclaim call. Per-label reclaim is required so a remove +
  re-register inside the grace window doesn't split counts
  across two slots.
- 5 pivotKeyVizColumns key form: settled on composite
  BucketID string (was ambiguous: 5 said struct, 7 said string).
  Picks the form that already matches the fan-out's
  rowsByBucket - no conversion at the boundary.
- 8 lens-5 TestAllLabelsAvoidSeparator phase: relabeled as
  "PR-C regression guard, written in PR-B" - clearer that the
  test is a no-op until AllLabels is populated, but writing it
  in PR-B prevents PR-C from landing a :-containing label.
- 7 PR-D+E gRPC path (Codex P2 line 477): adapter/admin_grpc.go
  has its own matrixToProto / per-row conversion that the prior
  scope missed. Added as item (e). Without this, GetKeyVizMatrix
  gRPC clients see collapsed unlabeled rows while HTTP/SPA
  responses show per-label rows - an API inconsistency. PR-D+E
  scope now lists all 5 required copies across the 3 response
  paths.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 36 +++++++++++++------
 1 file changed, 26 insertions(+), 10 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 33e0dfcb..8c51b9ec 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -147,12 +147,19 @@ Concretely PR-B's `RegisterRoute` body looks like:
 
 ```go
 // PR-B body:
-slot := newSlot(routeID, "", start, end, groupID)
+slot := s.reclaimRetiredSlot(routeID, "") // (RouteID, Label) dedupe — PR-B widened the signature
+if slot == nil {
+    slot = newSlot(routeID, "", start, end, groupID)
+}
 next.slots[slotKey{RouteID: routeID, Label: ""}] = slot
 
 // PR-C extends to:
 for _, label := range allLabelsWithLegacy() { // AllLabels ∪ {""}
-    next.slots[slotKey{RouteID: routeID, Label: label}] = newSlot(routeID, label, start, end, groupID)
+    slot := s.reclaimRetiredSlot(routeID, label) // reclaim per (routeID, label), not just legacy
+    if slot == nil {
+        slot = newSlot(routeID, label, start, end, groupID)
+    }
+    next.slots[slotKey{RouteID: routeID, Label: label}] = slot
 }
 ```
 
@@ -399,8 +406,13 @@ property is what `applyKeyVizRowBudget` and `sortKeyVizRowsByStart`
 need for deterministic tiebreak (both sort on `BucketID` last).
 
 `pivotKeyVizColumns` widens its pivot key from `uint64` (RouteID)
-to a struct of `(uint64, string)` to mirror the sampler's slot
-key — same shape, same reasoning as §4.1.1.
+to the **composite `BucketID` string** — matching the fan-out's
+existing `rowsByBucket map[string]*KeyVizRow` so both paths key on
+the same value (`"route:1:dynamo"`). An equivalent struct-of-
+`(uint64, string)` would work but adds a conversion step at the
+fan-out boundary; the string form is simpler and consistent with
+the rest of the handler's surface area. Same shape, same
+reasoning as §4.1.1.
 
 We use `:` (not `/`) as the route-vs-label separator so a future
 hierarchical label (§9 Q1) can use `/` without ambiguity.
@@ -474,7 +486,7 @@ Virtual-bucket rows in the heatmap render as
 | **PR-A** | Land this design doc. |
 | **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, slot-key type widens to `slotKey{uint64, string}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `LabelDynamo`/`LabelRedis`/etc. **constants** (declared but unused — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []string`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present but always empty. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
 | **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. PR-C **populates** `AllLabels` in `keyviz/labels.go` (the file itself was created in PR-B with an empty slice — see PR-B above) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extends `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** at PR-C deploy. (Earlier drafts told operators to scale the cap by `× (len(labels)+1)` — that was correct under Option B's slot-counting semantics but redundant under the chosen Option A. Codex round-6.) Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. PR-C also emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so a misconfigured cap surfaces in logs when it actually bites. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. |
-| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) plus the SPA `route:N / label` rendering AND the four single-node-path code changes the labeled wire format requires in `internal/admin/keyviz_handler.go`: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` (or equivalently `(uint64, string)`) — widening only the map without the `order` slice is a compile error and an obvious mismatch, but worth naming; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` — **first-level** Label copy, affects single-node and cluster deployments alike (every JSON response goes through this constructor); <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>(d) `mergeRowInto` (`internal/admin/keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this. Both copies are required: missing (b) leaves single-node responses with empty labels; missing (d) leaves cluster-merged responses with empty labels. Splitting any subset of (a)–(d) leaves a window where labeled rows collapse — they're a single PR. The fan-out merge key (`bucketID` composite) picks up the per-label separation automatically once `bucketIDFor` is updated. (Claude bot moderate on PR #694 round-4 + round-5.) |
+| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) plus the SPA `route:N / label` rendering AND the **five** code changes the labeled wire format requires across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error and an obvious mismatch, but worth naming; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` — **first-level** Label copy, affects single-node and cluster deployments alike (every JSON response goes through this constructor); <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) and emit composite `bucket_id`. Without this, `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows — an API inconsistency for production users of the admin RPC. (Codex P2 on round-7.) <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting any subset leaves a window where labeled rows collapse — they're a single PR. (Claude bot moderate on PR #694 round-4 + round-5.) |
 
 PRs B and C are independent of the wire format; the heatmap will
 keep showing one row per route until D+E ship, but the per-label
@@ -529,11 +541,15 @@ extension is "switch on the field". (Reviewer notes from PR
      respectively — not two rows with `Label=""`. Catches the
      mergeRowInto regression where Label is omitted from the
      `dst := &KeyVizRow{...}` construction.
-   - **Canonical labels avoid `:`** (PR-B regression guard):
-     range `AllLabels`, assert `!strings.ContainsRune(label,
-     ':')` for every constant. Pins the §5 hard constraint at
-     compile time; a future "redis:db0" would otherwise silently
-     break `bucket_id` parsing.
+   - **Canonical labels avoid `:`** (PR-C regression guard,
+     written in PR-B): range `AllLabels`, assert
+     `!strings.ContainsRune(label, ':')` for every constant.
+     The test is a no-op in PR-B (empty `AllLabels` ranges over
+     zero entries) but lands in PR-B so PR-C cannot populate
+     `AllLabels` with a `:`-containing label without the test
+     turning red. Pins the §5 hard constraint at compile time; a
+     future "redis:db0" would otherwise silently break
+     `bucket_id` parsing.
    - **`reclaimRetiredSlot` (RouteID, Label) dedupe**: retire
      `(routeID, "dynamo")`, re-register `(routeID, "dynamo")`,
      assert reclaim succeeds; then retire `(routeID, "dynamo")`

From 38e25bfec2cecddf04ab383fb747745499e03260 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 00:22:24 +0900
Subject: [PATCH 10/31] docs: address PR #694 round-8 (Claude bot: PR-B L0 +
 Sampler interface)

---
 docs/design/2026_04_28_proposed_keyviz_adapter_labels.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 8c51b9ec..ab489814 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -484,7 +484,7 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: `Observe(... label string)`, `routeSlot.Label`, slot-key type widens to `slotKey{uint64, string}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `LabelDynamo`/`LabelRedis`/etc. **constants** (declared but unused — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []string`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present but always empty. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
+| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label string)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, string}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `LabelDynamo`/`LabelRedis`/etc. **constants** (declared but unused — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []string`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
 | **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. PR-C **populates** `AllLabels` in `keyviz/labels.go` (the file itself was created in PR-B with an empty slice — see PR-B above) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extends `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** at PR-C deploy. (Earlier drafts told operators to scale the cap by `× (len(labels)+1)` — that was correct under Option B's slot-counting semantics but redundant under the chosen Option A. Codex round-6.) Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. PR-C also emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so a misconfigured cap surfaces in logs when it actually bites. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. |
 | **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) plus the SPA `route:N / label` rendering AND the **five** code changes the labeled wire format requires across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error and an obvious mismatch, but worth naming; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` — **first-level** Label copy, affects single-node and cluster deployments alike (every JSON response goes through this constructor); <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) and emit composite `bucket_id`. Without this, `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows — an API inconsistency for production users of the admin RPC. (Codex P2 on round-7.) <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting any subset leaves a window where labeled rows collapse — they're a single PR. (Claude bot moderate on PR #694 round-4 + round-5.) |
 

From 8d6d290450a24bdc98a67637dd1f4f8166889014 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 00:26:44 +0900
Subject: [PATCH 11/31] docs: address PR #694 round-8 (Claude bot: gRPC
 matrixToProto pivot-key)

---
 docs/design/2026_04_28_proposed_keyviz_adapter_labels.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index ab489814..0da03239 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -486,7 +486,7 @@ Virtual-bucket rows in the heatmap render as
 | **PR-A** | Land this design doc. |
 | **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label string)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, string}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `LabelDynamo`/`LabelRedis`/etc. **constants** (declared but unused — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []string`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
 | **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. PR-C **populates** `AllLabels` in `keyviz/labels.go` (the file itself was created in PR-B with an empty slice — see PR-B above) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extends `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** at PR-C deploy. (Earlier drafts told operators to scale the cap by `× (len(labels)+1)` — that was correct under Option B's slot-counting semantics but redundant under the chosen Option A. Codex round-6.) Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. PR-C also emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so a misconfigured cap surfaces in logs when it actually bites. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. |
-| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) plus the SPA `route:N / label` rendering AND the **five** code changes the labeled wire format requires across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error and an obvious mismatch, but worth naming; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` — **first-level** Label copy, affects single-node and cluster deployments alike (every JSON response goes through this constructor); <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) and emit composite `bucket_id`. Without this, `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows — an API inconsistency for production users of the admin RPC. (Codex P2 on round-7.) <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting any subset leaves a window where labeled rows collapse — they're a single PR. (Claude bot moderate on PR #694 round-4 + round-5.) |
+| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) plus the SPA `route:N / label` rendering AND the **five** code changes the labeled wire format requires across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error and an obvious mismatch, but worth naming; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` — **first-level** Label copy, affects single-node and cluster deployments alike (every JSON response goes through this constructor); <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same pivot-key widening as item (a) for the JSON handler; without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry and the second labeled row overwrites the first in the gRPC response, exactly mirroring the JSON handler bug item (a) prevents (Claude bot round-8 moderate); (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows — an API inconsistency for production users of the admin RPC. (Codex P2 on round-7 + Claude bot round-8.) <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting any subset leaves a window where labeled rows collapse — they're a single PR. (Claude bot moderate on PR #694 round-4 + round-5.) |
 
 PRs B and C are independent of the wire format; the heatmap will
 keep showing one row per route until D+E ship, but the per-label

From 5b83bd19a0bd05b59d4cd4c41e4ea4e510be1673 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 00:33:59 +0900
Subject: [PATCH 12/31] docs: address PR #694 round-9 (Codex P2 + carry-over)

Two items from round-9:

- Codex P2 line 598 (label catalog enforcement): the prior 9 Q2
  said "review burden catches accidental variants like DynamoDB
  vs dynamo" - that's a social contract, exactly what a type
  system can enforce. Switch to a typed `type Label string` with
  canonical constants of that type. Adapters that reference
  `keyviz.LabelDynamoDB` (typo) get a compile error because no
  such constant exists. Untyped literal strings still
  auto-convert (`s.Observe(..., "dynamo")` still compiles), so
  full enforcement needs a vet-style static check on call sites
  - that is a Phase-2-C+-or-later follow-up; the typed constants
  cover the high-probability typo case at compile time.
- 4.1.1, 5, 7 PR-B: propagate the typed Label through Observe
  signature, slotKey value field, routeSlot.Label, MatrixRow.Label.
  proto KeyVizRow.label and JSON KeyVizRow.Label stay plain
  string (wire-format flexibility for unknown labels across
  binary versions).

Also addresses Claude bot round-9 carry-over (gRPC rowsByID map
widening) - already shipped in 8d6d2904 (item (e1)).
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 76 ++++++++++++++-----
 1 file changed, 57 insertions(+), 19 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 0da03239..1bacbe1e 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -95,7 +95,7 @@ one based on hot-path cost vs. plumbing weight.
 Extend the sampler signature:
 
 ```go
-Observe(routeID uint64, op Op, keyLen, valueLen int, label string)
+Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)
 ```
 
 Empty `label` → existing behaviour, single row per route. Adapters
@@ -173,7 +173,7 @@ work.
 
 #### `MaxTrackedRoutes` semantics: count routes, not slots
 
-After the slot-key widens to `slotKey{uint64, string}`, the live
+After the slot-key widens to `slotKey{uint64, Label}`, the live
 table contains `len(AllLabels) + 1` slots per individually-tracked
 route — bare `len(next.slots)` no longer equals "number of
 routes." We pick **Option A (route-counting)**: the coarsening
@@ -234,7 +234,7 @@ threading it through a separate channel:
 type routeSlot struct {
     metaMu  sync.RWMutex
     RouteID uint64
-    Label   string         // new — empty for the legacy unlabeled slot
+    Label   Label          // new — empty for the legacy unlabeled slot, typed via keyviz/labels.go
     GroupID uint64         // Phase 2-C+, see PR-3a
     Start, End []byte
     ...
@@ -345,10 +345,17 @@ Three layers gain a label field. They are kept in sync at PR-D+E
 upstream still on PR-C never sees a wire shape mismatch:
 
 ```diff
- // keyviz/sampler.go — internal type, available from PR-B
+ // keyviz/sampler.go — internal type, available from PR-B.
+ // MatrixRow.Label uses the typed `keyviz.Label` so adapters
+ // referencing it via the `Sampler` interface can't accidentally
+ // hand-write a string literal that drifts from the canonical
+ // constant set. The proto and JSON shapes below use plain `string`
+ // because the wire form is operator-visible and we want to allow
+ // unknown labels through without a schema bump (an old binary
+ // returning a label a new SPA hasn't seen yet still renders).
  type MatrixRow struct {
    RouteID      uint64
-+  Label        string  // empty for legacy unlabeled traffic
++  Label        Label   // typed; see keyviz/labels.go
    Start, End   []byte
    …
  }
@@ -484,7 +491,7 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label string)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, string}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `LabelDynamo`/`LabelRedis`/etc. **constants** (declared but unused — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []string`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
+| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants** (declared but unused outside the sampler — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []Label`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
 | **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. PR-C **populates** `AllLabels` in `keyviz/labels.go` (the file itself was created in PR-B with an empty slice — see PR-B above) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extends `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** at PR-C deploy. (Earlier drafts told operators to scale the cap by `× (len(labels)+1)` — that was correct under Option B's slot-counting semantics but redundant under the chosen Option A. Codex round-6.) Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. PR-C also emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so a misconfigured cap surfaces in logs when it actually bites. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. |
 | **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) plus the SPA `route:N / label` rendering AND the **five** code changes the labeled wire format requires across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error and an obvious mismatch, but worth naming; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` — **first-level** Label copy, affects single-node and cluster deployments alike (every JSON response goes through this constructor); <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same pivot-key widening as item (a) for the JSON handler; without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry and the second labeled row overwrites the first in the gRPC response, exactly mirroring the JSON handler bug item (a) prevents (Claude bot round-8 moderate); (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows — an API inconsistency for production users of the admin RPC. (Codex P2 on round-7 + Claude bot round-8.) <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting any subset leaves a window where labeled rows collapse — they're a single PR. (Claude bot moderate on PR #694 round-4 + round-5.) |
 
@@ -584,19 +591,50 @@ extension is "switch on the field". (Reviewer notes from PR
    without parser ambiguity (split on the *first* `:` after
    `route`). (Gemini round-1 minor on PR #694.)
 2. **Label allocation discipline** — who owns the canonical label
-   set? Proposal: `keyviz/labels.go` exports `LabelDynamo`,
-   `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV` constants
-   alongside an `AllLabels []string` slice that ranges over them
-   in stable order. `RegisterRoute` iterates `AllLabels` to
-   pre-allocate the labeled slots — exposing the slice (rather
-   than reflecting over the file) keeps the iteration deterministic
-   and the test surface small. Constants live in the `keyviz`
-   package (not `adapter/`) so the sampler can read the full set
-   at registration time without an import cycle back to `adapter`.
-   Adapters refer to these constants; nothing stops a future
-   adapter from inventing its own label, but the review burden
-   of adding to the central file catches accidental variants
-   like `"DynamoDB"` vs `"dynamo"`.
+   set? Proposal: `keyviz/labels.go` defines a **typed string**
+   `type Label string` and exports the canonical constants of that
+   type:
+
+   ```go
+   type Label string
+   const (
+       LabelLegacy Label = ""        // legacy / unlabelled traffic, coordinator default
+       LabelDynamo Label = "dynamo"
+       LabelRedis  Label = "redis"
+       LabelS3     Label = "s3"
+       LabelSQS    Label = "sqs"
+       LabelRawKV  Label = "rawkv"
+   )
+   var AllLabels = []Label{LabelDynamo, LabelRedis, LabelS3, LabelSQS, LabelRawKV}
+   ```
+
+   `Observe` takes `label Label` (not `string`) and `RegisterRoute`
+   iterates `AllLabels` for pre-allocation. Constants live in the
+   `keyviz` package (not `adapter/`) so the sampler can read the
+   full set at registration time without an import cycle back to
+   `adapter`.
+
+   **Why a typed `Label` rather than plain `string`** (Codex round-9):
+   slot pre-creation iterates `AllLabels`, and `Observe` drops slot
+   misses silently — so any adapter that emits a label not in
+   `AllLabels` would silently lose its KeyViz samples. Earlier
+   drafts said "review burden catches accidental variants
+   (`DynamoDB` vs `dynamo`)" — that's a social contract, exactly
+   the kind of invariant a type system should enforce. With
+   `Label` as a named type, the natural call shape is
+   `s.Observe(..., keyviz.LabelDynamo)`; a typo
+   `keyviz.LabelDynamoDB` is a compile error because no such
+   constant exists. (Untyped string literals still auto-convert
+   to `Label`, so `s.Observe(..., "dynamo")` still compiles —
+   the type system doesn't fully prevent that path. Closing it
+   completely needs an additional vet-style static check that
+   greps `Observe(...)` call sites for non-constant labels; that
+   is a Phase-2-C+-or-later follow-up, not blocking PR-B.)
+
+   The result: typo'd constants fail at compile time (the high-
+   probability mistake), and the `AllLabels` slice acts as the
+   single source of truth for both `RegisterRoute`'s pre-creation
+   loop and any future "is this label known?" check.
 3. **Should the aggregator collapse same-route different-label
    rows for operators who don't want the breakdown?** Proposal:
    no — the SPA already lets operators pick which row to

From bda27ca0197f7de9e489cdb7d14cb29623fde9b6 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 00:36:53 +0900
Subject: [PATCH 13/31] docs: address PR #694 round-9 minor (Claude bot:
 AllLabels constants test + helper home)

---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 1bacbe1e..20cb8486 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -163,6 +163,14 @@ for _, label := range allLabelsWithLegacy() { // AllLabels ∪ {""}
 }
 ```
 
+`allLabelsWithLegacy()` is an unexported helper in
+**`keyviz/labels.go`** (same package as `sampler.go`) that returns
+`append(AllLabels, LabelLegacy)`. Co-locating it with `AllLabels`
+keeps the canonical set in one file and avoids `sampler.go`
+reconstructing the same expansion at every call site. The
+function is unexported because no caller outside `package
+keyviz` needs it. (Claude bot round-9 minor.)
+
 When `RegisterRoute` decides a route will be coarsened into a
 virtual bucket (i.e. it returns `false`), **no labeled slots are
 created** for that route — the `tbl.virtualForRoute[routeID]`
@@ -557,6 +565,18 @@ extension is "switch on the field". (Reviewer notes from PR
      turning red. Pins the §5 hard constraint at compile time; a
      future "redis:db0" would otherwise silently break
      `bucket_id` parsing.
+   - **All declared label constants are in `AllLabels`**
+     (PR-C regression guard, written in PR-B): range over the
+     named constants (`LabelDynamo`, `LabelRedis`, …) and assert
+     each appears in `AllLabels`. Catches the symmetric mistake
+     where PR-C adds `LabelNewAdapter = "newadapter"` to
+     `keyviz/labels.go` but forgets to append it to `AllLabels`
+     — the slot pre-creation loop wouldn't iterate over it,
+     `Observe(... LabelNewAdapter)` would miss `tbl.slots`, and
+     all KeyViz samples for that adapter would silently drop
+     in production. Skip when `len(AllLabels) == 0` (matching
+     the avoid-`:` test's pre-PR-C no-op pattern). (Claude bot
+     round-9.)
    - **`reclaimRetiredSlot` (RouteID, Label) dedupe**: retire
      `(routeID, "dynamo")`, re-register `(routeID, "dynamo")`,
      assert reclaim succeeds; then retire `(routeID, "dynamo")`

From 71a24f4b3cb035f6144fd32eba4416f53c8e0d1a Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 00:43:06 +0900
Subject: [PATCH 14/31] docs: address PR #694 round-10 (Codex P1+P2 + Claude
 minor)

---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 28 ++++++++++++-------
 1 file changed, 18 insertions(+), 10 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 20cb8486..5cc5c97e 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -140,8 +140,11 @@ preserves PR-B as a behavior-neutral refactor:
   routes after PR-B — no early coarsening, no virtual-bucket
   fold, no per-operator action required. (Codex P1 on round-5.)
 - After PR-C, slot count per route grows to `len(labels) + 1`.
-  Operators raise `--keyvizMaxTrackedRoutes` before deploying
-  PR-C per the §7 rollout note.
+  Operators **do not** need to raise `--keyvizMaxTrackedRoutes` —
+  the §4.1.1 Option A coarsening check divides slot count by
+  `len(AllLabels)+1`, so the operator-set cap continues to mean
+  "individual routes." (An earlier draft told operators to scale
+  the cap; that contradicted Option A. Codex round-10 P2.)
 
 Concretely PR-B's `RegisterRoute` body looks like:
 
@@ -500,14 +503,16 @@ Virtual-bucket rows in the heatmap render as
 |---|---|
 | **PR-A** | Land this design doc. |
 | **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants** (declared but unused outside the sampler — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []Label`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
-| **PR-C** | Adapter wiring: each adapter sets its own label at the dispatch entry into `ShardedCoordinator.Observe…`. PR-C **populates** `AllLabels` in `keyviz/labels.go` (the file itself was created in PR-B with an empty slice — see PR-B above) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extends `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** at PR-C deploy. (Earlier drafts told operators to scale the cap by `× (len(labels)+1)` — that was correct under Option B's slot-counting semantics but redundant under the chosen Option A. Codex round-6.) Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. PR-C also emits a structured `slog.Warn` from inside `RegisterRoute` at the moment a route is coarsened — `slog.Warn("route folded into virtual bucket", route_id=…, slots_used=…, max_tracked_routes=…)` — so a misconfigured cap surfaces in logs when it actually bites. A startup-time check would not catch this because routes are added dynamically (the catalog watcher calls `RegisterRoute` whenever a `SplitRange` lands), so a process that started below the cap can cross it later. |
-| **PR-D+E** | **Ship together.** Wire-format extension (proto + JSON `bucket_id` composite + optional `label`) plus the SPA `route:N / label` rendering AND the **five** code changes the labeled wire format requires across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error and an obvious mismatch, but worth naming; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` — **first-level** Label copy, affects single-node and cluster deployments alike (every JSON response goes through this constructor); <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same pivot-key widening as item (a) for the JSON handler; without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry and the second labeled row overwrites the first in the gRPC response, exactly mirroring the JSON handler bug item (a) prevents (Claude bot round-8 moderate); (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows — an API inconsistency for production users of the admin RPC. (Codex P2 on round-7 + Claude bot round-8.) <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting any subset leaves a window where labeled rows collapse — they're a single PR. (Claude bot moderate on PR #694 round-4 + round-5.) |
+| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. Populate `AllLabels` in `keyviz/labels.go` (created in PR-B with an empty slice) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). |
 
-PRs B and C are independent of the wire format; the heatmap will
-keep showing one row per route until D+E ship, but the per-label
-counts are already accumulating in the sampler so the wire
-extension is "switch on the field". (Reviewer notes from PR
-#694: Claude bot critical / moderate.)
+PR-B is independent of the bundled PR-C+D+E: PR-B widens types
+and prepares the slot machinery without changing observable
+behavior (legacy slots only, AllLabels empty, MaxTrackedRoutes
+divisor is 1). The bundled PR-C+D+E then ships everything
+operator-visible — adapter wiring, `AllLabels` populated, all
+five wire-format copies — in a single atomic deploy. (Reviewer
+notes from PR #694: Claude bot critical / moderate +
+Codex round-10.)
 
 ## 8. Five-lens checklist
 
@@ -558,7 +563,10 @@ extension is "switch on the field". (Reviewer notes from PR
      `dst := &KeyVizRow{...}` construction.
    - **Canonical labels avoid `:`** (PR-C regression guard,
      written in PR-B): range `AllLabels`, assert
-     `!strings.ContainsRune(label, ':')` for every constant.
+     `!strings.ContainsRune(string(label), ':')` for every
+     constant — explicit `string(label)` cast required because
+     `AllLabels []Label` is the typed slice and
+     `strings.ContainsRune` takes `string`.
      The test is a no-op in PR-B (empty `AllLabels` ranges over
      zero entries) but lands in PR-B so PR-C cannot populate
      `AllLabels` with a `:`-containing label without the test

From 5a14dd6bb352360474d0d67fa6a4554e54471ddd Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 00:51:20 +0900
Subject: [PATCH 15/31] docs: address PR #694 round-11 (Claude moderate + Codex
 P2 rolling-upgrade gate)

---
 ...2026_04_28_proposed_keyviz_adapter_labels.md | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 5cc5c97e..e3274138 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -107,10 +107,13 @@ keyed by `(routeID, label)` instead of just `routeID`. The map
 key is a struct of `{uint64, string}`; the `label` is a small
 interned constant (`"dynamo"`), so allocation should be zero.
 
-Storage: each (route, label) gets its own `routeSlot`. With 5
-adapters and a 1024-route budget, the worst case is 5120 slots
-per node — still well below the existing `MaxTrackedRoutes` cap
-when operators raise it for label use.
+Storage: each (route, label) gets its own `routeSlot`. Total
+slot memory is `MaxTrackedRoutes × (len(AllLabels)+1)` — bounded
+and documented. `MaxTrackedRoutes` retains its route-counting
+semantics under Option A (§4.1.1), so no operator action is
+required at PR-C+D+E rollout. (Vestigial "when operators raise
+it for label use" framing was rejected — Codex round-10 P2 +
+Claude bot round-11.)
 
 #### 4.1.1 Slot lifecycle and the lockless invariant
 
@@ -442,7 +445,7 @@ parsers split on the *first* `:` after `route`.
 `:`. The constants in `keyviz/labels.go` (§9 Q2) are the only
 source of label values, and we enforce this at the constants
 file with a `func TestAllLabelsAvoidSeparator(t *testing.T)` test
-that asserts `!strings.ContainsRune(l, ':')` for every member of
+that asserts `!strings.ContainsRune(string(l), ':')` for every member of
 `AllLabels`. Without this constraint a future label like
 `"redis:db0"` would silently break `bucket_id` parsing — the
 parser would split at the wrong `:`. (CodeRabbit major on PR
@@ -503,7 +506,7 @@ Virtual-bucket rows in the heatmap render as
 |---|---|
 | **PR-A** | Land this design doc. |
 | **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants** (declared but unused outside the sampler — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []Label`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
-| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. Populate `AllLabels` in `keyviz/labels.go` (created in PR-B with an empty slice) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). |
+| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. Populate `AllLabels` in `keyviz/labels.go` (created in PR-B with an empty slice) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. PR-C+D+E adds `--keyvizLabelsEnabled` (default `false`) on every node; when false, adapters call `Observe(..., keyviz.LabelLegacy)` regardless of which adapter they are. The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. Once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true` (one flag-flip rolling restart), and all nodes simultaneously start emitting the labeled format. This decouples binary rollout from feature activation and prevents the mixed-version fragmentation in cluster heatmaps. (Codex round-11 P2.) |
 
 PR-B is independent of the bundled PR-C+D+E: PR-B widens types
 and prepares the slot machinery without changing observable
@@ -626,7 +629,7 @@ Codex round-10.)
    ```go
    type Label string
    const (
-       LabelLegacy Label = ""        // legacy / unlabelled traffic, coordinator default
+       LabelLegacy Label = ""        // legacy / unlabeled traffic, coordinator default
        LabelDynamo Label = "dynamo"
        LabelRedis  Label = "redis"
        LabelS3     Label = "s3"

From 545b1a16faa8dc8ac0d25b6cfd1823819e67beeb Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 01:00:04 +0900
Subject: [PATCH 16/31] docs: address PR #694 round-12 (Claude bot:
 rollout-gate refinements)

---
 docs/design/2026_04_28_proposed_keyviz_adapter_labels.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index e3274138..f1bc14e5 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -506,7 +506,7 @@ Virtual-bucket rows in the heatmap render as
 |---|---|
 | **PR-A** | Land this design doc. |
 | **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants** (declared but unused outside the sampler — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []Label`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
-| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. Populate `AllLabels` in `keyviz/labels.go` (created in PR-B with an empty slice) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. PR-C+D+E adds `--keyvizLabelsEnabled` (default `false`) on every node; when false, adapters call `Observe(..., keyviz.LabelLegacy)` regardless of which adapter they are. The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. Once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true` (one flag-flip rolling restart), and all nodes simultaneously start emitting the labeled format. This decouples binary rollout from feature activation and prevents the mixed-version fragmentation in cluster heatmaps. (Codex round-11 P2.) |
+| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. Populate `AllLabels` in `keyviz/labels.go` (created in PR-B with an empty slice) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. PR-C+D+E adds `--keyvizLabelsEnabled` (default `false`) on every node; when false, the **`ShardedCoordinator` overrides the adapter-supplied label to `keyviz.LabelLegacy` at the single `sampler.Observe(...)` call site** in `kv/sharded_coordinator.go` (one `if !s.keyvizLabelsEnabled { label = keyviz.LabelLegacy }` guard, not a 5-file flag-read duplication across adapters; Claude bot round-12 moderate). The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. <br>**Flag-flip activation**: once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true`. The flag is a startup-only `flag.Bool` (no live-toggle / config-reload path; Claude bot round-12 minor); changing it requires a process restart. There are two restart strategies: (a) **simultaneous restart of all nodes** — KeyViz heatmap is briefly unavailable during the restart window but no mixed-format fragmentation occurs; (b) **rolling restart** — for the duration of the restart (typically minutes), the heatmap shows a transient mixed view because the legacy `route:N` rows from not-yet-restarted nodes don't merge with `route:N:label` rows from restarted nodes. Since KeyViz is a monitoring view (not a consistency-sensitive system), the rolling-restart fragmentation is acceptable and clears as the final node restarts; operators who want zero fragmentation should use the simultaneous restart. (Claude bot round-12 moderate.) <br>**Memory-vs-flag separation**: `--keyvizLabelsEnabled` controls whether labeled traffic flows into the pre-created labeled slots — it does **not** defer slot allocation. Labeled slots are pre-created by `RegisterRoute` whenever `AllLabels` is non-empty (i.e., from PR-C+D+E deploy time onward, regardless of the flag), so memory grows to `MaxTrackedRoutes × (len(AllLabels)+1)` at binary deploy time. The flag is a traffic-routing toggle, not a slot-pre-creation toggle. (Claude bot round-12 minor.) (Codex round-11 P2 originated this rollout gate.) |
 
 PR-B is independent of the bundled PR-C+D+E: PR-B widens types
 and prepares the slot machinery without changing observable

From f6c4dd70672ea7b793585e5c0cc039220e1b0f17 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 02:07:13 +0900
Subject: [PATCH 17/31] docs: address PR #694 round-13 (Copilot review)

Three items from Copilot's first review pass on `1c18d2e6`:

- 7 / line 384 proto comment claim: doc said "the existing
  comment already reserves it for 'future per-Observe label'",
  but proto/admin.proto:95 actually declares
  `string label = 4;` with no doc comment, and the nearby
  bucket_id comment (line 91) only lists the legacy
  `route:<id>` / `virtual:<id>` forms. Replaced the false
  claim with an accurate description: PR-D+E will start
  filling the field AND add the proto comments
  (label-field doc comment + bucket_id comment update for
  the `route:<id>:<label>` composite) in the same commit.
  Updated the inline `// currently unused` comment to
  `// (no doc comment today)` so the doc accurately
  reflects today's proto and the planned diff.
- PR description location inconsistency: doc said
  `keyviz/labels.go`, PR description said
  `adapter/keyviz_labels.go`. Picked `keyviz/labels.go`
  (sampler-internal package, where slotKey + Label type
  also live) and edited the PR description to match.
- 1 / line 9 Status/Author/Date header: docs/design/README.md
  requires `Status: ... / Author: ... / Date: YYYY-MM-DD`
  under the title; this doc had only YAML frontmatter.
  Added the markdown header (Status: Proposed / Author:
  bootjp / Date: 2026-04-28) preserving the YAML for
  filename-rename automation.

Copilot's fourth item (markdown table `||` extra column at
implementation-plan rows) is a misreport: `grep -c '^||'`
returns 0 and `grep -c '^| '` returns 4; the table at
lines 504-509 uses standard single-pipe `| col |` format
already. No change.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 28 +++++++++++++------
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index f1bc14e5..9ce544a4 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -7,6 +7,10 @@ date: 2026-04-28
 
 # KeyViz adapter / namespace labels
 
+Status: Proposed
+Author: bootjp
+Date: 2026-04-28
+
 ## 1. Background
 
 Phase 2-A through 2-C ship a fully functional KeyViz heatmap, but
@@ -377,18 +381,26 @@ upstream still on PR-C never sees a wire shape mismatch:
  // proto/admin.proto — wire form, PR-D+E.
  //
  // Reuse the **existing** field 4 (`string label`), which is
- // already declared on KeyVizRow but currently unused — no
- // schema migration, no field-number bump. The existing comment
- // already reserves it for "future per-Observe label", which is
- // exactly this proposal's payload. (CodeRabbit critical on PR
- // #694 caught this — an earlier draft proposed adding a
- // duplicate `string label = 13;` which would have collided.)
+ // already declared on KeyVizRow but currently unused and
+ // uncommented — no schema migration, no field-number bump.
+ // The current proto declares `string label = 4;` with no
+ // doc comment, and the nearby `bucket_id` comment only mentions
+ // the legacy `route:<id>` / `virtual:<id>` forms. PR-D+E starts
+ // filling the field AND updates the proto comments in the same
+ // commit (both the new label-field doc comment and the
+ // `bucket_id` comment so it lists the composite
+ // `route:<id>:<label>` form). (CodeRabbit critical on PR
+ // #694 caught the duplicate-field-13 mistake — an earlier draft
+ // proposed adding `string label = 13;` which would have collided
+ // with the existing field 4. Copilot review surfaced the
+ // related "comment already reserves it" misstatement — there is
+ // no such reserving comment yet; PR-D+E adds it.)
  message KeyVizRow {
    string bucket_id = 1;
    bytes start = 2;
    bytes end = 3;
--  string label = 4;        // currently unused
-+  string label = 4;        // PR-D+E: filled with the per-Observe label
+-  string label = 4;        // (no doc comment today)
++  string label = 4;        // PR-D+E: per-Observe label; empty for legacy / virtual rows
    …
  }
 

From de7dad556e7ecf37e89531c340178ff96b2117b0 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 02:15:55 +0900
Subject: [PATCH 18/31] =?UTF-8?q?docs:=20address=20PR=20#694=20round-14=20?=
 =?UTF-8?q?(Codex=20P2=20=C2=A76=20contradiction)?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

One P2 item from Codex review on `f6c4dd70`:

- §6 / line 494 "merge logic needs no changes" contradicts
  §7 PR-C+D+E item (d). Old wording: "the aggregator's merge
  logic does not need to change at all once bucketID is
  composite". §7 (item d) requires `mergeRowInto`
  (`internal/admin/keyviz_fanout.go:509`) to add
  `dst.Label = row.Label`. An implementer following §6
  literally skips that copy → labels work on single-node
  responses but get silently dropped (`Label=""`) after
  fan-out merge, undermining the feature for clustered
  deployments.

Split the §6 statement into two: the merge KEY logic is
unchanged (composite bucketID already separates same-route
different-label rows correctly), but the merge VALUE path
still needs the explicit `dst.Label = row.Label` field copy
to propagate the label into the merged output. Cross-link to
the item (d) in §7 PR-C+D+E so a reader of either section
finds the same instruction.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 28 ++++++++++++++-----
 1 file changed, 21 insertions(+), 7 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 9ce544a4..d9145f18 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -490,13 +490,27 @@ The fan-out aggregator's per-cell merge key gains the label:
   same-route different-label rows correctly.
 
 Reads still sum, writes still max-with-conflict; nothing about
-the merge rules changes other than the wire shape of `bucketID`.
-The aggregator's merge logic does not need to change at all once
-`bucketID` is composite; that's why PR-D and PR-E are described
-as **a single PR-D+E** in §7 rather than two separate PRs. (An
-earlier draft of this section claimed they were "shippable in
-either order" — that contradicted §7's "ship together"
-requirement; resolved in favour of §7. Codex round-6.)
+the merge **rules** changes other than the wire shape of
+`bucketID`. The merge **key** logic is unchanged: composite
+`bucketID` already separates same-route different-label rows
+correctly, so the aggregator's bucketing/grouping path needs no
+edits.
+
+The merge **value** path, however, still needs one explicit
+field copy: `mergeRowInto` (`internal/admin/keyviz_fanout.go:509`)
+must add `dst.Label = row.Label` so the label propagates from the
+per-node `KeyVizRow` into the merged output. Without that copy,
+labels work on single-node responses but are silently dropped
+(`Label = ""`) after fan-out merge — the feature would be invisible
+in clustered deployments. This is item (d) ("second-level Label
+copy") in §7 PR-C+D+E. PR-D and PR-E are still bundled as a single
+PR-D+E because the wire-format and pivot-key changes are
+inseparable; the `mergeRowInto` value copy is part of the same
+bundle. (An earlier draft claimed PR-D and PR-E were "shippable in
+either order" — contradicted §7's "ship together" requirement;
+resolved in favour of §7. Codex round-6. Codex round-13 caught
+the related "merge logic needs no changes" wording, which had
+implied no `mergeRowInto` edit at all.)
 
 ### 6.5 Virtual buckets and labels
 

From 14f7b98a66843b2ba56dfc179468571318d2ea67 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 02:26:31 +0900
Subject: [PATCH 19/31] docs: address PR #694 round-15 (Copilot 2nd pass)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Three of four items from Copilot's second review pass:

- §4.1 line 109 hot-path cost description: previous wording
  said "one extra map lookup per Observe", but the existing
  code already does a single tbl.slots[routeID] lookup
  (with a virtualForRoute fallback on miss only). The slot
  key widening from uint64 to {uint64, Label} keeps the same
  lookup count; only the key shape changes (slightly wider
  hash + equality work per lookup, same miss rate because
  PR-C+D+E pre-creates labeled slots at RegisterRoute time).
  Reworded to reflect the real change.
- Frontmatter `phase: keyviz / follow-up`: Copilot pointed
  out other KeyViz design docs use machine-friendly tokens
  like `2-B` / `2-C` (see 2026_04_27_proposed_keyviz_spa_-
  integration.md and 2026_04_27_proposed_keyviz_cluster_-
  fanout.md). Switched to `phase: 2-C+` to fit the
  established scheme — this design IS a follow-up
  enhancement on phase 2-C.
- Status/Author/Date duplication: round-13 added a body
  metadata block per docs/design/README.md "Document header"
  recommendation, but Copilot's second pass observed that
  this duplicates the YAML frontmatter (`status`, `date`),
  inviting drift, AND that other KeyViz docs use frontmatter
  only. Resolved by dropping the body block and adding
  `author: bootjp` to the frontmatter — single source of
  truth, consistent with sibling KeyViz docs. The README's
  "body header" recommendation is divergent from actual
  practice across the design directory; a separate PR
  could update the README, but is out of scope here.

The fourth Copilot item (line 535 implementation-plan table
"`||` prefix on each row") is a repeated misreport: the
table at line 533 onwards uses standard `| col | col |`
format. `grep -c '^||'` still returns 0 against current
HEAD; `grep -c '^| '` returns 4 (header + separator + 2
data rows that aren't `<br>`-padded; the PR-B + PR-C+D+E
data rows are dense single-pipe rows that don't wrap). No
change.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 25 ++++++++++++-------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index d9145f18..7c75ee0d 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -1,16 +1,13 @@
 ---
 status: proposed
-phase: keyviz / follow-up
+phase: 2-C+
 parent_design: docs/admin_ui_key_visualizer_design.md
+author: bootjp
 date: 2026-04-28
 ---
 
 # KeyViz adapter / namespace labels
 
-Status: Proposed
-Author: bootjp
-Date: 2026-04-28
-
 ## 1. Background
 
 Phase 2-A through 2-C ship a fully functional KeyViz heatmap, but
@@ -106,10 +103,20 @@ Empty `label` → existing behaviour, single row per route. Adapters
 set their own constant string (`"dynamo"`, `"redis"`, …) at the
 dispatch site they already own.
 
-Cost on hot path: one extra map lookup per Observe — slot is now
-keyed by `(routeID, label)` instead of just `routeID`. The map
-key is a struct of `{uint64, string}`; the `label` is a small
-interned constant (`"dynamo"`), so allocation should be zero.
+Cost on hot path: still one `slots` map lookup per `Observe` on
+the hit path (the existing code already does a single
+`tbl.slots[routeID]` lookup, with a fallback into
+`virtualForRoute` only on miss). What changes is the **key
+shape**: `(routeID, label)` instead of just `routeID`, so the
+map key is a `{uint64, Label}` struct — slightly wider hash and
+equality work per lookup. `label` is a small interned constant
+(`"dynamo"`), so the lookup itself allocates nothing. The miss
+rate is unchanged because PR-C+D+E pre-creates a slot for every
+member of `AllLabels` at `RegisterRoute` time, so labeled
+`Observe` calls hit the same way today's unlabeled calls do.
+(Copilot round-15 caught the earlier "one extra map lookup"
+phrasing — the lookup count is unchanged; only the key shape
+widens.)
 
 Storage: each (route, label) gets its own `routeSlot`. Total
 slot memory is `MaxTrackedRoutes × (len(AllLabels)+1)` — bounded

From 6ee41469ee715b742e907069c7882b5e01c7498f Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 02:39:52 +0900
Subject: [PATCH 20/31] docs: address PR #694 round-16 (Copilot 3rd pass)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Two items from Copilot's third review pass on `14f7b98a`:

- §4.1.1 / line 132 "no fallback" wording: the original
  text said "The current `Observe` is `Load → map-lookup
  → atomic.Add` with no fallback: a miss is silently
  dropped." Reality: current `MemSampler.Observe` falls
  back to `tbl.virtualForRoute[...]` on `tbl.slots[...]`
  miss; only the both-miss case drops the sample. The
  invariant we are preserving is **not** "no fallback" —
  it is "`Observe` never allocates and never creates
  slots on the hot path". Reworded to describe the actual
  fallback behavior and pin the no-allocate invariant
  separately.
- §8 / line 564 Performance lens "One extra map lookup
  per Observe" contradicts §4.1 (line 109) which I just
  fixed in round-15 to clarify the lookup count is
  unchanged and only the key shape widens. Aligned the
  Performance lens text with §4.1: "still one map lookup
  per Observe; only extra work is wider hash and equality
  on `{uint64, Label}` vs `uint64`".
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 34 +++++++++++++------
 1 file changed, 23 insertions(+), 11 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 7c75ee0d..49500ef6 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -128,12 +128,19 @@ Claude bot round-11.)
 
 #### 4.1.1 Slot lifecycle and the lockless invariant
 
-The current `Observe` is `Load → map-lookup → atomic.Add` with no
-fallback: a miss is silently dropped. This is intentional —
-`RegisterRoute` pre-creates every slot before traffic arrives, so
-the hot path never needs to allocate. We keep that invariant by
-making `RegisterRoute` **pre-create one slot per known label** at
-registration time, but only on the individual-tracking path. The
+The current `Observe` is a lockless `Load → lookup → atomic.Add`
+hot path. On a miss in `tbl.slots[...]`, it may fall back to
+`tbl.virtualForRoute[...]`; if both lookups miss, the sample is
+silently dropped. The invariant is **not** "no fallback" — it is
+"`Observe` never allocates and never creates slots on the hot
+path". `RegisterRoute` pre-creates every slot before traffic
+arrives, so the hot path never needs to allocate. We keep that
+invariant by making `RegisterRoute` **pre-create one slot per
+known label** at registration time, but only on the
+individual-tracking path. (Copilot round-15-3rd-pass caught the
+earlier "no fallback" wording — the fallback exists today and
+must be preserved; what stays unchanged is the no-allocate
+property.) The
 label set is the canonical `keyviz/labels.go` constants (§9 Q2);
 per-route slot count is `len(labels) + 1` (the +1 is the empty-
 label legacy slot, kept so callers that pass `label=""` still hit
@@ -561,11 +568,16 @@ Codex round-10.)
    in the routesMu COW path; the hot-path Load + map lookup keeps
    the same shape (one lookup, one atomic add). Burst test
    updates: parametrise on (route, label) instead of just route.
-3. **Performance** — One extra map lookup per Observe via the
-   wider key. `BenchmarkObserveParallel` already pins the hot-path
-   cost; the new bench should land within run-to-run variance of
-   the current 4 ns/op. If it doesn't, the design is wrong and we
-   fall back to Option 4.2.
+3. **Performance** — The hot path still does one map lookup per
+   `Observe` (same as today); the only extra work is wider hash
+   and equality on the `{uint64, Label}` key vs the current
+   `uint64` key. `BenchmarkObserveParallel` already pins the
+   hot-path cost; the new bench should land within run-to-run
+   variance of the current 4 ns/op. If it doesn't, the design is
+   wrong and we fall back to Option 4.2. (Copilot round-15-3rd-
+   pass caught the earlier "one extra map lookup" wording, which
+   contradicted §4.1's correctly-described "same lookup count,
+   wider key shape".)
 4. **Data consistency** — Cluster fan-out merge gains a tuple
    field; the dedup invariant (per-cell, per-(route, label,
    group, term, window)) still holds. Old SPA against new server

From cee5460400d9fb43b583b19a022e11a8a94e7a3d Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 02:52:05 +0900
Subject: [PATCH 21/31] docs: address PR #694 round-17 (Codex P1 + Copilot 4th
 pass)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Three items from reviews on `6ee41469`:

- Codex P1 (line 549) — pre-allocation while labels disabled:
  the prior "memory-vs-flag separation" framing (round-12)
  said labeled slots are pre-created whenever AllLabels is
  populated, regardless of the flag. Codex P1 caught that this
  imposes a multi-x memory penalty on clusters that deploy the
  bundled binary but never enable labels, contradicting the
  §2.1 minimal-penalty goal. Reversed: --keyvizLabelsEnabled
  now gates BOTH traffic-routing AND slot pre-creation. With
  the flag off, RegisterRoute pre-creates only the legacy
  empty-label slot (today's baseline). Updated §4.1
  (slot-memory description), §4.1.1 (RegisterRoute behavior +
  PR-C body pseudocode + MaxTrackedRoutes divisor), and §7
  PR-C+D+E rollout row (Adapter wiring + Memory growth +
  appended "Flag also gates pre-allocation" subsection
  replacing the rejected "Memory-vs-flag separation" claim).
- Copilot (line 46) §2.1 "Zero hot-path penalty" overstated:
  the slots map key widens from uint64 to {uint64, Label}
  regardless of flag state, so hash/equality work is
  slightly wider even when labels are disabled. Reworded to
  "minimal hot-path penalty: no additional lookups and no
  additional allocations, with only the slightly wider hash
  and equality work that comes from the slotKey{uint64,
  Label} map key compared to today's bare uint64".
- Copilot (line 196) allLabelsWithLegacy() slice-reuse
  footgun: append(AllLabels, LabelLegacy) may reuse
  AllLabels' backing array, which is a bug if AllLabels is
  later appended to (the legacy entry can be overwritten).
  Specified the helper must return a freshly allocated
  slice (allocate new []Label of length len(AllLabels)+1,
  copy AllLabels, then append LabelLegacy).

The §4.1.1 PR-C pseudocode now reads `labelsToCreate`
(flag-gated, defaults to {LabelLegacy} only, becomes
allLabelsWithLegacy() when the flag is on) instead of an
unconditional loop over all labels. The MaxTrackedRoutes
coarsening check uses runtime `slotsPerRoute` (1 or
len(AllLabels)+1 depending on the flag) so the operator-set
cap continues to count routes in both regimes.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 122 ++++++++++++------
 1 file changed, 82 insertions(+), 40 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 49500ef6..6245e95a 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -42,8 +42,14 @@ breakdown in the heatmap.
 - Attribution **inside a single route**: a row that today reads
   `route:1, total=378` should optionally split into sub-rows like
   `route:1 / dynamo`, `route:1 / s3`, `route:1 / redis`, …
-- Zero hot-path penalty when labels are **not** configured: a
-  default deployment continues to behave exactly as today.
+- Minimal hot-path penalty when labels are **not** configured: a
+  default deployment incurs **no additional lookups and no
+  additional allocations**, with only the slightly wider hash
+  and equality work that comes from the `slotKey{uint64, Label}`
+  map key compared to today's bare `uint64`. (Copilot
+  round-15-4th-pass observed that the original "zero penalty"
+  framing was overstated — the key shape widens regardless of
+  whether labels are enabled.)
 - Adapter-side wiring is the natural place to set labels (every
   adapter already has the dispatch entry into
   `ShardedCoordinator.Observe…` — see `kv/sharded_coordinator.go`).
@@ -118,9 +124,14 @@ member of `AllLabels` at `RegisterRoute` time, so labeled
 phrasing — the lookup count is unchanged; only the key shape
 widens.)
 
-Storage: each (route, label) gets its own `routeSlot`. Total
-slot memory is `MaxTrackedRoutes × (len(AllLabels)+1)` — bounded
-and documented. `MaxTrackedRoutes` retains its route-counting
+Storage: each (route, label) gets its own `routeSlot` **when
+labels are enabled** (`--keyvizLabelsEnabled=true`). Total slot
+memory is `MaxTrackedRoutes × (len(AllLabels)+1)` with labels
+enabled, and `MaxTrackedRoutes × 1` with labels disabled (legacy
+slot only — same as today's baseline). The pre-allocation cost is
+gated on the flag, so a cluster that deploys the PR-C+D+E binary
+but leaves the flag off pays no memory penalty (Codex round-15-
+4th-pass P1). `MaxTrackedRoutes` retains its route-counting
 semantics under Option A (§4.1.1), so no operator action is
 required at PR-C+D+E rollout. (Vestigial "when operators raise
 it for label use" framing was rejected — Codex round-10 P2 +
@@ -136,16 +147,20 @@ silently dropped. The invariant is **not** "no fallback" — it is
 path". `RegisterRoute` pre-creates every slot before traffic
 arrives, so the hot path never needs to allocate. We keep that
 invariant by making `RegisterRoute` **pre-create one slot per
-known label** at registration time, but only on the
-individual-tracking path. (Copilot round-15-3rd-pass caught the
-earlier "no fallback" wording — the fallback exists today and
-must be preserved; what stays unchanged is the no-allocate
-property.) The
-label set is the canonical `keyviz/labels.go` constants (§9 Q2);
-per-route slot count is `len(labels) + 1` (the +1 is the empty-
-label legacy slot, kept so callers that pass `label=""` still hit
-a slot). Total individual slots in the table is therefore
-`len(routes) × (len(labels) + 1)`.
+known label** at registration time when `--keyvizLabelsEnabled=true`,
+but only on the individual-tracking path. (Copilot round-15-3rd-pass
+caught the earlier "no fallback" wording — the fallback exists today
+and must be preserved; what stays unchanged is the no-allocate
+property.) The label set is the canonical `keyviz/labels.go`
+constants (§9 Q2); per-route slot count is `len(labels) + 1` when
+labels are enabled (the +1 is the empty-label legacy slot, kept so
+callers that pass `label=""` still hit a slot) and `1` (legacy slot
+only) when labels are disabled. Total individual slots in the table
+is therefore `len(routes) × (len(labels) + 1)` with labels enabled
+and `len(routes) × 1` with labels disabled. (Codex round-15-4th-pass
+P1 caught the earlier framing that pre-allocated labeled siblings
+even when the flag was off — that imposed an unnecessary multi-x
+memory penalty on clusters that never enabled labels.)
 
 #### Pre-allocation is gated on PR-C, not PR-B
 
@@ -160,12 +175,20 @@ preserves PR-B as a behavior-neutral refactor:
   A deployment running 1024 routes today still supports 1024
   routes after PR-B — no early coarsening, no virtual-bucket
   fold, no per-operator action required. (Codex P1 on round-5.)
-- After PR-C, slot count per route grows to `len(labels) + 1`.
-  Operators **do not** need to raise `--keyvizMaxTrackedRoutes` —
-  the §4.1.1 Option A coarsening check divides slot count by
-  `len(AllLabels)+1`, so the operator-set cap continues to mean
-  "individual routes." (An earlier draft told operators to scale
-  the cap; that contradicted Option A. Codex round-10 P2.)
+- After PR-C with `--keyvizLabelsEnabled=true`, slot count per
+  route grows to `len(labels) + 1`. Operators **do not** need to
+  raise `--keyvizMaxTrackedRoutes` — the §4.1.1 Option A
+  coarsening check divides slot count by `slotsPerRoute` (which
+  is `len(AllLabels)+1` when the flag is on, `1` when off), so
+  the operator-set cap continues to mean "individual routes."
+  (An earlier draft told operators to scale the cap; that
+  contradicted Option A. Codex round-10 P2.)
+- After PR-C with `--keyvizLabelsEnabled=false` (the default
+  immediately after deploying the bundled binary), slot count per
+  route is **1** (legacy slot only) — `RegisterRoute` skips
+  labeled-sibling pre-creation, so memory matches today's
+  baseline. The flag-flip restart introduces the labeled siblings.
+  (Codex round-15-4th-pass P1.)
 
 Concretely PR-B's `RegisterRoute` body looks like:
 
@@ -177,8 +200,12 @@ if slot == nil {
 }
 next.slots[slotKey{RouteID: routeID, Label: ""}] = slot
 
-// PR-C extends to:
-for _, label := range allLabelsWithLegacy() { // AllLabels ∪ {""}
+// PR-C extends to (flag-gated, Codex round-15-4th-pass P1):
+labelsToCreate := []Label{LabelLegacy} // legacy slot always
+if s.opts.KeyVizLabelsEnabled {
+    labelsToCreate = allLabelsWithLegacy() // AllLabels ∪ {LabelLegacy}, fresh slice
+}
+for _, label := range labelsToCreate {
     slot := s.reclaimRetiredSlot(routeID, label) // reclaim per (routeID, label), not just legacy
     if slot == nil {
         slot = newSlot(routeID, label, start, end, groupID)
@@ -189,11 +216,19 @@ for _, label := range allLabelsWithLegacy() { // AllLabels ∪ {""}
 
 `allLabelsWithLegacy()` is an unexported helper in
 **`keyviz/labels.go`** (same package as `sampler.go`) that returns
-`append(AllLabels, LabelLegacy)`. Co-locating it with `AllLabels`
-keeps the canonical set in one file and avoids `sampler.go`
-reconstructing the same expansion at every call site. The
-function is unexported because no caller outside `package
-keyviz` needs it. (Claude bot round-9 minor.)
+a **freshly allocated slice** containing `AllLabels` followed by
+`LabelLegacy`. Concretely: allocate a new `[]Label` of length
+`len(AllLabels)+1`, copy `AllLabels` into it, then append
+`LabelLegacy`. Naively returning `append(AllLabels, LabelLegacy)`
+would be a footgun — Go's `append` may reuse the backing array of
+`AllLabels`, which would then visibly mutate when a future caller
+appends to `AllLabels` directly (the previously written legacy
+entry could be overwritten). Co-locating the helper with
+`AllLabels` keeps the canonical set in one file and avoids
+`sampler.go` reconstructing the same expansion at every call site.
+The function is unexported because no caller outside `package
+keyviz` needs it. (Claude bot round-9 minor; Copilot
+round-15-4th-pass added the fresh-slice safety requirement.)
 
 When `RegisterRoute` decides a route will be coarsened into a
 virtual bucket (i.e. it returns `false`), **no labeled slots are
@@ -207,19 +242,26 @@ work.
 
 After the slot-key widens to `slotKey{uint64, Label}`, the live
 table contains `len(AllLabels) + 1` slots per individually-tracked
-route — bare `len(next.slots)` no longer equals "number of
-routes." We pick **Option A (route-counting)**: the coarsening
-check at `keyviz/sampler.go:416` divides by `len(AllLabels) + 1`
-so `MaxTrackedRoutes` continues to count routes, not slots. This
-preserves the operator-visible meaning of the existing flag —
-`--keyvizMaxTrackedRoutes=1024` still means "1024 individual
-routes," not "1024 slots that may be ~170 routes." Total slot
-memory is `MaxTrackedRoutes × (len(AllLabels) + 1)`, which we
-document but never count against the cap.
+route when `--keyvizLabelsEnabled=true` (and `1` slot per route
+when the flag is off) — bare `len(next.slots)` no longer equals
+"number of routes." We pick **Option A (route-counting)**: the
+coarsening check at `keyviz/sampler.go:416` divides by
+`slotsPerRoute`, which is `len(AllLabels) + 1` when the flag is
+enabled and `1` when off. `MaxTrackedRoutes` continues to count
+routes, not slots, in both regimes. This preserves the operator-
+visible meaning of the existing flag — `--keyvizMaxTrackedRoutes=1024`
+still means "1024 individual routes," not "1024 slots that may be
+~170 routes." Total slot memory is `MaxTrackedRoutes ×
+slotsPerRoute` (i.e. `MaxTrackedRoutes × (len(AllLabels) + 1)`
+when labels are enabled, `MaxTrackedRoutes × 1` when off), which
+we document but never count against the cap.
 
 ```go
-// keyviz/sampler.go:416 (PR-B)
-slotsPerRoute := len(keyviz.AllLabels) + 1 // 1 today (empty AllLabels), 6 after PR-C
+// keyviz/sampler.go:416 (PR-B + PR-C+D+E flag-gated, Codex round-15-4th-pass P1)
+slotsPerRoute := 1 // legacy slot only (PR-B, or PR-C+D+E with flag off)
+if s.opts.KeyVizLabelsEnabled {
+    slotsPerRoute = len(keyviz.AllLabels) + 1 // 6 after PR-C with flag on
+}
 if len(next.slots) / slotsPerRoute < s.opts.MaxTrackedRoutes {
     // ... pre-create slot(s) for this route
 }
@@ -546,7 +588,7 @@ Virtual-bucket rows in the heatmap render as
 |---|---|
 | **PR-A** | Land this design doc. |
 | **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants** (declared but unused outside the sampler — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []Label`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
-| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. Populate `AllLabels` in `keyviz/labels.go` (created in PR-B with an empty slice) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) alongside the legacy empty-label slot. **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `len(AllLabels)+1`, so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots, documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. PR-C+D+E adds `--keyvizLabelsEnabled` (default `false`) on every node; when false, the **`ShardedCoordinator` overrides the adapter-supplied label to `keyviz.LabelLegacy` at the single `sampler.Observe(...)` call site** in `kv/sharded_coordinator.go` (one `if !s.keyvizLabelsEnabled { label = keyviz.LabelLegacy }` guard, not a 5-file flag-read duplication across adapters; Claude bot round-12 moderate). The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. <br>**Flag-flip activation**: once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true`. The flag is a startup-only `flag.Bool` (no live-toggle / config-reload path; Claude bot round-12 minor); changing it requires a process restart. There are two restart strategies: (a) **simultaneous restart of all nodes** — KeyViz heatmap is briefly unavailable during the restart window but no mixed-format fragmentation occurs; (b) **rolling restart** — for the duration of the restart (typically minutes), the heatmap shows a transient mixed view because the legacy `route:N` rows from not-yet-restarted nodes don't merge with `route:N:label` rows from restarted nodes. Since KeyViz is a monitoring view (not a consistency-sensitive system), the rolling-restart fragmentation is acceptable and clears as the final node restarts; operators who want zero fragmentation should use the simultaneous restart. (Claude bot round-12 moderate.) <br>**Memory-vs-flag separation**: `--keyvizLabelsEnabled` controls whether labeled traffic flows into the pre-created labeled slots — it does **not** defer slot allocation. Labeled slots are pre-created by `RegisterRoute` whenever `AllLabels` is non-empty (i.e., from PR-C+D+E deploy time onward, regardless of the flag), so memory grows to `MaxTrackedRoutes × (len(AllLabels)+1)` at binary deploy time. The flag is a traffic-routing toggle, not a slot-pre-creation toggle. (Claude bot round-12 minor.) (Codex round-11 P2 originated this rollout gate.) |
+| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. Populate `AllLabels` in `keyviz/labels.go` (created in PR-B with an empty slice) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) **only when `--keyvizLabelsEnabled=true`**; with the flag off, `RegisterRoute` continues to pre-create just the legacy empty-label slot, identical to today's behavior (Codex round-15-4th-pass P1). **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `slotsPerRoute` (`len(AllLabels)+1` when the flag is on, `1` when off), so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots when labels are enabled, `MaxTrackedRoutes × 1` when off (today's baseline); documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. PR-C+D+E adds `--keyvizLabelsEnabled` (default `false`) on every node; when false, the **`ShardedCoordinator` overrides the adapter-supplied label to `keyviz.LabelLegacy` at the single `sampler.Observe(...)` call site** in `kv/sharded_coordinator.go` (one `if !s.keyvizLabelsEnabled { label = keyviz.LabelLegacy }` guard, not a 5-file flag-read duplication across adapters; Claude bot round-12 moderate). The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. <br>**Flag-flip activation**: once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true`. The flag is a startup-only `flag.Bool` (no live-toggle / config-reload path; Claude bot round-12 minor); changing it requires a process restart. There are two restart strategies: (a) **simultaneous restart of all nodes** — KeyViz heatmap is briefly unavailable during the restart window but no mixed-format fragmentation occurs; (b) **rolling restart** — for the duration of the restart (typically minutes), the heatmap shows a transient mixed view because the legacy `route:N` rows from not-yet-restarted nodes don't merge with `route:N:label` rows from restarted nodes. Since KeyViz is a monitoring view (not a consistency-sensitive system), the rolling-restart fragmentation is acceptable and clears as the final node restarts; operators who want zero fragmentation should use the simultaneous restart. (Claude bot round-12 moderate.) <br>**Flag also gates pre-allocation**: `--keyvizLabelsEnabled` is **both** the traffic-routing toggle (override label to `LabelLegacy` at the coordinator) **and** the slot-pre-creation toggle. When `false`, `RegisterRoute` pre-creates only the legacy empty-label slot (`len(next.slots) += 1` per route, identical to today's behavior); labeled siblings are not allocated. When `true`, `RegisterRoute` pre-creates the legacy slot **and** one labeled sibling per `AllLabels` member (`len(next.slots) += len(AllLabels)+1` per route). Memory therefore stays at the today's level (`MaxTrackedRoutes × 1`) for clusters that deploy the bundled binary but leave the flag off, and grows to `MaxTrackedRoutes × (len(AllLabels)+1)` only when the operator opts in. The flag-flip activation requires a process restart (see preceding paragraph), and `RegisterRoute` re-runs at startup, so the new pre-creation regime is in effect immediately after the restart — there is no live re-allocation path needed. **An earlier draft separated memory and traffic-routing into two regimes ("memory-vs-flag separation", round-12) where labeled slots were pre-created regardless of the flag; that was wrong (Codex round-15-4th-pass P1) — it imposed a multi-x memory penalty on clusters that never enable labels, contradicting the §2.1 minimal-penalty goal.** (Codex round-11 P2 originated this rollout gate.) |
 
 PR-B is independent of the bundled PR-C+D+E: PR-B widens types
 and prepares the slot machinery without changing observable

From 0e4db28abdaa9f747ba599cb40972b5b54778703 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 05:00:59 +0900
Subject: [PATCH 22/31] docs: address PR #694 round-20 (Copilot 6th pass)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Two real defects from Copilot's 6th review pass on `cee54604`:

- §4.1.1 `allLabelsWithLegacy()` off-by-one (round-17 bug):
  the round-17 commit said "allocate a new []Label of length
  len(AllLabels)+1, copy AllLabels into it, then append
  LabelLegacy" — that produces length len(AllLabels)+2, not
  the intended +1. Fixed to use the standard Go idiom:
  `result := make([]Label, len(AllLabels), len(AllLabels)+1);
  copy(result, AllLabels); result = append(result, LabelLegacy)`
  — length len(AllLabels), capacity len(AllLabels)+1, then
  append yields length len(AllLabels)+1 with one allocation.
- §4.2 Recommendation "smallest hot-path delta (one map lookup)"
  contradicts §4.1's "same lookup count, only key shape widens"
  (which round-15 + round-16 corrected). Round-15/16 missed
  this earlier sentence. Fixed: "The hot path remains a single
  map lookup per Observe; only the lookup key widens (so the
  change is wider hash and equality work, not an additional
  lookup)."
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 46 ++++++++++++-------
 1 file changed, 29 insertions(+), 17 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 6245e95a..79c48b7e 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -217,18 +217,25 @@ for _, label := range labelsToCreate {
 `allLabelsWithLegacy()` is an unexported helper in
 **`keyviz/labels.go`** (same package as `sampler.go`) that returns
 a **freshly allocated slice** containing `AllLabels` followed by
-`LabelLegacy`. Concretely: allocate a new `[]Label` of length
-`len(AllLabels)+1`, copy `AllLabels` into it, then append
-`LabelLegacy`. Naively returning `append(AllLabels, LabelLegacy)`
-would be a footgun — Go's `append` may reuse the backing array of
-`AllLabels`, which would then visibly mutate when a future caller
-appends to `AllLabels` directly (the previously written legacy
-entry could be overwritten). Co-locating the helper with
-`AllLabels` keeps the canonical set in one file and avoids
-`sampler.go` reconstructing the same expansion at every call site.
-The function is unexported because no caller outside `package
-keyviz` needs it. (Claude bot round-9 minor; Copilot
-round-15-4th-pass added the fresh-slice safety requirement.)
+`LabelLegacy`. Concretely: `result := make([]Label,
+len(AllLabels), len(AllLabels)+1); copy(result, AllLabels);
+result = append(result, LabelLegacy)` — pre-size the backing
+array via the capacity hint so the final length is exactly
+`len(AllLabels)+1` with one allocation. Naively returning
+`append(AllLabels, LabelLegacy)` would be a footgun — Go's
+`append` may reuse the backing array of `AllLabels`, which would
+then visibly mutate when a future caller appends to `AllLabels`
+directly (the previously written legacy entry could be
+overwritten). Co-locating the helper with `AllLabels` keeps the
+canonical set in one file and avoids `sampler.go` reconstructing
+the same expansion at every call site. The function is
+unexported because no caller outside `package keyviz` needs it.
+(Claude bot round-9 minor; Copilot round-15-4th-pass added the
+fresh-slice safety requirement; Copilot round-19 caught the
+off-by-one in the prior "length len(AllLabels)+1, then append"
+phrasing — that produced length `len(AllLabels)+2`. Now uses
+length `len(AllLabels)`, capacity `len(AllLabels)+1`, then
+append, yielding the intended length `len(AllLabels)+1`.)
 
 When `RegisterRoute` decides a route will be coarsened into a
 virtual bucket (i.e. it returns `false`), **no labeled slots are
@@ -406,11 +413,16 @@ adapter at the catalog layer. The sampler stays single-keyed.
   identity bakes a different separation into the route topology.
   Reverses the multi-group startup-flag story.
 
-**Recommendation: Option 4.1.** Lowest plumbing weight, smallest
-hot-path delta (one map lookup), and the label originates where
-it is most naturally available (the adapter's dispatch entry).
-The `routeSlot` map shape changes from `map[uint64]*routeSlot` to
-`map[slotKey]*routeSlot` with `slotKey = {uint64, string}`.
+**Recommendation: Option 4.1.** Lowest plumbing weight, and the
+label originates where it is most naturally available (the
+adapter's dispatch entry). The hot path remains a **single** map
+lookup per `Observe`; only the lookup key widens (so the change
+is wider hash and equality work, not an additional lookup). The
+`routeSlot` map shape changes from `map[uint64]*routeSlot` to
+`map[slotKey]*routeSlot` with `slotKey = {uint64, Label}`.
+(Copilot round-19 caught the earlier "smallest hot-path delta
+(one map lookup)" wording, which could be misread as adding an
+extra lookup.)
 
 ## 5. Wire format extension
 

From 7b99d65cf1b88d326b97c46f4717af34f5f02a34 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 05:10:17 +0900
Subject: [PATCH 23/31] docs: address PR #694 round-21 (CodeRabbit nitpick)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

One nitpick from CodeRabbit's review on `0e4db28a`:

- §4.1.1 line 371 "atomic with respect to the routesMu lock"
  was wordy. Replaced with "runs under the routesMu lock" —
  same meaning, more concise.

Not addressed (also nitpick, deliberately skipped):
- Lines 599-603 PR-C+D+E row density: CodeRabbit suggested
  splitting the dense single-cell row into nested bullets.
  The cell already uses `<br>` for visual breaks and the
  inline cross-references between items (a)-(e) and the
  L0/L1/L2/L3 propagation chain depend on prose continuity.
  Splitting it now risks another reviewer round flipping
  the format back. CodeRabbit explicitly tagged this as
  optional; leaving the format stable.
---
 docs/design/2026_04_28_proposed_keyviz_adapter_labels.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 79c48b7e..046fe5f4 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -368,7 +368,7 @@ PR-B replaces it with a check on the legacy slot:
 `if _, ok := cur.slots[slotKey{RouteID: routeID}]; ok { return true }`
 — the legacy slot's presence implies the labeled siblings were
 already created by an earlier `RegisterRoute` call (the
-pre-creation loop is atomic with respect to the routesMu lock).
+pre-creation loop runs under the `routesMu` lock).
 
 Two alternatives we explicitly reject:
 

From c7637537a214f053ca69944a9b5d590f19371f89 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 09:27:51 +0900
Subject: [PATCH 24/31] docs: address PR #694 round-22 (Copilot 7th pass)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Two of three items from Copilot's 7th review pass on `7b99d65c`:

- §4.1 line 110 section heading "Per-Observe label string"
  inconsistent with typed `Label`: renamed to "Per-Observe
  label", added explicit `type Label string` reference to
  §9 Q2, replaced "constant string" prose with "canonical
  constant" terminology, and surfaced the compile-time typo
  protection rationale that was previously buried in §9.
- §8 line 632 hardcoded "current 4 ns/op" benchmark target:
  Go ns/op figures are machine/Go-version/thermal dependent,
  so a fixed number can't anchor a regression check.
  Reframed as relative regression check: "no statistically
  significant regression versus a same-run baseline (or a
  recorded benchmark run attached to the PR)".

Not addressed (deliberately deferred):

- Inline review attribution noise (e.g. "Copilot round-15…",
  "Codex round-15-4th-pass P1") embedded throughout the
  narrative. CodeRabbit-style cleanup would relocate these
  to a dedicated "Revision notes" appendix. The design has
  stabilized over rounds 1-22; another large structural
  edit risks triggering yet more review cycles. The
  attributions do clutter the prose, but they also provide
  load-bearing context for non-obvious design choices
  ("why is the divisor len(AllLabels)+1?" → answered by
  "Codex round-6 + round-10 P2 wanted route-counting
  preserved"). Defer to a separate cleanup PR after merge,
  to avoid further reviewer-loop churn on this PR.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 26 +++++++++++--------
 1 file changed, 15 insertions(+), 11 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 046fe5f4..ab58a933 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -97,17 +97,20 @@ After (adapters tag their traffic):
 Four ways to get the label from the adapter to the sampler. Pick
 one based on hot-path cost vs. plumbing weight.
 
-### 4.1 (Recommended) Per-Observe label string
+### 4.1 (Recommended) Per-Observe label
 
-Extend the sampler signature:
+Extend the sampler signature with a typed `Label` (defined as
+`type Label string` in `keyviz/labels.go`, see §9 Q2):
 
 ```go
 Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)
 ```
 
-Empty `label` → existing behaviour, single row per route. Adapters
-set their own constant string (`"dynamo"`, `"redis"`, …) at the
-dispatch site they already own.
+Empty `label` (`LabelLegacy`) → existing behaviour, single row
+per route. Adapters set their own canonical constant
+(`LabelDynamo`, `LabelRedis`, …) at the dispatch site they
+already own — typed constants prevent typos like `"DynamoDB"` /
+`"dynamo"` drift at compile time.
 
 Cost on hot path: still one `slots` map lookup per `Observe` on
 the hit path (the existing code already does a single
@@ -626,12 +629,13 @@ Codex round-10.)
    `Observe` (same as today); the only extra work is wider hash
    and equality on the `{uint64, Label}` key vs the current
    `uint64` key. `BenchmarkObserveParallel` already pins the
-   hot-path cost; the new bench should land within run-to-run
-   variance of the current 4 ns/op. If it doesn't, the design is
-   wrong and we fall back to Option 4.2. (Copilot round-15-3rd-
-   pass caught the earlier "one extra map lookup" wording, which
-   contradicted §4.1's correctly-described "same lookup count,
-   wider key shape".)
+   hot-path cost. The acceptance check is **no statistically
+   significant regression** versus a same-run baseline (or a
+   recorded benchmark run attached to the PR), not an absolute
+   ns/op target — `BenchmarkObserveParallel` results vary by
+   machine, Go version, and CPU thermal state. If the benchmark
+   regresses materially, the design is wrong and we fall back to
+   Option 4.2.
 4. **Data consistency** — Cluster fan-out merge gains a tuple
    field; the dedup invariant (per-cell, per-(route, label,
    group, term, window)) still holds. Old SPA against new server

From e5c7e39d5d2732f9a1750a3bb6750c2b22efa187 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 09:37:02 +0900
Subject: [PATCH 25/31] docs: address PR #694 round-23 (CodeRabbit minor)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Two minor items from CodeRabbit's review on `c7637537`:

- §4.1.1 line 273 sampler.go snippet `len(keyviz.AllLabels)`
  was a real bug — the snippet is in `keyviz/sampler.go` and
  references the package-local `AllLabels`, so the
  `keyviz.` qualifier is non-idiomatic and would not compile
  if copy-pasted into the package. Fixed to `len(AllLabels)`
  with an inline note clarifying the in-package context.
- §8 line 535 `#694.)` at line start trips markdownlint
  MD018 (no-missing-space-atx — looks like an H1 with no
  space). Wrapped the PR reference in backticks (`#694`)
  and joined the two lines so the hash is no longer at
  line start.
---
 docs/design/2026_04_28_proposed_keyviz_adapter_labels.md | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index ab58a933..d53a48b5 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -270,7 +270,7 @@ we document but never count against the cap.
 // keyviz/sampler.go:416 (PR-B + PR-C+D+E flag-gated, Codex round-15-4th-pass P1)
 slotsPerRoute := 1 // legacy slot only (PR-B, or PR-C+D+E with flag off)
 if s.opts.KeyVizLabelsEnabled {
-    slotsPerRoute = len(keyviz.AllLabels) + 1 // 6 after PR-C with flag on
+    slotsPerRoute = len(AllLabels) + 1 // 6 after PR-C with flag on (in-package, no keyviz. qualifier)
 }
 if len(next.slots) / slotsPerRoute < s.opts.MaxTrackedRoutes {
     // ... pre-create slot(s) for this route
@@ -531,8 +531,7 @@ file with a `func TestAllLabelsAvoidSeparator(t *testing.T)` test
 that asserts `!strings.ContainsRune(string(l), ':')` for every member of
 `AllLabels`. Without this constraint a future label like
 `"redis:db0"` would silently break `bucket_id` parsing — the
-parser would split at the wrong `:`. (CodeRabbit major on PR
-#694.)
+parser would split at the wrong `:`. (CodeRabbit major on PR `#694`.)
 
 `route_ids` / `aggregate` / virtual-bucket semantics from §5 of
 the parent design are unchanged — labels are an orthogonal axis

From 795c504f56893bf7393d0938e7311d1574aa3ff2 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 10:12:43 +0900
Subject: [PATCH 26/31] docs: address PR #694 round-23 (Copilot 8th pass)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Three real items from Copilot's 8th review pass on `e5c7e39d`:

- §4.1 line 137 "no memory penalty" overstatement: even with
  --keyvizLabelsEnabled=false, slot map key widens from
  uint64 to slotKey{uint64, Label} so per-slot key storage
  grows. Reframed: the flag avoids the *multiplicative*
  len(AllLabels)+1 slot count increase (the dominant cost),
  but each slot's wider key adds a small per-slot overhead
  vs today even with the flag off.
- §5 line 456 "unused and uncommented" reads as
  "commented-out" code. Changed to "unused and undocumented
  (no doc comment on the field)" for clarity.
- §9 Q2 line 742 AllLabels was `var []Label{...}` (exported
  mutable slice) — importing packages could
  append/reorder/zero it and silently break slot
  pre-creation. Switched to fixed-size array
  `[5]Label{...}` so any mutation attempt becomes a compile
  error. Required propagating the change:
  - allLabelsWithLegacy() now uses `copy(result,
    AllLabels[:])` (slice-of-array) since copy needs slice
    args.
  - §7 PR-B row updated to declare AllLabels as the
    populated array from PR-B, with behavior neutrality
    coming from the round-17 flag-gating
    (--keyvizLabelsEnabled=false default) instead of an
    empty-slice trick.
  - §8 TestAllLabelsAvoidSeparator description corrected
    from "AllLabels []Label is the typed slice" to
    "AllLabels is the typed [5]Label array".

The PR-B "deliberately-empty AllLabels" framing from
earlier rounds is now redundant (round-17's flag gate
already makes PR-B behavior-neutral). Documented the
sequencing change in §9 Q2 so the round-history is
internally consistent.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 98 ++++++++++++++-----
 1 file changed, 71 insertions(+), 27 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index d53a48b5..f14e6c47 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -131,14 +131,20 @@ Storage: each (route, label) gets its own `routeSlot` **when
 labels are enabled** (`--keyvizLabelsEnabled=true`). Total slot
 memory is `MaxTrackedRoutes × (len(AllLabels)+1)` with labels
 enabled, and `MaxTrackedRoutes × 1` with labels disabled (legacy
-slot only — same as today's baseline). The pre-allocation cost is
-gated on the flag, so a cluster that deploys the PR-C+D+E binary
-but leaves the flag off pays no memory penalty (Codex round-15-
-4th-pass P1). `MaxTrackedRoutes` retains its route-counting
-semantics under Option A (§4.1.1), so no operator action is
-required at PR-C+D+E rollout. (Vestigial "when operators raise
-it for label use" framing was rejected — Codex round-10 P2 +
-Claude bot round-11.)
+slot only — same number of slots as today's baseline). The
+pre-allocation cost is gated on the flag, so a cluster that
+deploys the PR-C+D+E binary but leaves the flag off avoids the
+**multiplicative** `len(AllLabels)+1` slot-count increase, though
+each slot's map key is `slotKey{uint64, Label}` (a struct with a
+string header) instead of a bare `uint64`, so there is a small
+per-slot overhead vs today even with the flag off. The dominant
+memory cost — the multiplicative slot count — is the part the
+flag actually gates (Codex round-15-4th-pass P1; Copilot round-23
+caught the "no memory penalty" overstatement).
+`MaxTrackedRoutes` retains its route-counting semantics under
+Option A (§4.1.1), so no operator action is required at PR-C+D+E
+rollout. (Vestigial "when operators raise it for label use"
+framing was rejected — Codex round-10 P2 + Claude bot round-11.)
 
 #### 4.1.1 Slot lifecycle and the lockless invariant
 
@@ -221,15 +227,18 @@ for _, label := range labelsToCreate {
 **`keyviz/labels.go`** (same package as `sampler.go`) that returns
 a **freshly allocated slice** containing `AllLabels` followed by
 `LabelLegacy`. Concretely: `result := make([]Label,
-len(AllLabels), len(AllLabels)+1); copy(result, AllLabels);
+len(AllLabels), len(AllLabels)+1); copy(result, AllLabels[:]);
 result = append(result, LabelLegacy)` — pre-size the backing
 array via the capacity hint so the final length is exactly
-`len(AllLabels)+1` with one allocation. Naively returning
-`append(AllLabels, LabelLegacy)` would be a footgun — Go's
-`append` may reuse the backing array of `AllLabels`, which would
-then visibly mutate when a future caller appends to `AllLabels`
-directly (the previously written legacy entry could be
-overwritten). Co-locating the helper with `AllLabels` keeps the
+`len(AllLabels)+1` with one allocation. The explicit `AllLabels[:]`
+re-slice is required because §9 Q2 declares `AllLabels` as a
+fixed-size array (`[5]Label`) and Go's `copy` builtin needs slice
+arguments. Naively returning `append(AllLabels[:], LabelLegacy)`
+would be a footgun — Go's `append` against a slice-of-array may
+share storage with the underlying array, which would then visibly
+mutate the canonical `AllLabels` (silently overwriting the legacy
+entry into the array's tail or, more commonly, panicking because
+the array is full). Co-locating the helper with `AllLabels` keeps the
 canonical set in one file and avoids `sampler.go` reconstructing
 the same expansion at every call site. The function is
 unexported because no caller outside `package keyviz` needs it.
@@ -453,7 +462,8 @@ upstream still on PR-C never sees a wire shape mismatch:
  //
  // Reuse the **existing** field 4 (`string label`), which is
  // already declared on KeyVizRow but currently unused and
- // uncommented — no schema migration, no field-number bump.
+ // undocumented (no doc comment on the field) — no schema
+ // migration, no field-number bump.
  // The current proto declares `string label = 4;` with no
  // doc comment, and the nearby `bucket_id` comment only mentions
  // the legacy `route:<id>` / `virtual:<id>` forms. PR-D+E starts
@@ -601,7 +611,7 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **`RegisterRoute` only pre-creates the legacy empty-label slot** — labeled-sibling pre-creation lands in PR-C so PR-B is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants** (declared but unused outside the sampler — adapters wire them in PR-C), and a **deliberately-empty `AllLabels []Label`**. The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with `AllLabels` empty their bodies are no-ops over (Claude bot round-6). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
+| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **`RegisterRoute` only pre-creates the legacy empty-label slot when `--keyvizLabelsEnabled=false` (the default)** — labeled-sibling pre-creation is gated on the flag (round-17), so PR-B with the default flag is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants**, and `AllLabels` as a **fixed-size array** `[5]Label{LabelDynamo, ..., LabelRawKV}` (round-23 — array form prevents importing-package mutation; see §9 Q2). The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with the flag default to false, `RegisterRoute` skips the labeled-sibling loop body so per-route slot count stays at 1 (Claude bot round-6 + round-23 array correction). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
 | **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. Populate `AllLabels` in `keyviz/labels.go` (created in PR-B with an empty slice) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) **only when `--keyvizLabelsEnabled=true`**; with the flag off, `RegisterRoute` continues to pre-create just the legacy empty-label slot, identical to today's behavior (Codex round-15-4th-pass P1). **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `slotsPerRoute` (`len(AllLabels)+1` when the flag is on, `1` when off), so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots when labels are enabled, `MaxTrackedRoutes × 1` when off (today's baseline); documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. PR-C+D+E adds `--keyvizLabelsEnabled` (default `false`) on every node; when false, the **`ShardedCoordinator` overrides the adapter-supplied label to `keyviz.LabelLegacy` at the single `sampler.Observe(...)` call site** in `kv/sharded_coordinator.go` (one `if !s.keyvizLabelsEnabled { label = keyviz.LabelLegacy }` guard, not a 5-file flag-read duplication across adapters; Claude bot round-12 moderate). The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. <br>**Flag-flip activation**: once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true`. The flag is a startup-only `flag.Bool` (no live-toggle / config-reload path; Claude bot round-12 minor); changing it requires a process restart. There are two restart strategies: (a) **simultaneous restart of all nodes** — KeyViz heatmap is briefly unavailable during the restart window but no mixed-format fragmentation occurs; (b) **rolling restart** — for the duration of the restart (typically minutes), the heatmap shows a transient mixed view because the legacy `route:N` rows from not-yet-restarted nodes don't merge with `route:N:label` rows from restarted nodes. Since KeyViz is a monitoring view (not a consistency-sensitive system), the rolling-restart fragmentation is acceptable and clears as the final node restarts; operators who want zero fragmentation should use the simultaneous restart. (Claude bot round-12 moderate.) <br>**Flag also gates pre-allocation**: `--keyvizLabelsEnabled` is **both** the traffic-routing toggle (override label to `LabelLegacy` at the coordinator) **and** the slot-pre-creation toggle. When `false`, `RegisterRoute` pre-creates only the legacy empty-label slot (`len(next.slots) += 1` per route, identical to today's behavior); labeled siblings are not allocated. When `true`, `RegisterRoute` pre-creates the legacy slot **and** one labeled sibling per `AllLabels` member (`len(next.slots) += len(AllLabels)+1` per route). Memory therefore stays at the today's level (`MaxTrackedRoutes × 1`) for clusters that deploy the bundled binary but leave the flag off, and grows to `MaxTrackedRoutes × (len(AllLabels)+1)` only when the operator opts in. The flag-flip activation requires a process restart (see preceding paragraph), and `RegisterRoute` re-runs at startup, so the new pre-creation regime is in effect immediately after the restart — there is no live re-allocation path needed. **An earlier draft separated memory and traffic-routing into two regimes ("memory-vs-flag separation", round-12) where labeled slots were pre-created regardless of the flag; that was wrong (Codex round-15-4th-pass P1) — it imposed a multi-x memory penalty on clusters that never enable labels, contradicting the §2.1 minimal-penalty goal.** (Codex round-11 P2 originated this rollout gate.) |
 
 PR-B is independent of the bundled PR-C+D+E: PR-B widens types
@@ -670,14 +680,14 @@ Codex round-10.)
      written in PR-B): range `AllLabels`, assert
      `!strings.ContainsRune(string(label), ':')` for every
      constant — explicit `string(label)` cast required because
-     `AllLabels []Label` is the typed slice and
-     `strings.ContainsRune` takes `string`.
-     The test is a no-op in PR-B (empty `AllLabels` ranges over
-     zero entries) but lands in PR-B so PR-C cannot populate
-     `AllLabels` with a `:`-containing label without the test
-     turning red. Pins the §5 hard constraint at compile time; a
-     future "redis:db0" would otherwise silently break
-     `bucket_id` parsing.
+     `AllLabels` is the typed `[5]Label` array and
+     `strings.ContainsRune` takes `string`. The test runs in
+     PR-B against the populated array (round-23 array form, see
+     §9 Q2) and prevents anyone from later changing a constant
+     to a `:`-containing literal without the test turning red.
+     Pins the §5 hard constraint at compile time; a future
+     "redis:db0" would otherwise silently break `bucket_id`
+     parsing.
    - **All declared label constants are in `AllLabels`**
      (PR-C regression guard, written in PR-B): range over the
      named constants (`LabelDynamo`, `LabelRedis`, …) and assert
@@ -738,14 +748,48 @@ Codex round-10.)
        LabelSQS    Label = "sqs"
        LabelRawKV  Label = "rawkv"
    )
-   var AllLabels = []Label{LabelDynamo, LabelRedis, LabelS3, LabelSQS, LabelRawKV}
+   // Fixed-size array, not a slice — append/reorder/mutate by
+   // an importing package becomes a compile error, eliminating
+   // the silent slot-drop class of bugs that mutating the
+   // canonical set would cause. (Copilot round-23 caught the
+   // exported-mutable-slice footgun; round-23 chose the array
+   // approach over alternatives like an unexported slice with
+   // an accessor function returning a fresh copy, because the
+   // array gives compile-time enforcement at zero per-iteration
+   // allocation cost.)
+   var AllLabels = [5]Label{LabelDynamo, LabelRedis, LabelS3, LabelSQS, LabelRawKV}
    ```
 
    `Observe` takes `label Label` (not `string`) and `RegisterRoute`
    iterates `AllLabels` for pre-allocation. Constants live in the
    `keyviz` package (not `adapter/`) so the sampler can read the
    full set at registration time without an import cycle back to
-   `adapter`.
+   `adapter`. `AllLabels` is intentionally a **fixed-size array**
+   (`[5]Label`) rather than a slice — `append`/reorder/mutation by
+   an importing package would be a compile error rather than a
+   silent runtime corruption that drops samples. Iteration uses
+   `range AllLabels` (works identically for arrays and slices) and
+   `len(AllLabels)` returns 5 at compile time. The
+   `allLabelsWithLegacy()` helper converts to a freshly allocated
+   slice via `make([]Label, len(AllLabels), len(AllLabels)+1);
+   copy(result, AllLabels[:]); result = append(result, LabelLegacy)`
+   — the explicit `AllLabels[:]` re-slice is required because Go's
+   `copy` builtin needs slice arguments. Bumping `AllLabels` length
+   from 5 (when adding a new adapter) requires a deliberate doc +
+   array-size edit, which is the intended friction; the
+   `TestAllLabelConstantsInAllLabels` regression guard from §8
+   then verifies the new constant is present in `AllLabels`.
+
+   **PR sequencing note**: §7 PR-B previously declared `AllLabels`
+   as a *deliberately-empty slice* (`var AllLabels []Label{}`) to
+   make the pre-creation loop a no-op until PR-C populated it.
+   With the round-23 array form, AllLabels is non-empty from PR-B
+   onward — but the round-17 flag-gating means PR-B's
+   `RegisterRoute` only pre-creates labeled siblings when
+   `--keyvizLabelsEnabled=true`, which defaults to false. So
+   PR-B remains behavior-neutral via the flag, not via an empty
+   slice. The §7 PR-B row already mentions the flag default; this
+   is the same mechanism reused.
 
    **Why a typed `Label` rather than plain `string`** (Codex round-9):
    slot pre-creation iterates `AllLabels`, and `Observe` drops slot

From 09c1c04ed527e5d4ea02416dffe5410e82bb5520 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 10:23:09 +0900
Subject: [PATCH 27/31] =?UTF-8?q?docs:=20address=20PR=20#694=20round-24=20?=
 =?UTF-8?q?(Claude=20bot=20=E2=80=94=20final=20fixes)?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Round-24 finished review verdict on `795c504f`:
"design is ready to merge as PR-A after these 4 fixes".
All 4 items addressed:

- §7 line 619 closing paragraph "AllLabels empty" was a
  stale reference: round-23 array form makes AllLabels
  populated from PR-B; the divisor=1 reason is the
  --keyvizLabelsEnabled=false default, not an empty array.
  Updated to "AllLabels populated [5]Label array... flag
  default false keeps divisor at 1".
- §7 line 621 "AllLabels populated" was the right framing
  before round-23 (PR-C populated an empty slice). With
  the array, PR-C+D+E only wires adapters; constants are
  already declared. Updated to "adapter wiring (constants
  already declared in keyviz/labels.go from PR-B...)".
- §7 PR-C+D+E adapter-wiring description "Populate
  AllLabels...(created in PR-B with an empty slice)" was
  the inverse of round-23. Updated to "constants already
  declared from PR-B as part of the [5]Label array; PR-C+D+E
  only wires adapters to pass them, no AllLabels
  modification needed".
- §7 PR-B row "with AllLabels empty in PR-B the divisor is
  1" updated to "with the flag default to false in PR-B
  the divisor is 1" (round-23 made AllLabels populated;
  the flag is what keeps the divisor at 1).
- §7 PR-B row "Missing L0... after PR-C populates AllLabels"
  updated to "after PR-C+D+E wires the adapters to pass
  labels" (the populate-step doesn't exist anymore;
  adapter-wiring is the round-23-correct trigger).
- §7 PR-B row "With PR-B's empty-only slots" updated to
  "With PR-B's flag-default-false (only legacy slots
  pre-created)".
- §8 lens-5 TestAllLabelConstantsInAllLabels "Skip when
  len(AllLabels) == 0" was dead code — populated array
  has length 5 at compile time. Removed the skip; updated
  description to "PR-B regression guard — array is
  populated from PR-B so this is a live test from PR-B
  onward, not a no-op".
- §8 lens-5 TestAllLabelsAvoidSeparator "PR-C regression
  guard, written in PR-B" was contradictory after the
  array change. Updated to "PR-B regression guard — the
  array is populated from PR-B so the test is live
  immediately".
- §9 Q2 mutation claim was overstated: "append/reorder/
  mutation by an importing package would be a compile
  error" — only `append` is type-blocked. Element
  reassignment (AllLabels[0] = ...) and full-var
  reassignment compile against the exported var. Scoped
  the claim to `append` specifically and noted that
  TestAllLabelConstantsInAllLabels is the runtime guard
  for the rest.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 108 +++++++++++-------
 1 file changed, 69 insertions(+), 39 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index f14e6c47..d226f30c 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -611,17 +611,22 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **`RegisterRoute` only pre-creates the legacy empty-label slot when `--keyvizLabelsEnabled=false` (the default)** — labeled-sibling pre-creation is gated on the flag (round-17), so PR-B with the default flag is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants**, and `AllLabels` as a **fixed-size array** `[5]Label{LabelDynamo, ..., LabelRawKV}` (round-23 — array form prevents importing-package mutation; see §9 Q2). The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with the flag default to false, `RegisterRoute` skips the labeled-sibling loop body so per-route slot count stays at 1 (Claude bot round-6 + round-23 array correction). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C populates `AllLabels`; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's empty-only slots, `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `len(AllLabels)+1` so the cap continues to count **routes**, not slots; with `AllLabels` empty in PR-B the divisor is 1 and behavior is identical to today. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
-| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. Populate `AllLabels` in `keyviz/labels.go` (created in PR-B with an empty slice) with the five adapter constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) **only when `--keyvizLabelsEnabled=true`**; with the flag off, `RegisterRoute` continues to pre-create just the legacy empty-label slot, identical to today's behavior (Codex round-15-4th-pass P1). **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `slotsPerRoute` (`len(AllLabels)+1` when the flag is on, `1` when off), so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots when labels are enabled, `MaxTrackedRoutes × 1` when off (today's baseline); documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. PR-C+D+E adds `--keyvizLabelsEnabled` (default `false`) on every node; when false, the **`ShardedCoordinator` overrides the adapter-supplied label to `keyviz.LabelLegacy` at the single `sampler.Observe(...)` call site** in `kv/sharded_coordinator.go` (one `if !s.keyvizLabelsEnabled { label = keyviz.LabelLegacy }` guard, not a 5-file flag-read duplication across adapters; Claude bot round-12 moderate). The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. <br>**Flag-flip activation**: once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true`. The flag is a startup-only `flag.Bool` (no live-toggle / config-reload path; Claude bot round-12 minor); changing it requires a process restart. There are two restart strategies: (a) **simultaneous restart of all nodes** — KeyViz heatmap is briefly unavailable during the restart window but no mixed-format fragmentation occurs; (b) **rolling restart** — for the duration of the restart (typically minutes), the heatmap shows a transient mixed view because the legacy `route:N` rows from not-yet-restarted nodes don't merge with `route:N:label` rows from restarted nodes. Since KeyViz is a monitoring view (not a consistency-sensitive system), the rolling-restart fragmentation is acceptable and clears as the final node restarts; operators who want zero fragmentation should use the simultaneous restart. (Claude bot round-12 moderate.) <br>**Flag also gates pre-allocation**: `--keyvizLabelsEnabled` is **both** the traffic-routing toggle (override label to `LabelLegacy` at the coordinator) **and** the slot-pre-creation toggle. When `false`, `RegisterRoute` pre-creates only the legacy empty-label slot (`len(next.slots) += 1` per route, identical to today's behavior); labeled siblings are not allocated. When `true`, `RegisterRoute` pre-creates the legacy slot **and** one labeled sibling per `AllLabels` member (`len(next.slots) += len(AllLabels)+1` per route). Memory therefore stays at the today's level (`MaxTrackedRoutes × 1`) for clusters that deploy the bundled binary but leave the flag off, and grows to `MaxTrackedRoutes × (len(AllLabels)+1)` only when the operator opts in. The flag-flip activation requires a process restart (see preceding paragraph), and `RegisterRoute` re-runs at startup, so the new pre-creation regime is in effect immediately after the restart — there is no live re-allocation path needed. **An earlier draft separated memory and traffic-routing into two regimes ("memory-vs-flag separation", round-12) where labeled slots were pre-created regardless of the flag; that was wrong (Codex round-15-4th-pass P1) — it imposed a multi-x memory penalty on clusters that never enable labels, contradicting the §2.1 minimal-penalty goal.** (Codex round-11 P2 originated this rollout gate.) |
+| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **`RegisterRoute` only pre-creates the legacy empty-label slot when `--keyvizLabelsEnabled=false` (the default)** — labeled-sibling pre-creation is gated on the flag (round-17), so PR-B with the default flag is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants**, and `AllLabels` as a **fixed-size array** `[5]Label{LabelDynamo, ..., LabelRawKV}` (round-23 — array form prevents importing-package mutation; see §9 Q2). The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with the flag default to false, `RegisterRoute` skips the labeled-sibling loop body so per-route slot count stays at 1 (Claude bot round-6 + round-23 array correction). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C+D+E wires the adapters to pass labels; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's flag-default-false (only legacy slots pre-created), `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C+D+E to flip the flag and start emitting labels. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `slotsPerRoute` (= `len(AllLabels)+1` when the flag is on, `1` when off; round-17 + round-23) so the cap continues to count **routes**, not slots; with the flag default to false in PR-B the divisor is 1 and behavior is identical to today (Claude round-24 fix to the prior "AllLabels empty in PR-B" framing — round-23 made AllLabels populated from PR-B, so the divisor=1 reason is the flag, not the array length). Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
+| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. The five canonical constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`) are already declared in `keyviz/labels.go` from PR-B as part of the `[5]Label` array (round-23 — see §9 Q2 and the PR-B row above); PR-C+D+E only wires the adapters to pass them, no `AllLabels` modification needed (Claude round-24 fix to the prior "populate empty slice" framing). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) **only when `--keyvizLabelsEnabled=true`**; with the flag off, `RegisterRoute` continues to pre-create just the legacy empty-label slot, identical to today's behavior (Codex round-15-4th-pass P1). **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `slotsPerRoute` (`len(AllLabels)+1` when the flag is on, `1` when off), so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots when labels are enabled, `MaxTrackedRoutes × 1` when off (today's baseline); documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. PR-C+D+E adds `--keyvizLabelsEnabled` (default `false`) on every node; when false, the **`ShardedCoordinator` overrides the adapter-supplied label to `keyviz.LabelLegacy` at the single `sampler.Observe(...)` call site** in `kv/sharded_coordinator.go` (one `if !s.keyvizLabelsEnabled { label = keyviz.LabelLegacy }` guard, not a 5-file flag-read duplication across adapters; Claude bot round-12 moderate). The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. <br>**Flag-flip activation**: once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true`. The flag is a startup-only `flag.Bool` (no live-toggle / config-reload path; Claude bot round-12 minor); changing it requires a process restart. There are two restart strategies: (a) **simultaneous restart of all nodes** — KeyViz heatmap is briefly unavailable during the restart window but no mixed-format fragmentation occurs; (b) **rolling restart** — for the duration of the restart (typically minutes), the heatmap shows a transient mixed view because the legacy `route:N` rows from not-yet-restarted nodes don't merge with `route:N:label` rows from restarted nodes. Since KeyViz is a monitoring view (not a consistency-sensitive system), the rolling-restart fragmentation is acceptable and clears as the final node restarts; operators who want zero fragmentation should use the simultaneous restart. (Claude bot round-12 moderate.) <br>**Flag also gates pre-allocation**: `--keyvizLabelsEnabled` is **both** the traffic-routing toggle (override label to `LabelLegacy` at the coordinator) **and** the slot-pre-creation toggle. When `false`, `RegisterRoute` pre-creates only the legacy empty-label slot (`len(next.slots) += 1` per route, identical to today's behavior); labeled siblings are not allocated. When `true`, `RegisterRoute` pre-creates the legacy slot **and** one labeled sibling per `AllLabels` member (`len(next.slots) += len(AllLabels)+1` per route). Memory therefore stays at the today's level (`MaxTrackedRoutes × 1`) for clusters that deploy the bundled binary but leave the flag off, and grows to `MaxTrackedRoutes × (len(AllLabels)+1)` only when the operator opts in. The flag-flip activation requires a process restart (see preceding paragraph), and `RegisterRoute` re-runs at startup, so the new pre-creation regime is in effect immediately after the restart — there is no live re-allocation path needed. **An earlier draft separated memory and traffic-routing into two regimes ("memory-vs-flag separation", round-12) where labeled slots were pre-created regardless of the flag; that was wrong (Codex round-15-4th-pass P1) — it imposed a multi-x memory penalty on clusters that never enable labels, contradicting the §2.1 minimal-penalty goal.** (Codex round-11 P2 originated this rollout gate.) |
 
 PR-B is independent of the bundled PR-C+D+E: PR-B widens types
 and prepares the slot machinery without changing observable
-behavior (legacy slots only, AllLabels empty, MaxTrackedRoutes
-divisor is 1). The bundled PR-C+D+E then ships everything
-operator-visible — adapter wiring, `AllLabels` populated, all
-five wire-format copies — in a single atomic deploy. (Reviewer
-notes from PR #694: Claude bot critical / moderate +
-Codex round-10.)
+behavior (legacy slots only, `AllLabels` is the populated
+`[5]Label` array but `--keyvizLabelsEnabled` defaults to false
+so `RegisterRoute` skips the labeled-sibling loop and the
+`MaxTrackedRoutes` divisor stays at 1; round-23 array form, see
+§9 Q2). The bundled PR-C+D+E then ships everything
+operator-visible — adapter wiring (constants are already
+declared in `keyviz/labels.go` from PR-B; PR-C+D+E only wires
+adapters to pass them at the dispatch entry, no change to
+`AllLabels` itself), all five wire-format copies — in a single
+atomic deploy. (Reviewer notes from PR #694: Claude bot critical
+/ moderate + Codex round-10 + Claude round-24 fix.)
 
 ## 8. Five-lens checklist
 
@@ -676,30 +681,36 @@ Codex round-10.)
      respectively — not two rows with `Label=""`. Catches the
      mergeRowInto regression where Label is omitted from the
      `dst := &KeyVizRow{...}` construction.
-   - **Canonical labels avoid `:`** (PR-C regression guard,
-     written in PR-B): range `AllLabels`, assert
+   - **Canonical labels avoid `:`** (PR-B regression guard —
+     the `[5]Label` array is populated from PR-B (round-23, see
+     §9 Q2) so the test is live immediately, not a no-op until
+     PR-C): range `AllLabels`, assert
      `!strings.ContainsRune(string(label), ':')` for every
      constant — explicit `string(label)` cast required because
      `AllLabels` is the typed `[5]Label` array and
-     `strings.ContainsRune` takes `string`. The test runs in
-     PR-B against the populated array (round-23 array form, see
-     §9 Q2) and prevents anyone from later changing a constant
-     to a `:`-containing literal without the test turning red.
+     `strings.ContainsRune` takes `string`. Prevents anyone
+     from later changing a constant to a `:`-containing literal
+     without the test turning red. (Claude round-24 corrected
+     the stale "PR-C regression guard, written in PR-B" label
+     — with the populated array, the test is a live PR-B test.)
      Pins the §5 hard constraint at compile time; a future
      "redis:db0" would otherwise silently break `bucket_id`
      parsing.
    - **All declared label constants are in `AllLabels`**
-     (PR-C regression guard, written in PR-B): range over the
-     named constants (`LabelDynamo`, `LabelRedis`, …) and assert
-     each appears in `AllLabels`. Catches the symmetric mistake
-     where PR-C adds `LabelNewAdapter = "newadapter"` to
-     `keyviz/labels.go` but forgets to append it to `AllLabels`
-     — the slot pre-creation loop wouldn't iterate over it,
-     `Observe(... LabelNewAdapter)` would miss `tbl.slots`, and
-     all KeyViz samples for that adapter would silently drop
-     in production. Skip when `len(AllLabels) == 0` (matching
-     the avoid-`:` test's pre-PR-C no-op pattern). (Claude bot
-     round-9.)
+     (PR-B regression guard — the `[5]Label` array is populated
+     from PR-B (round-23, see §9 Q2), so this is a live test
+     from PR-B onward, not a no-op): range over the named
+     constants (`LabelDynamo`, `LabelRedis`, …) and assert each
+     appears in `AllLabels`. Catches the mistake where a future
+     adapter PR adds `LabelNewAdapter = "newadapter"` to
+     `keyviz/labels.go` but forgets to update `AllLabels`'s
+     literal — the slot pre-creation loop wouldn't iterate over
+     it, `Observe(..., LabelNewAdapter)` would miss `tbl.slots`,
+     and all KeyViz samples for that adapter would silently drop
+     in production. (Claude bot round-9; Claude round-24
+     removed the dead `len(AllLabels) == 0` skip — with the
+     populated array, length is always 5 at compile time, so
+     the guard is dead code.)
    - **`reclaimRetiredSlot` (RouteID, Label) dedupe**: retire
      `(routeID, "dynamo")`, re-register `(routeID, "dynamo")`,
      assert reclaim succeeds; then retire `(routeID, "dynamo")`
@@ -748,15 +759,24 @@ Codex round-10.)
        LabelSQS    Label = "sqs"
        LabelRawKV  Label = "rawkv"
    )
-   // Fixed-size array, not a slice — append/reorder/mutate by
-   // an importing package becomes a compile error, eliminating
-   // the silent slot-drop class of bugs that mutating the
-   // canonical set would cause. (Copilot round-23 caught the
-   // exported-mutable-slice footgun; round-23 chose the array
-   // approach over alternatives like an unexported slice with
-   // an accessor function returning a fresh copy, because the
-   // array gives compile-time enforcement at zero per-iteration
-   // allocation cost.)
+   // Fixed-size array prevents the most common mutation pattern
+   // (`append(keyviz.AllLabels, extra)` is a type error since
+   // AllLabels is [5]Label, not []Label). Element reassignment
+   // (`keyviz.AllLabels[0] = ...`) and full-var reassignment
+   // (`keyviz.AllLabels = [5]Label{...}`) still compile against
+   // an exported `var`, so they are not a compile-time guarantee
+   // — but they require deliberate intent and would be caught
+   // in code review and by the TestAllLabelConstantsInAllLabels
+   // regression guard from §8 (which fails if any declared
+   // constant is missing from AllLabels). The combination of
+   // type-system block on `append` + runtime test on contents
+   // is the round-23 enforcement story; an unexported var with
+   // an accessor function returning a fresh copy was considered
+   // and rejected because the array gives the same `append`
+   // protection at zero per-iteration allocation cost.
+   // (Copilot round-23 caught the exported-mutable-slice
+   // footgun; Claude round-24 corrected the overstated
+   // "mutation is a compile error" claim.)
    var AllLabels = [5]Label{LabelDynamo, LabelRedis, LabelS3, LabelSQS, LabelRawKV}
    ```
 
@@ -765,11 +785,21 @@ Codex round-10.)
    `keyviz` package (not `adapter/`) so the sampler can read the
    full set at registration time without an import cycle back to
    `adapter`. `AllLabels` is intentionally a **fixed-size array**
-   (`[5]Label`) rather than a slice — `append`/reorder/mutation by
-   an importing package would be a compile error rather than a
-   silent runtime corruption that drops samples. Iteration uses
-   `range AllLabels` (works identically for arrays and slices) and
-   `len(AllLabels)` returns 5 at compile time. The
+   (`[5]Label`) rather than a slice — the most common mutation
+   pattern (`append(keyviz.AllLabels, extra)`) becomes a
+   compile-time type error since `AllLabels` is `[5]Label`, not
+   `[]Label`. Element reassignment
+   (`keyviz.AllLabels[0] = keyviz.Label("hacked")`) and full-var
+   reassignment (`keyviz.AllLabels = [5]Label{...}`) still
+   compile against an exported `var`, so the type system is not
+   a complete guard — but they require deliberate intent, would
+   be caught in code review, and the runtime
+   `TestAllLabelConstantsInAllLabels` from §8 fails if any
+   declared constant goes missing. (Claude round-24 corrected
+   the prior overstated "mutation is a compile error" claim.)
+   Iteration uses `range AllLabels` (works identically for
+   arrays and slices) and `len(AllLabels)` returns 5 at compile
+   time. The
    `allLabelsWithLegacy()` helper converts to a freshly allocated
    slice via `make([]Label, len(AllLabels), len(AllLabels)+1);
    copy(result, AllLabels[:]); result = append(result, LabelLegacy)`

From 788b6858327919847b07e40451effd0748a34061 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 10:31:05 +0900
Subject: [PATCH 28/31] docs: address PR #694 round-25 (Claude R25 + Codex P2)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Round-25 (Claude bot R25 finished review + Codex P2):

Claude bot R25 verdict on `09c1c04e` was "design is ready
to merge as PR-A" with 4 minor stale-language items. All 4
addressed:

- §8 lens-5 Slot pre-creation count test: "(PR-C, post-
  AllLabels population)" + "PR-B counterpart (with empty
  AllLabels)" → "(PR-C+D+E, with --keyvizLabelsEnabled=
  true)" + "PR-B counterpart (with --keyvizLabelsEnabled=
  false, the default)". Added explicit note that PR-B vs
  PR-C+D+E counterparts differ on the flag, not on
  AllLabels length (the [5]Label array is populated from
  PR-B per round-23).
- §8 lens-5 RemoveRoute symmetric teardown test: same
  fix — "(PR-C)" + "(empty AllLabels)" → "(PR-C+D+E,
  flag on)" + "(with the flag off)". Counterpart
  assertion explained as "only the legacy slot was
  pre-created (flag off), so only it is retired".
- §7 PR-C+D+E opening sentence: "PR-C (adapter wiring +
  AllLabels population)" → "PR-C (adapter wiring only —
  AllLabels constants are already declared in PR-B as
  part of the [5]Label array)".

Codex P2: PR-B row's behavior-neutrality claim depends on
--keyvizLabelsEnabled being default-false, but PR-C+D+E
row said the flag is "added in PR-C+D+E". Implementer
following PR-C+D+E literally would conclude PR-B cannot
enforce its own gating. Fixed by:

- §7 PR-B row prepended: "PR-B introduces the
  --keyvizLabelsEnabled startup flag (default false) on
  the server binary, so the flag exists from PR-B
  onward and PR-B's behavior-neutrality claim does not
  depend on PR-C+D+E (Codex round-25 P2)".
- §7 PR-C+D+E rollout-gate paragraph: "PR-C+D+E adds
  --keyvizLabelsEnabled" → "--keyvizLabelsEnabled is
  introduced in PR-B... PR-C+D+E adds the coordinator
  override path". Clarifies PR-B introduces the flag
  for pre-creation gating; PR-C+D+E adds the
  coordinator-side override path on top, which only
  matters once adapters pass non-empty labels.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 38 ++++++++++---------
 1 file changed, 21 insertions(+), 17 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index d226f30c..e5dd9826 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -611,8 +611,8 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **`RegisterRoute` only pre-creates the legacy empty-label slot when `--keyvizLabelsEnabled=false` (the default)** — labeled-sibling pre-creation is gated on the flag (round-17), so PR-B with the default flag is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants**, and `AllLabels` as a **fixed-size array** `[5]Label{LabelDynamo, ..., LabelRawKV}` (round-23 — array form prevents importing-package mutation; see §9 Q2). The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with the flag default to false, `RegisterRoute` skips the labeled-sibling loop body so per-route slot count stays at 1 (Claude bot round-6 + round-23 array correction). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C+D+E wires the adapters to pass labels; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's flag-default-false (only legacy slots pre-created), `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C+D+E to flip the flag and start emitting labels. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `slotsPerRoute` (= `len(AllLabels)+1` when the flag is on, `1` when off; round-17 + round-23) so the cap continues to count **routes**, not slots; with the flag default to false in PR-B the divisor is 1 and behavior is identical to today (Claude round-24 fix to the prior "AllLabels empty in PR-B" framing — round-23 made AllLabels populated from PR-B, so the divisor=1 reason is the flag, not the array length). Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
-| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring + `AllLabels` population) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. The five canonical constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`) are already declared in `keyviz/labels.go` from PR-B as part of the `[5]Label` array (round-23 — see §9 Q2 and the PR-B row above); PR-C+D+E only wires the adapters to pass them, no `AllLabels` modification needed (Claude round-24 fix to the prior "populate empty slice" framing). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) **only when `--keyvizLabelsEnabled=true`**; with the flag off, `RegisterRoute` continues to pre-create just the legacy empty-label slot, identical to today's behavior (Codex round-15-4th-pass P1). **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `slotsPerRoute` (`len(AllLabels)+1` when the flag is on, `1` when off), so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots when labels are enabled, `MaxTrackedRoutes × 1` when off (today's baseline); documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. PR-C+D+E adds `--keyvizLabelsEnabled` (default `false`) on every node; when false, the **`ShardedCoordinator` overrides the adapter-supplied label to `keyviz.LabelLegacy` at the single `sampler.Observe(...)` call site** in `kv/sharded_coordinator.go` (one `if !s.keyvizLabelsEnabled { label = keyviz.LabelLegacy }` guard, not a 5-file flag-read duplication across adapters; Claude bot round-12 moderate). The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. <br>**Flag-flip activation**: once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true`. The flag is a startup-only `flag.Bool` (no live-toggle / config-reload path; Claude bot round-12 minor); changing it requires a process restart. There are two restart strategies: (a) **simultaneous restart of all nodes** — KeyViz heatmap is briefly unavailable during the restart window but no mixed-format fragmentation occurs; (b) **rolling restart** — for the duration of the restart (typically minutes), the heatmap shows a transient mixed view because the legacy `route:N` rows from not-yet-restarted nodes don't merge with `route:N:label` rows from restarted nodes. Since KeyViz is a monitoring view (not a consistency-sensitive system), the rolling-restart fragmentation is acceptable and clears as the final node restarts; operators who want zero fragmentation should use the simultaneous restart. (Claude bot round-12 moderate.) <br>**Flag also gates pre-allocation**: `--keyvizLabelsEnabled` is **both** the traffic-routing toggle (override label to `LabelLegacy` at the coordinator) **and** the slot-pre-creation toggle. When `false`, `RegisterRoute` pre-creates only the legacy empty-label slot (`len(next.slots) += 1` per route, identical to today's behavior); labeled siblings are not allocated. When `true`, `RegisterRoute` pre-creates the legacy slot **and** one labeled sibling per `AllLabels` member (`len(next.slots) += len(AllLabels)+1` per route). Memory therefore stays at the today's level (`MaxTrackedRoutes × 1`) for clusters that deploy the bundled binary but leave the flag off, and grows to `MaxTrackedRoutes × (len(AllLabels)+1)` only when the operator opts in. The flag-flip activation requires a process restart (see preceding paragraph), and `RegisterRoute` re-runs at startup, so the new pre-creation regime is in effect immediately after the restart — there is no live re-allocation path needed. **An earlier draft separated memory and traffic-routing into two regimes ("memory-vs-flag separation", round-12) where labeled slots were pre-created regardless of the flag; that was wrong (Codex round-15-4th-pass P1) — it imposed a multi-x memory penalty on clusters that never enable labels, contradicting the §2.1 minimal-penalty goal.** (Codex round-11 P2 originated this rollout gate.) |
+| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **PR-B introduces the `--keyvizLabelsEnabled` startup flag** (default `false`) on the server binary, so the flag exists from PR-B onward and PR-B's behavior-neutrality claim does not depend on PR-C+D+E (Codex round-25 P2). **`RegisterRoute` only pre-creates the legacy empty-label slot when `--keyvizLabelsEnabled=false` (the default)** — labeled-sibling pre-creation is gated on the flag (round-17), so PR-B with the default flag is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants**, and `AllLabels` as a **fixed-size array** `[5]Label{LabelDynamo, ..., LabelRawKV}` (round-23 — array form prevents importing-package mutation; see §9 Q2). The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with the flag default to false, `RegisterRoute` skips the labeled-sibling loop body so per-route slot count stays at 1 (Claude bot round-6 + round-23 array correction). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C+D+E wires the adapters to pass labels; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's flag-default-false (only legacy slots pre-created), `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C+D+E to flip the flag and start emitting labels. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `slotsPerRoute` (= `len(AllLabels)+1` when the flag is on, `1` when off; round-17 + round-23) so the cap continues to count **routes**, not slots; with the flag default to false in PR-B the divisor is 1 and behavior is identical to today (Claude round-24 fix to the prior "AllLabels empty in PR-B" framing — round-23 made AllLabels populated from PR-B, so the divisor=1 reason is the flag, not the array length). Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
+| **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring only — `AllLabels` constants are already declared in PR-B as part of the `[5]Label` array, see §9 Q2) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. The five canonical constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`) are already declared in `keyviz/labels.go` from PR-B as part of the `[5]Label` array (round-23 — see §9 Q2 and the PR-B row above); PR-C+D+E only wires the adapters to pass them, no `AllLabels` modification needed (Claude round-24 fix to the prior "populate empty slice" framing). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) **only when `--keyvizLabelsEnabled=true`**; with the flag off, `RegisterRoute` continues to pre-create just the legacy empty-label slot, identical to today's behavior (Codex round-15-4th-pass P1). **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `slotsPerRoute` (`len(AllLabels)+1` when the flag is on, `1` when off), so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots when labels are enabled, `MaxTrackedRoutes × 1` when off (today's baseline); documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. **`--keyvizLabelsEnabled` is introduced in PR-B** (default `false`) so PR-B's behavior-neutrality claim (`RegisterRoute` skips labeled-sibling pre-creation, slot count stays at 1) holds without any PR-C+D+E dependency. PR-C+D+E adds the **coordinator override path**: when the flag is false, the **`ShardedCoordinator` overrides the adapter-supplied label to `keyviz.LabelLegacy` at the single `sampler.Observe(...)` call site** in `kv/sharded_coordinator.go` (one `if !s.keyvizLabelsEnabled { label = keyviz.LabelLegacy }` guard, not a 5-file flag-read duplication across adapters; Claude bot round-12 moderate). The override path only matters in PR-C+D+E because that is when adapters first start passing non-empty labels; in PR-B all adapters still pass `label = ""` so the override is a no-op. (Codex round-25 P2 caught the prior wording that said "PR-C+D+E adds the flag" — it implied PR-B couldn't enforce its own gating, when in fact PR-B introduces the flag for pre-creation gating and PR-C+D+E only adds the override path on top.) The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. <br>**Flag-flip activation**: once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true`. The flag is a startup-only `flag.Bool` (no live-toggle / config-reload path; Claude bot round-12 minor); changing it requires a process restart. There are two restart strategies: (a) **simultaneous restart of all nodes** — KeyViz heatmap is briefly unavailable during the restart window but no mixed-format fragmentation occurs; (b) **rolling restart** — for the duration of the restart (typically minutes), the heatmap shows a transient mixed view because the legacy `route:N` rows from not-yet-restarted nodes don't merge with `route:N:label` rows from restarted nodes. Since KeyViz is a monitoring view (not a consistency-sensitive system), the rolling-restart fragmentation is acceptable and clears as the final node restarts; operators who want zero fragmentation should use the simultaneous restart. (Claude bot round-12 moderate.) <br>**Flag also gates pre-allocation**: `--keyvizLabelsEnabled` is **both** the traffic-routing toggle (override label to `LabelLegacy` at the coordinator) **and** the slot-pre-creation toggle. When `false`, `RegisterRoute` pre-creates only the legacy empty-label slot (`len(next.slots) += 1` per route, identical to today's behavior); labeled siblings are not allocated. When `true`, `RegisterRoute` pre-creates the legacy slot **and** one labeled sibling per `AllLabels` member (`len(next.slots) += len(AllLabels)+1` per route). Memory therefore stays at the today's level (`MaxTrackedRoutes × 1`) for clusters that deploy the bundled binary but leave the flag off, and grows to `MaxTrackedRoutes × (len(AllLabels)+1)` only when the operator opts in. The flag-flip activation requires a process restart (see preceding paragraph), and `RegisterRoute` re-runs at startup, so the new pre-creation regime is in effect immediately after the restart — there is no live re-allocation path needed. **An earlier draft separated memory and traffic-routing into two regimes ("memory-vs-flag separation", round-12) where labeled slots were pre-created regardless of the flag; that was wrong (Codex round-15-4th-pass P1) — it imposed a multi-x memory penalty on clusters that never enable labels, contradicting the §2.1 minimal-penalty goal.** (Codex round-11 P2 originated this rollout gate.) |
 
 PR-B is independent of the bundled PR-C+D+E: PR-B widens types
 and prepares the slot machinery without changing observable
@@ -656,24 +656,28 @@ atomic deploy. (Reviewer notes from PR #694: Claude bot critical
    sees the label-collapsed view; new SPA against old server
    sees the legacy view; both are coherent.
 5. **Test coverage** — New test categories:
-   - **Slot pre-creation count (PR-C, post-`AllLabels`
-     population)**: register 1 route, assert
-     `len(tbl.slots) == len(AllLabels) + 1`. Catches a PR-C
+   - **Slot pre-creation count (PR-C+D+E, with
+     `--keyvizLabelsEnabled=true`)**: register 1 route, assert
+     `len(tbl.slots) == len(AllLabels) + 1`. Catches a PR-C+D+E
      regression that skips a label constant in the pre-creation
      loop — without this the missing slot only surfaces when
      that adapter fires traffic. **PR-B counterpart** (with
-     empty `AllLabels`): register 1 route, assert
-     `len(tbl.slots) == 1` (just the legacy slot); pins
-     PR-B's behavior-neutral refactor goal.
-   - **`RemoveRoute` symmetric teardown (PR-C)**: register 1
-     route, `RemoveRoute(routeID)`, assert `len(tbl.slots) == 0`
-     and `len(retiredSlots) == len(AllLabels) + 1`. Catches a
-     PR-C regression where only the legacy slot is retired and
-     the labeled siblings leak into the live table. **PR-B
-     counterpart** (empty `AllLabels`): register 1 route,
-     `RemoveRoute(routeID)`, assert
-     `len(retiredSlots) == 1` — the loop body executes once
-     for the empty-label slot only.
+     `--keyvizLabelsEnabled=false`, the default): register 1
+     route, assert `len(tbl.slots) == 1` (just the legacy slot);
+     pins PR-B's behavior-neutral refactor goal. The `[5]Label`
+     array is populated from PR-B (round-23, see §9 Q2); the
+     PR-B vs PR-C+D+E counterparts differ on the flag, not on
+     `AllLabels` length.
+   - **`RemoveRoute` symmetric teardown (PR-C+D+E, flag on)**:
+     register 1 route, `RemoveRoute(routeID)`, assert
+     `len(tbl.slots) == 0` and
+     `len(retiredSlots) == len(AllLabels) + 1`. Catches a
+     regression where only the legacy slot is retired and the
+     labeled siblings leak into the live table. **PR-B
+     counterpart** (with the flag off): register 1 route,
+     `RemoveRoute(routeID)`, assert `len(retiredSlots) == 1`
+     — only the legacy slot was pre-created (flag off), so only
+     it is retired.
    - **`mergeRowInto` Label copy** (PR-D+E): two nodes, one with
      `KeyVizRow{Label: "dynamo"}`, one with `KeyVizRow{Label:
      "redis"}` for the same routeID. The merged response must

From e7b0c132aed097b07d5c1a16ee3e0858d8380140 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 10:37:07 +0900
Subject: [PATCH 29/31] docs: address PR #694 round-26 (Codex P2)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Round-26 Codex P2 on `788b6858`:

The round-25 PR-B row said "RegisterRoute only pre-creates
the legacy empty-label slot when --keyvizLabelsEnabled=false
(the default) — labeled-sibling pre-creation is gated on the
flag (round-17)". This implied that an operator setting
--keyvizLabelsEnabled=true in PR-B would cause RegisterRoute
to pre-create len(AllLabels)+1 slots, reintroducing the
early memory/route-budget regression PR-C+D+E tries to
defer.

Per Codex round-26 P2, PR-B should be unambiguously
legacy-slot-only regardless of the flag. The flag-honoring
RegisterRoute code lands in PR-C+D+E, not PR-B.

Updates:

- §7 PR-B row: clarified that PR-B introduces the flag for
  CLI parsing + Opts plumbing only; RegisterRoute in PR-B
  unconditionally pre-creates only the legacy slot
  regardless of the flag value. The flag-honoring loop
  lands in PR-C+D+E. Per-route slot count is unconditionally
  1 in PR-B.
- §7 closing paragraph: same clarification — PR-B is
  legacy-slots-only because the flag-honoring
  pre-creation loop is in PR-C+D+E, not because the flag
  is default-false.
- §4.1.1 PR-B/PR-C+D+E body pseudocode (sampler.go:416
  slotsPerRoute calculation): split into two phases —
  PR-B body stops at `slotsPerRoute := 1` (literal, no
  flag check), PR-C+D+E extends to the flag-gated
  conditional branch. Comment updated accordingly.

The flag still exists from PR-B (per round-25 P2 — PR-B
needs it on Opts for downstream wiring), but PR-B treats
it as dormant for slot pre-creation. PR-C+D+E activates
the flag's pre-creation effect in tandem with the
adapter-wiring + wire-format changes.
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 32 +++++++++++++------
 1 file changed, 23 insertions(+), 9 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index e5dd9826..8101f5f3 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -276,10 +276,18 @@ when labels are enabled, `MaxTrackedRoutes × 1` when off), which
 we document but never count against the cap.
 
 ```go
-// keyviz/sampler.go:416 (PR-B + PR-C+D+E flag-gated, Codex round-15-4th-pass P1)
-slotsPerRoute := 1 // legacy slot only (PR-B, or PR-C+D+E with flag off)
+// keyviz/sampler.go:416
+//
+// PR-B body: slotsPerRoute is the literal `1`; the
+// flag-honoring branch lands in PR-C+D+E (Codex round-26 P2 —
+// PR-B is unconditionally legacy-slot-only regardless of the
+// flag).
+slotsPerRoute := 1 // PR-B body stops here
+
+// PR-C+D+E extends to (Codex round-15-4th-pass P1):
+slotsPerRoute := 1 // legacy slot only (when flag off)
 if s.opts.KeyVizLabelsEnabled {
-    slotsPerRoute = len(AllLabels) + 1 // 6 after PR-C with flag on (in-package, no keyviz. qualifier)
+    slotsPerRoute = len(AllLabels) + 1 // 6 with flag on (in-package, no keyviz. qualifier)
 }
 if len(next.slots) / slotsPerRoute < s.opts.MaxTrackedRoutes {
     // ... pre-create slot(s) for this route
@@ -611,16 +619,22 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **PR-B introduces the `--keyvizLabelsEnabled` startup flag** (default `false`) on the server binary, so the flag exists from PR-B onward and PR-B's behavior-neutrality claim does not depend on PR-C+D+E (Codex round-25 P2). **`RegisterRoute` only pre-creates the legacy empty-label slot when `--keyvizLabelsEnabled=false` (the default)** — labeled-sibling pre-creation is gated on the flag (round-17), so PR-B with the default flag is a behavior-neutral refactor (per-route slot count is unchanged at 1). **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants**, and `AllLabels` as a **fixed-size array** `[5]Label{LabelDynamo, ..., LabelRawKV}` (round-23 — array form prevents importing-package mutation; see §9 Q2). The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop and `RegisterRoute`'s pre-creation loop both compile; with the flag default to false, `RegisterRoute` skips the labeled-sibling loop body so per-route slot count stays at 1 (Claude bot round-6 + round-23 array correction). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C+D+E wires the adapters to pass labels; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's flag-default-false (only legacy slots pre-created), `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C+D+E to flip the flag and start emitting labels. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `slotsPerRoute` (= `len(AllLabels)+1` when the flag is on, `1` when off; round-17 + round-23) so the cap continues to count **routes**, not slots; with the flag default to false in PR-B the divisor is 1 and behavior is identical to today (Claude round-24 fix to the prior "AllLabels empty in PR-B" framing — round-23 made AllLabels populated from PR-B, so the divisor=1 reason is the flag, not the array length). Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
+| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **PR-B introduces the `--keyvizLabelsEnabled` startup flag** (default `false`) on the server binary — CLI parsing and the `KeyVizLabelsEnabled bool` option on `ShardedCoordinatorOpts` / `MemSamplerOpts` ship in PR-B, so the flag exists from PR-B onward and PR-B's behavior-neutrality claim does not depend on PR-C+D+E (Codex round-25 P2). **`RegisterRoute` in PR-B always pre-creates only the legacy empty-label slot, regardless of the flag** (Codex round-26 P2 — earlier round-25 wording said pre-creation is "gated on the flag" in PR-B, which would have allowed an operator to enable the flag in PR-B and start allocating `len(AllLabels)+1` slots before any of the PR-C+D+E adapter-wiring or wire-format machinery existed). The `RegisterRoute` flag-honoring code lands in **PR-C+D+E**, not PR-B; in PR-B the flag is **dormant** for slot pre-creation (it exists for option plumbing only). Per-route slot count in PR-B is therefore unconditionally 1, regardless of the flag, and PR-B is fully behavior-neutral. **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants**, and `AllLabels` as a **fixed-size array** `[5]Label{LabelDynamo, ..., LabelRawKV}` (round-23 — array form prevents importing-package mutation; see §9 Q2). The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop compiles in PR-B; `RegisterRoute`'s flag-honoring pre-creation loop is **not yet present in PR-B** (it lands in PR-C+D+E per the previous paragraph), so PR-B's `RegisterRoute` body is a single legacy-slot create call regardless of the flag. With PR-B unconditionally pre-creating only the legacy slot, per-route slot count stays at 1 (Claude bot round-6 + round-23 array correction + Codex round-26 P2 unconditionality). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C+D+E wires the adapters to pass labels; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's flag-default-false (only legacy slots pre-created), `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C+D+E to flip the flag and start emitting labels. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `slotsPerRoute` (= `len(AllLabels)+1` when the flag is on, `1` when off; round-17 + round-23) so the cap continues to count **routes**, not slots; with the flag default to false in PR-B the divisor is 1 and behavior is identical to today (Claude round-24 fix to the prior "AllLabels empty in PR-B" framing — round-23 made AllLabels populated from PR-B, so the divisor=1 reason is the flag, not the array length). Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
 | **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring only — `AllLabels` constants are already declared in PR-B as part of the `[5]Label` array, see §9 Q2) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. The five canonical constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`) are already declared in `keyviz/labels.go` from PR-B as part of the `[5]Label` array (round-23 — see §9 Q2 and the PR-B row above); PR-C+D+E only wires the adapters to pass them, no `AllLabels` modification needed (Claude round-24 fix to the prior "populate empty slice" framing). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) **only when `--keyvizLabelsEnabled=true`**; with the flag off, `RegisterRoute` continues to pre-create just the legacy empty-label slot, identical to today's behavior (Codex round-15-4th-pass P1). **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `slotsPerRoute` (`len(AllLabels)+1` when the flag is on, `1` when off), so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots when labels are enabled, `MaxTrackedRoutes × 1` when off (today's baseline); documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. **`--keyvizLabelsEnabled` is introduced in PR-B** (default `false`) so PR-B's behavior-neutrality claim (`RegisterRoute` skips labeled-sibling pre-creation, slot count stays at 1) holds without any PR-C+D+E dependency. PR-C+D+E adds the **coordinator override path**: when the flag is false, the **`ShardedCoordinator` overrides the adapter-supplied label to `keyviz.LabelLegacy` at the single `sampler.Observe(...)` call site** in `kv/sharded_coordinator.go` (one `if !s.keyvizLabelsEnabled { label = keyviz.LabelLegacy }` guard, not a 5-file flag-read duplication across adapters; Claude bot round-12 moderate). The override path only matters in PR-C+D+E because that is when adapters first start passing non-empty labels; in PR-B all adapters still pass `label = ""` so the override is a no-op. (Codex round-25 P2 caught the prior wording that said "PR-C+D+E adds the flag" — it implied PR-B couldn't enforce its own gating, when in fact PR-B introduces the flag for pre-creation gating and PR-C+D+E only adds the override path on top.) The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. <br>**Flag-flip activation**: once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true`. The flag is a startup-only `flag.Bool` (no live-toggle / config-reload path; Claude bot round-12 minor); changing it requires a process restart. There are two restart strategies: (a) **simultaneous restart of all nodes** — KeyViz heatmap is briefly unavailable during the restart window but no mixed-format fragmentation occurs; (b) **rolling restart** — for the duration of the restart (typically minutes), the heatmap shows a transient mixed view because the legacy `route:N` rows from not-yet-restarted nodes don't merge with `route:N:label` rows from restarted nodes. Since KeyViz is a monitoring view (not a consistency-sensitive system), the rolling-restart fragmentation is acceptable and clears as the final node restarts; operators who want zero fragmentation should use the simultaneous restart. (Claude bot round-12 moderate.) <br>**Flag also gates pre-allocation**: `--keyvizLabelsEnabled` is **both** the traffic-routing toggle (override label to `LabelLegacy` at the coordinator) **and** the slot-pre-creation toggle. When `false`, `RegisterRoute` pre-creates only the legacy empty-label slot (`len(next.slots) += 1` per route, identical to today's behavior); labeled siblings are not allocated. When `true`, `RegisterRoute` pre-creates the legacy slot **and** one labeled sibling per `AllLabels` member (`len(next.slots) += len(AllLabels)+1` per route). Memory therefore stays at the today's level (`MaxTrackedRoutes × 1`) for clusters that deploy the bundled binary but leave the flag off, and grows to `MaxTrackedRoutes × (len(AllLabels)+1)` only when the operator opts in. The flag-flip activation requires a process restart (see preceding paragraph), and `RegisterRoute` re-runs at startup, so the new pre-creation regime is in effect immediately after the restart — there is no live re-allocation path needed. **An earlier draft separated memory and traffic-routing into two regimes ("memory-vs-flag separation", round-12) where labeled slots were pre-created regardless of the flag; that was wrong (Codex round-15-4th-pass P1) — it imposed a multi-x memory penalty on clusters that never enable labels, contradicting the §2.1 minimal-penalty goal.** (Codex round-11 P2 originated this rollout gate.) |
 
 PR-B is independent of the bundled PR-C+D+E: PR-B widens types
 and prepares the slot machinery without changing observable
-behavior (legacy slots only, `AllLabels` is the populated
-`[5]Label` array but `--keyvizLabelsEnabled` defaults to false
-so `RegisterRoute` skips the labeled-sibling loop and the
-`MaxTrackedRoutes` divisor stays at 1; round-23 array form, see
-§9 Q2). The bundled PR-C+D+E then ships everything
+behavior (legacy slots only — `RegisterRoute` in PR-B
+unconditionally pre-creates only the legacy slot regardless of
+`--keyvizLabelsEnabled`, since the flag-honoring pre-creation
+loop lands in PR-C+D+E, not PR-B; per Codex round-26 P2 — see
+the PR-B row above). `AllLabels` is the populated `[5]Label`
+array from PR-B (round-23, see §9 Q2), but its only consumers in
+PR-B are `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop
+(over an empty live-slot set under the unconditional PR-B
+behavior) and the `TestAllLabels…` regression guards in §8.
+`MaxTrackedRoutes` divisor stays at 1 in PR-B because per-route
+slot count is 1. The bundled PR-C+D+E then ships everything
 operator-visible — adapter wiring (constants are already
 declared in `keyviz/labels.go` from PR-B; PR-C+D+E only wires
 adapters to pass them at the dispatch entry, no change to

From 44d7cd453ea25bbb132805ee5f1cf8ba28cff7ab Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 10:43:11 +0900
Subject: [PATCH 30/31] docs: address PR #694 round-27 (Claude R27)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Claude R27 verdict on `e7b0c132`: "design is ready to merge
as PR-A after these 2 fixes". Both items addressed:

- §9 Q2 PR sequencing note (lines 835-840) directly
  contradicted the round-26 §7 PR-B fix. Said "PR-B's
  RegisterRoute only pre-creates labeled siblings when
  --keyvizLabelsEnabled=true" — opposite of round-26's
  "unconditionally legacy-slot-only regardless of the
  flag". Updated to match: "With the round-23 array form,
  AllLabels is non-empty from PR-B onward, AND with the
  round-26 unconditional model, PR-B's RegisterRoute
  always pre-creates only the legacy slot regardless of
  the flag — the flag-honoring loop is simply not present
  in PR-B (lands in PR-C+D+E). PR-B remains
  behavior-neutral unconditionally, not via flag default
  or empty slice." The flag is introduced in PR-B for
  Opts plumbing only; its read-site doesn't exist in PR-B
  so its value has no observable effect.
- §4.1.1 dual `slotsPerRoute :=` in single Go code block
  (lines 285-291) is a Go compile error — two `:=`
  declarations in the same scope ("no new variables on
  left side of :="). Split into two separate code blocks:
  PR-B body (unconditional `slotsPerRoute := 1`) and
  PR-C+D+E body (replaces the PR-B body with the flag-gated
  conditional). Added explicit narrative: "PR-C+D+E
  replaces the literal 1 with the flag-gated conditional".
---
 ...26_04_28_proposed_keyviz_adapter_labels.md | 47 ++++++++++++-------
 1 file changed, 31 insertions(+), 16 deletions(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 8101f5f3..4c9b2e01 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -275,16 +275,21 @@ slotsPerRoute` (i.e. `MaxTrackedRoutes × (len(AllLabels) + 1)`
 when labels are enabled, `MaxTrackedRoutes × 1` when off), which
 we document but never count against the cap.
 
+PR-B body (unconditional, no flag check — Codex round-26 P2):
+
 ```go
-// keyviz/sampler.go:416
-//
-// PR-B body: slotsPerRoute is the literal `1`; the
-// flag-honoring branch lands in PR-C+D+E (Codex round-26 P2 —
-// PR-B is unconditionally legacy-slot-only regardless of the
-// flag).
-slotsPerRoute := 1 // PR-B body stops here
-
-// PR-C+D+E extends to (Codex round-15-4th-pass P1):
+// keyviz/sampler.go:416 — PR-B body
+slotsPerRoute := 1 // legacy slot only; unconditional in PR-B
+if len(next.slots) / slotsPerRoute < s.opts.MaxTrackedRoutes {
+    // ... pre-create slot(s) for this route
+}
+```
+
+PR-C+D+E replaces the literal `1` with the flag-gated
+conditional (Codex round-15-4th-pass P1):
+
+```go
+// keyviz/sampler.go:416 — PR-C+D+E body (replaces PR-B body above)
 slotsPerRoute := 1 // legacy slot only (when flag off)
 if s.opts.KeyVizLabelsEnabled {
     slotsPerRoute = len(AllLabels) + 1 // 6 with flag on (in-package, no keyviz. qualifier)
@@ -294,6 +299,11 @@ if len(next.slots) / slotsPerRoute < s.opts.MaxTrackedRoutes {
 }
 ```
 
+(Two separate code blocks — the dual `slotsPerRoute :=` form
+in a single Go scope is a compile error, "no new variables on
+left side of `:=`"; Claude round-27 minor caught the
+single-block presentation.)
+
 The alternative (Option B: `MaxTrackedRoutes` redefined as
 "slots") would silently halve the route capacity at PR-C
 rollout — an operator setting 1024 today would suddenly track
@@ -831,13 +841,18 @@ atomic deploy. (Reviewer notes from PR #694: Claude bot critical
    **PR sequencing note**: §7 PR-B previously declared `AllLabels`
    as a *deliberately-empty slice* (`var AllLabels []Label{}`) to
    make the pre-creation loop a no-op until PR-C populated it.
-   With the round-23 array form, AllLabels is non-empty from PR-B
-   onward — but the round-17 flag-gating means PR-B's
-   `RegisterRoute` only pre-creates labeled siblings when
-   `--keyvizLabelsEnabled=true`, which defaults to false. So
-   PR-B remains behavior-neutral via the flag, not via an empty
-   slice. The §7 PR-B row already mentions the flag default; this
-   is the same mechanism reused.
+   With the round-23 array form, `AllLabels` is non-empty from PR-B
+   onward, AND with the round-26 unconditional model, PR-B's
+   `RegisterRoute` always pre-creates only the legacy slot
+   regardless of the flag — the flag-honoring pre-creation loop
+   is simply not present in PR-B (it lands in PR-C+D+E along with
+   the adapter wiring and wire-format changes). So PR-B remains
+   behavior-neutral **unconditionally**, not via a flag default
+   or an empty slice. The flag is introduced in PR-B for option
+   plumbing only (`KeyVizLabelsEnabled bool` on `Opts`), so
+   PR-C+D+E can read it without an Opts-API change in PR-C+D+E
+   itself; in PR-B the flag's read-site doesn't exist yet, so its
+   value has no observable effect. (Codex round-25 P2 + round-26 P2.)
 
    **Why a typed `Label` rather than plain `string`** (Codex round-9):
    slot pre-creation iterates `AllLabels`, and `Observe` drops slot

From c788a96c5a6a1921a43b5aa548c577e1ef783c48 Mon Sep 17 00:00:00 2001
From: "Yoshiaki Ueda (bootjp)" <contact@bootjp.me>
Date: Wed, 29 Apr 2026 10:45:28 +0900
Subject: [PATCH 31/31] docs: address PR #694 round-28 (Codex P1 #2 stale
 divisor)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Codex round-27 P1 #2 on `e7b0c132` (still applicable to
`44d7cd45`): PR-B row claimed RegisterRoute is unconditionally
legacy-slot-only (round-26 fix), but later in the same row
said "MaxTrackedRoutes coarsening divides by slotsPerRoute
(= len(AllLabels)+1 when the flag is on, 1 when off)".
Implementer setting --keyvizLabelsEnabled=true on PR-B
binary would compute divisor=6 with only 1 slot per route,
expanding tracked-route capacity 6× beyond MaxTrackedRoutes.

Fixed by aligning the §7 PR-B row's divisor description with
round-26's unconditional model:

- "MaxTrackedRoutes coarsening check in PR-B uses
  slotsPerRoute as the literal 1 — same unconditionality
  reason as the RegisterRoute body above; the flag-gated
  conditional lands in PR-C+D+E, not PR-B"
- "PR-B with slotsPerRoute=1 keeps the cap counting routes
  (1 slot per route × N routes ≤ MaxTrackedRoutes),
  behavior identical to today regardless of any flag value"

Also tightened the secondary "flag-default-false" reference
in the L0 chain failure-mode discussion: the legacy-only
behavior comes from the unconditional code structure, not
from the flag default. Updated to "With PR-B unconditionally
pre-creating only the legacy slot (regardless of flag, per
round-26 P2)".

Codex round-27 P1 #1 (PR-B labeled-siblings claim in §9 Q2)
was already addressed in round-27 commit 44d7cd45.
---
 docs/design/2026_04_28_proposed_keyviz_adapter_labels.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
index 4c9b2e01..88ad5e4e 100644
--- a/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
+++ b/docs/design/2026_04_28_proposed_keyviz_adapter_labels.md
@@ -629,7 +629,7 @@ Virtual-bucket rows in the heatmap render as
 | PR | Scope |
 |---|---|
 | **PR-A** | Land this design doc. |
-| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **PR-B introduces the `--keyvizLabelsEnabled` startup flag** (default `false`) on the server binary — CLI parsing and the `KeyVizLabelsEnabled bool` option on `ShardedCoordinatorOpts` / `MemSamplerOpts` ship in PR-B, so the flag exists from PR-B onward and PR-B's behavior-neutrality claim does not depend on PR-C+D+E (Codex round-25 P2). **`RegisterRoute` in PR-B always pre-creates only the legacy empty-label slot, regardless of the flag** (Codex round-26 P2 — earlier round-25 wording said pre-creation is "gated on the flag" in PR-B, which would have allowed an operator to enable the flag in PR-B and start allocating `len(AllLabels)+1` slots before any of the PR-C+D+E adapter-wiring or wire-format machinery existed). The `RegisterRoute` flag-honoring code lands in **PR-C+D+E**, not PR-B; in PR-B the flag is **dormant** for slot pre-creation (it exists for option plumbing only). Per-route slot count in PR-B is therefore unconditionally 1, regardless of the flag, and PR-B is fully behavior-neutral. **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants**, and `AllLabels` as a **fixed-size array** `[5]Label{LabelDynamo, ..., LabelRawKV}` (round-23 — array form prevents importing-package mutation; see §9 Q2). The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop compiles in PR-B; `RegisterRoute`'s flag-honoring pre-creation loop is **not yet present in PR-B** (it lands in PR-C+D+E per the previous paragraph), so PR-B's `RegisterRoute` body is a single legacy-slot create call regardless of the flag. With PR-B unconditionally pre-creating only the legacy slot, per-route slot count stays at 1 (Claude bot round-6 + round-23 array correction + Codex round-26 P2 unconditionality). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C+D+E wires the adapters to pass labels; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B's flag-default-false (only legacy slots pre-created), `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the wiring is in place for PR-C+D+E to flip the flag and start emitting labels. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) divides by `slotsPerRoute` (= `len(AllLabels)+1` when the flag is on, `1` when off; round-17 + round-23) so the cap continues to count **routes**, not slots; with the flag default to false in PR-B the divisor is 1 and behavior is identical to today (Claude round-24 fix to the prior "AllLabels empty in PR-B" framing — round-23 made AllLabels populated from PR-B, so the divisor=1 reason is the flag, not the array length). Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
+| **PR-B** | Sampler API extension: extend the `Sampler` interface declaration (`keyviz/sampler.go:74`) AND the `MemSampler.Observe` implementation to `Observe(routeID uint64, op Op, keyLen, valueLen int, label Label)` (Claude bot round-8 minor: surfaces immediately as a compile error if any mock implementer is missed, but naming it keeps the scope complete). `routeSlot.Label`, slot-key type widens to `slotKey{uint64, Label}`. **PR-B introduces the `--keyvizLabelsEnabled` startup flag** (default `false`) on the server binary — CLI parsing and the `KeyVizLabelsEnabled bool` option on `ShardedCoordinatorOpts` / `MemSamplerOpts` ship in PR-B, so the flag exists from PR-B onward and PR-B's behavior-neutrality claim does not depend on PR-C+D+E (Codex round-25 P2). **`RegisterRoute` in PR-B always pre-creates only the legacy empty-label slot, regardless of the flag** (Codex round-26 P2 — earlier round-25 wording said pre-creation is "gated on the flag" in PR-B, which would have allowed an operator to enable the flag in PR-B and start allocating `len(AllLabels)+1` slots before any of the PR-C+D+E adapter-wiring or wire-format machinery existed). The `RegisterRoute` flag-honoring code lands in **PR-C+D+E**, not PR-B; in PR-B the flag is **dormant** for slot pre-creation (it exists for option plumbing only). Per-route slot count in PR-B is therefore unconditionally 1, regardless of the flag, and PR-B is fully behavior-neutral. **PR-B creates `keyviz/labels.go`** with the `slotKey` struct, the `type Label string` definition, the `LabelLegacy`/`LabelDynamo`/`LabelRedis`/etc. **typed constants**, and `AllLabels` as a **fixed-size array** `[5]Label{LabelDynamo, ..., LabelRawKV}` (round-23 — array form prevents importing-package mutation; see §9 Q2). The constants must exist at compile time so `RemoveRoute`'s `allLabelsWithLegacy()` teardown loop compiles in PR-B; `RegisterRoute`'s flag-honoring pre-creation loop is **not yet present in PR-B** (it lands in PR-C+D+E per the previous paragraph), so PR-B's `RegisterRoute` body is a single legacy-slot create call regardless of the flag. With PR-B unconditionally pre-creating only the legacy slot, per-route slot count stays at 1 (Claude bot round-6 + round-23 array correction + Codex round-26 P2 unconditionality). `MatrixRow.Label` field present. **L0 label copy** (Claude bot round-8 moderate): update `appendDrainedRow` (`keyviz/sampler.go:828`) to include `Label: slot.Label` in the `MatrixRow` struct literal — `slot.Label` is set once at `RegisterRoute` time and immutable, so read it directly rather than threading through `snapshotMeta`. This is the first link in the propagation chain: `slot.Label` → `MatrixRow.Label` (L0, here) → `KeyVizRow.Label` via `newKeyVizRowFrom` (L1, PR-D+E) → merged `KeyVizRow.Label` via `mergeRowInto` (L2, PR-D+E). Missing L0 silently truncates the chain and leaves `MatrixRow.Label = ""` even after PR-C+D+E wires the adapters to pass labels; L1 and L2 then faithfully copy the empty string and the feature collapses. With PR-B unconditionally pre-creating only the legacy slot (regardless of flag, per round-26 P2), `slot.Label` is `""` so the emitted row is `""`-labeled (identical to today's behavior), but the type-widening + L0 wiring is in place for PR-C+D+E to land the flag-honoring `RegisterRoute` loop and start emitting labels. `reclaimRetiredSlot` dedupes on `(RouteID, Label)` (forward-prep; only `Label=""` exists). The `MaxTrackedRoutes` coarsening check (`keyviz/sampler.go:416`) in PR-B uses `slotsPerRoute` as the **literal `1`** — same unconditionality reason as the `RegisterRoute` body above (Codex round-26 + round-27 P1: the flag-gated `slotsPerRoute := 1; if KeyVizLabelsEnabled { slotsPerRoute = len(AllLabels)+1 }` lands in PR-C+D+E, not PR-B). PR-B with `slotsPerRoute = 1` keeps the cap counting routes (1 slot per route × N routes ≤ MaxTrackedRoutes), behavior identical to today regardless of any flag value. Update existing tests **and the coordinator call sites in `kv/sharded_coordinator.go`** (the only non-test caller of `Sampler.Observe`) to pass `label = ""`. |
 | **PR-C+D+E** | **Bundled — must ship together.** PR-C (adapter wiring only — `AllLabels` constants are already declared in PR-B as part of the `[5]Label` array, see §9 Q2) and PR-D+E (wire format + pivot-key widening) are not separately shippable: once an adapter calls `Observe(..., LabelDynamo)`, the sampler emits multiple `MatrixRow` per route in a single column (one per label); without the pivot-key widening from PR-D+E, `pivotKeyVizColumns` and `matrixToProto` collapse those rows back into a single `RouteID`-keyed entry where each labeled row overwrites the previous one — non-deterministic data loss in the intermediate state. Earlier drafts framed PR-C and PR-D+E as separately shippable; that was wrong (Codex round-10 P1). The bundle covers: <br>**Adapter wiring (was PR-C)**: each adapter sets its label at the `ShardedCoordinator.Observe…` dispatch entry. The five canonical constants (`LabelDynamo`, `LabelRedis`, `LabelS3`, `LabelSQS`, `LabelRawKV`) are already declared in `keyviz/labels.go` from PR-B as part of the `[5]Label` array (round-23 — see §9 Q2 and the PR-B row above); PR-C+D+E only wires the adapters to pass them, no `AllLabels` modification needed (Claude round-24 fix to the prior "populate empty slice" framing). Extend `RegisterRoute` to pre-create the labeled siblings (one slot per `AllLabels` member) **only when `--keyvizLabelsEnabled=true`**; with the flag off, `RegisterRoute` continues to pre-create just the legacy empty-label slot, identical to today's behavior (Codex round-15-4th-pass P1). **MaxTrackedRoutes is unchanged in semantics** — the §4.1.1 coarsening check divides slot count by `slotsPerRoute` (`len(AllLabels)+1` when the flag is on, `1` when off), so the existing operator-set cap continues to mean "individually-tracked routes" exactly as before; **operators do not need to raise `--keyvizMaxTrackedRoutes`** (an earlier draft told operators to scale the cap by `× (len(labels)+1)` — that contradicted Option A; Codex round-6). Memory growth: `MaxTrackedRoutes × (len(AllLabels) + 1)` slots when labels are enabled, `MaxTrackedRoutes × 1` when off (today's baseline); documented but not capped. Emit `slog.Warn` from inside `RegisterRoute` when a route coarsens. <br>**Wire-format extension (was PR-D+E)**: proto + JSON `bucket_id` composite + optional `label` field, plus the SPA `route:N / label` rendering AND the **five** code changes across the three response paths: <br>**Single-node JSON path (`internal/admin/keyviz_handler.go`)**: <br>(a) `pivotKeyVizColumns` `rowsByID` map AND `order` slice both widen from `uint64` to the composite `BucketID string` — widening only the map without the `order` slice is a compile error; <br>(b) `newKeyVizRowFrom` (`keyviz_handler.go:368`) copies `mr.Label → row.Label` via an explicit `string(mr.Label)` cast (`MatrixRow.Label` is the typed `keyviz.Label`; `KeyVizRow.Label` is plain `string` for wire-format flexibility — Claude bot round-10 minor) — **first-level** Label copy, affects single-node and cluster deployments alike; <br>(c) `bucketIDFor` (`keyviz_handler.go:383`) returns the composite `"route:<id>:<label>"` when `mr.Label != ""`, falling back to the legacy `"route:<id>"` for empty labels — without this `BucketID` is non-unique and `applyKeyVizRowBudget` / `sortKeyVizRowsByStart` lose their deterministic tiebreak; <br>**Fan-out JSON path (`internal/admin/keyviz_fanout.go`)**: <br>(d) `mergeRowInto` (`keyviz_fanout.go:509`) adds `dst.Label = row.Label` — **second-level** Label copy, only the cluster fan-out path touches this; <br>**gRPC path (`adapter/admin_grpc.go`)**: <br>(e) `matrixToProto` (`admin_grpc.go:599`) and the per-row conversion it drives: (e1) **widen `rowsByID` (line 603) and `order` (line 604) from `uint64` to the composite `BucketID string` key** — same widening as item (a); without it `(routeID=1, label="dynamo")` and `(routeID=1, label="redis")` collapse to the same map entry; (e2) copy `MatrixRow.Label → KeyVizRow.label` (proto field 4) via `string(mr.Label)` cast (same typed→untyped reasoning as item (b)); (e3) emit composite `bucket_id` (`"route:<id>:<label>"`). Without (e1)–(e3), `GetKeyVizMatrix` gRPC clients receive collapsed unlabeled rows even though HTTP/SPA responses now show per-label rows. <br>All five copies are required; missing any one leaves a flavour of deployment with empty labels. Splitting the bundle into separate PR-C and PR-D+E was the original framing but is now rejected (see opening paragraph of this row). <br>**Operator-controlled rollout gate (rolling-upgrade safety)**: a normal rolling upgrade temporarily mixes nodes that emit legacy `route:<id>` rows with nodes that emit labeled `route:<id>:<label>` rows. The fan-out aggregator keys strictly by `BucketID` in `mergeRowInto`, so those rows do **not** merge — operators would see fragmented unlabeled-plus-labeled data per route until every node converges. **`--keyvizLabelsEnabled` is introduced in PR-B** (default `false`) so PR-B's behavior-neutrality claim (`RegisterRoute` skips labeled-sibling pre-creation, slot count stays at 1) holds without any PR-C+D+E dependency. PR-C+D+E adds the **coordinator override path**: when the flag is false, the **`ShardedCoordinator` overrides the adapter-supplied label to `keyviz.LabelLegacy` at the single `sampler.Observe(...)` call site** in `kv/sharded_coordinator.go` (one `if !s.keyvizLabelsEnabled { label = keyviz.LabelLegacy }` guard, not a 5-file flag-read duplication across adapters; Claude bot round-12 moderate). The override path only matters in PR-C+D+E because that is when adapters first start passing non-empty labels; in PR-B all adapters still pass `label = ""` so the override is a no-op. (Codex round-25 P2 caught the prior wording that said "PR-C+D+E adds the flag" — it implied PR-B couldn't enforce its own gating, when in fact PR-B introduces the flag for pre-creation gating and PR-C+D+E only adds the override path on top.) The bundled binary is therefore safe to roll out one node at a time — every node, mixed or fully upgraded, emits the legacy format. <br>**Flag-flip activation**: once the fleet is fully on the new binary, the operator flips `--keyvizLabelsEnabled=true`. The flag is a startup-only `flag.Bool` (no live-toggle / config-reload path; Claude bot round-12 minor); changing it requires a process restart. There are two restart strategies: (a) **simultaneous restart of all nodes** — KeyViz heatmap is briefly unavailable during the restart window but no mixed-format fragmentation occurs; (b) **rolling restart** — for the duration of the restart (typically minutes), the heatmap shows a transient mixed view because the legacy `route:N` rows from not-yet-restarted nodes don't merge with `route:N:label` rows from restarted nodes. Since KeyViz is a monitoring view (not a consistency-sensitive system), the rolling-restart fragmentation is acceptable and clears as the final node restarts; operators who want zero fragmentation should use the simultaneous restart. (Claude bot round-12 moderate.) <br>**Flag also gates pre-allocation**: `--keyvizLabelsEnabled` is **both** the traffic-routing toggle (override label to `LabelLegacy` at the coordinator) **and** the slot-pre-creation toggle. When `false`, `RegisterRoute` pre-creates only the legacy empty-label slot (`len(next.slots) += 1` per route, identical to today's behavior); labeled siblings are not allocated. When `true`, `RegisterRoute` pre-creates the legacy slot **and** one labeled sibling per `AllLabels` member (`len(next.slots) += len(AllLabels)+1` per route). Memory therefore stays at the today's level (`MaxTrackedRoutes × 1`) for clusters that deploy the bundled binary but leave the flag off, and grows to `MaxTrackedRoutes × (len(AllLabels)+1)` only when the operator opts in. The flag-flip activation requires a process restart (see preceding paragraph), and `RegisterRoute` re-runs at startup, so the new pre-creation regime is in effect immediately after the restart — there is no live re-allocation path needed. **An earlier draft separated memory and traffic-routing into two regimes ("memory-vs-flag separation", round-12) where labeled slots were pre-created regardless of the flag; that was wrong (Codex round-15-4th-pass P1) — it imposed a multi-x memory penalty on clusters that never enable labels, contradicting the §2.1 minimal-penalty goal.** (Codex round-11 P2 originated this rollout gate.) |
 
 PR-B is independent of the bundled PR-C+D+E: PR-B widens types