From d667b526eac6c46f8b7ff87f10ccda4b4d831a3c Mon Sep 17 00:00:00 2001 From: "Yoshiaki Ueda (bootjp)" Date: Thu, 4 Jun 2026 21:36:55 +0900 Subject: [PATCH 1/3] docs(design): rename Composed-1 docs *_proposed_* -> *_partial_* (M5a shipped) Per CLAUDE.md design-doc lifecycle (rename after the first milestone ships): both Composed-1 design docs move from *_proposed_* to *_partial_* now that M5a has fully landed on main. Parent doc: 2026_05_29_proposed_composed1_cross_group_commit_guard.md -> 2026_05_29_partial_composed1_cross_group_commit_guard.md Status header updated to record: - M1-M4 shipped via PR #900 - M5a shipped via PRs #911 / #916 / #924 / #925 - M5b (route-shuffle nemesis) still open M5 doc: 2026_06_02_proposed_composed1_m5_jepsen_route_shuffle.md -> 2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md Status header enumerates the M5a delivery PRs, the cross-PR PR #926 fix discovered during E2E, and a separate ResourceNotFoundException follow-up issue still under investigation. Parent-doc cross-reference link updated to point at the renamed partner so internal navigation stays correct after the rename. Doc-only change. All other content preserved verbatim per the lifecycle convention (only the status block evolves; the design body is the as-shipped record from each milestone). --- ...ial_composed1_cross_group_commit_guard.md} | 8 +++++-- ...tial_composed1_m5_jepsen_route_shuffle.md} | 21 ++++++++++++++++--- 2 files changed, 24 insertions(+), 5 deletions(-) rename docs/design/{2026_05_29_proposed_composed1_cross_group_commit_guard.md => 2026_05_29_partial_composed1_cross_group_commit_guard.md} (98%) rename docs/design/{2026_06_02_proposed_composed1_m5_jepsen_route_shuffle.md => 2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md} (96%) diff --git a/docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md b/docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md similarity index 98% rename from docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md rename to docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md index eba55ff70..a4bd01e26 100644 --- a/docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md +++ b/docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md @@ -1,8 +1,12 @@ # Composed-1 — cross-group commit-time ownership guard -Status: Proposed +Status: Partial — M1–M4 shipped via PR #900; M5a shipped via PRs +#911 / #916 / #924 / #925; M5b (route-shuffle nemesis) still +open. See companion doc +`2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md` for the +detailed M5 design and per-milestone state. Author: bootjp -Date: 2026-05-29 +Date: 2026-05-29 (renamed *_proposed_* → *_partial_* on 2026-06-04) > **Forward-looking proposal.** Today's implementation is vacuously > safe with respect to Composed-1 because `SplitRange` is the only diff --git a/docs/design/2026_06_02_proposed_composed1_m5_jepsen_route_shuffle.md b/docs/design/2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md similarity index 96% rename from docs/design/2026_06_02_proposed_composed1_m5_jepsen_route_shuffle.md rename to docs/design/2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md index 01ea9e407..a2af04328 100644 --- a/docs/design/2026_06_02_proposed_composed1_m5_jepsen_route_shuffle.md +++ b/docs/design/2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md @@ -1,10 +1,25 @@ # Composed-1 M5 — Jepsen route-shuffle workload -Status: Proposed +Status: Partial. M5a shipped: + - PR #911 — `cmd/elastickv-split` + `cmd/elastickv-route-key` CLI helpers + - PR #916 — `dynamodb-append-multi-table-workload` (Clojure) + - PR #925 — `cmd/elastickv-list-routes` + setup-hook + `verify-multi-group-routing!` + - PR #924 — `scripts/run-jepsen-m5-local.sh` single-process + two-group launcher (with cross-PR integration to #925) + - PR #926 (separate) — `--host 127.0.0.1` fix discovered during + M5a E2E + +M5b (route-shuffle nemesis) is still open. Plus one known follow-up +issue from M5a E2E: workers report `ResourceNotFoundException` for +every txn despite `create-all-tables!` reporting success and ListRoutes +showing the expected two-group catalog — likely a CreateTable +race / sync issue under 5-parallel client setup; tracked separately. + Author: bootjp -Date: 2026-06-02 +Date: 2026-06-02 (renamed *_proposed_* → *_partial_* on 2026-06-04) Parent design: -[`2026_05_29_proposed_composed1_cross_group_commit_guard.md`](2026_05_29_proposed_composed1_cross_group_commit_guard.md) +[`2026_05_29_partial_composed1_cross_group_commit_guard.md`](2026_05_29_partial_composed1_cross_group_commit_guard.md) > **Forward-looking proposal, same posture as the parent doc.** > Today's `SplitRange` is same-group only (per CLAUDE.md and From b527dd5fcb62a4bc19c479208646ef126ef6615e Mon Sep 17 00:00:00 2001 From: "Yoshiaki Ueda (bootjp)" Date: Fri, 5 Jun 2026 01:31:49 +0900 Subject: [PATCH 2/3] docs/code(composed1): update stale doc references after rename + tighten Status (codex P2 + gemini medium on PR #927) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit codex P2 — 16 ファイルに古い *_proposed_* パス参照が残存。 全部 *_partial_* に更新 (cmd/, distribution/, jepsen/, kv/, main.go, proto/, scripts/). generated proto/internal.pb.go は source proto/internal.proto の同期更新と一致するため sed で同時置換 (regen 不要)。 gemini medium — markdown 行頭の '#911' が見出しとして誤解釈 される懸念。 親 doc の Status section を 1 行にまとめて回避。 go build ./... -> exit 0 --- cmd/elastickv-list-routes/main.go | 2 +- cmd/elastickv-route-key/main.go | 2 +- cmd/elastickv-split/main.go | 2 +- distribution/engine.go | 2 +- ...2026_05_29_partial_composed1_cross_group_commit_guard.md | 6 +----- jepsen/src/elastickv/dynamodb_multi_table_workload.clj | 2 +- kv/coordinator.go | 2 +- kv/coordinator_txn_test.go | 2 +- kv/fsm.go | 6 +++--- kv/route_history.go | 2 +- kv/sharded_coordinator.go | 2 +- kv/transcoder.go | 2 +- main.go | 2 +- proto/internal.pb.go | 2 +- proto/internal.proto | 2 +- scripts/run-jepsen-m5-local.sh | 2 +- 16 files changed, 18 insertions(+), 22 deletions(-) diff --git a/cmd/elastickv-list-routes/main.go b/cmd/elastickv-list-routes/main.go index 3734b3e3e..5419b0faf 100644 --- a/cmd/elastickv-list-routes/main.go +++ b/cmd/elastickv-list-routes/main.go @@ -4,7 +4,7 @@ // table-route keys on the expected Raft groups before any workload // op runs. // -// Per the design doc (docs/design/2026_06_02_proposed_composed1_m5_jepsen_route_shuffle.md +// Per the design doc (docs/design/2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md // §3.3), the Jepsen client's setup! shells out to this tool rather // than re-implementing the gRPC client in Clojure: a JSON contract is // stable across versions and a future ListRoutes schema change shows diff --git a/cmd/elastickv-route-key/main.go b/cmd/elastickv-route-key/main.go index 2670cc543..2739992e2 100644 --- a/cmd/elastickv-route-key/main.go +++ b/cmd/elastickv-route-key/main.go @@ -1,7 +1,7 @@ // elastickv-route-key prints the byte-string of a DynamoDB // table-route key for a given table name. Used by the Composed-1 // M5 launch script (`scripts/run-jepsen-local.sh`) and OQ-7 -// (docs/design/2026_06_02_proposed_composed1_m5_jepsen_route_shuffle.md) +// (docs/design/2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md) // to compute `--shardRanges` boundary keys without inlining the // base64-encoding logic in shell — the encoding is part of the // routing surface and any drift would silently mis-route. diff --git a/cmd/elastickv-split/main.go b/cmd/elastickv-split/main.go index 2b1987631..029fd3add 100644 --- a/cmd/elastickv-split/main.go +++ b/cmd/elastickv-split/main.go @@ -1,7 +1,7 @@ // elastickv-split is a single-shot CLI that invokes the // Distribution.SplitRange RPC on a running elastickv cluster. Per // the Composed-1 M5 design doc -// (docs/design/2026_06_02_proposed_composed1_m5_jepsen_route_shuffle.md +// (docs/design/2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md // §3.1), the Jepsen route-shuffle nemesis shells out to this tool // rather than re-implementing the gRPC client in Clojure: keeping // the request construction and the SplitRangeRequest field diff --git a/distribution/engine.go b/distribution/engine.go index d7a36f5f0..4bf3611ea 100644 --- a/distribution/engine.go +++ b/distribution/engine.go @@ -37,7 +37,7 @@ type Engine struct { ts atomic.Uint64 hotspotThreshold uint64 // history is the M2 versioned-snapshot ring for Composed-1 - // (docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md + // (docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md // §M2). Keyed by catalogVersion; populated on every successful // ApplySnapshot and seeded by NewEngineWithDefaultRoute so a // transaction that observed catalogVersion = 0 (the engine's diff --git a/docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md b/docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md index a4bd01e26..5a1a416e0 100644 --- a/docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md +++ b/docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md @@ -1,10 +1,6 @@ # Composed-1 — cross-group commit-time ownership guard -Status: Partial — M1–M4 shipped via PR #900; M5a shipped via PRs -#911 / #916 / #924 / #925; M5b (route-shuffle nemesis) still -open. See companion doc -`2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md` for the -detailed M5 design and per-milestone state. +Status: Partial — M1–M4 shipped via PR #900; M5a shipped via PRs #911 / #916 / #924 / #925; M5b (route-shuffle nemesis) still open. See companion doc `2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md` for the detailed M5 design and per-milestone state. Author: bootjp Date: 2026-05-29 (renamed *_proposed_* → *_partial_* on 2026-06-04) diff --git a/jepsen/src/elastickv/dynamodb_multi_table_workload.clj b/jepsen/src/elastickv/dynamodb_multi_table_workload.clj index c277f8b47..5bbbdb723 100644 --- a/jepsen/src/elastickv/dynamodb_multi_table_workload.clj +++ b/jepsen/src/elastickv/dynamodb_multi_table_workload.clj @@ -2,7 +2,7 @@ "Composed-1 M5a multi-table variant of the DynamoDB list-append workload. Why a separate workload exists (see - docs/design/2026_06_02_proposed_composed1_m5_jepsen_route_shuffle.md §3.3): + docs/design/2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md §3.3): the single-table elastickv.dynamodb-workload cannot exercise the 2PC path because kv/shard_key.go normalises every DynamoDB table-meta, item, and GSI key for one table to a single per-table route key diff --git a/kv/coordinator.go b/kv/coordinator.go index 3938f7cdb..06a276ddf 100644 --- a/kv/coordinator.go +++ b/kv/coordinator.go @@ -1085,7 +1085,7 @@ func elemToMutation(req *Elem[OP]) *pb.Mutation { // was captured at — flows into pb.Request.ObservedRouteVersion so the M3 // Composed-1 FSM apply-time gate can re-validate ownership against the // route catalog snapshot at txn-begin (M1 plumbing, see -// docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md). +// docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md). // Zero is the legacy "unpinned" sentinel. func onePhaseTxnRequestWithPrevCommit(startTS, commitTS, prevCommitTS uint64, primaryKey []byte, reqs []*Elem[OP], readKeys [][]byte, observedRouteVersion uint64) *pb.Request { muts := make([]*pb.Mutation, 0, len(reqs)+1) diff --git a/kv/coordinator_txn_test.go b/kv/coordinator_txn_test.go index 09864e801..5b098bb03 100644 --- a/kv/coordinator_txn_test.go +++ b/kv/coordinator_txn_test.go @@ -145,7 +145,7 @@ func TestCoordinateDispatchTxn_PassesReadKeysToRaftEntry(t *testing.T) { // through the dispatchTxn boundary into pb.Request.ObservedRouteVersion. // // This is the M1 round-trip witness for the Composed-1 plumbing per -// docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md +// docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md // §M1: at this milestone the FSM ignores the field, so the only assertion // is that the value survives the encoder. M3 will add the FSM apply-time // gate that *uses* this value. diff --git a/kv/fsm.go b/kv/fsm.go index 8cb3f0c4a..e2bf3436a 100644 --- a/kv/fsm.go +++ b/kv/fsm.go @@ -72,7 +72,7 @@ type kvFSM struct { // short-circuit as "unpinned" (no Composed-1 enforcement) — // matching the pre-feature behaviour byte-for-byte. Concrete // production type is *distribution.Engine. See - // docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md + // docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md // §4.2 prerequisite block + §M2. routes RouteHistory // shardGroupID is the Raft group ID this FSM serves. Used by @@ -171,7 +171,7 @@ func WithCutoverSource(src CutoverSource) FSMOption { // historical owner-of-key resolution. Zero is reserved for the // not-wired case. // -// See docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md +// See docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md // §M2 + §4.2 prerequisite block. func WithRouteHistory(routes RouteHistory, shardGroupID uint64) FSMOption { return func(f *kvFSM) { @@ -551,7 +551,7 @@ func (f *kvFSM) handleTxnRequest(ctx context.Context, r *pb.Request, commitTS ui } // verifyComposed1 is the M3 apply-time Composed-1 gate per -// docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md +// docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md // §4.2(a) + §4.4. Runs two checks before the txn's writes land: // // (a) Observed-version owner — the txn's read-set was captured diff --git a/kv/route_history.go b/kv/route_history.go index 9a85e4866..766d0cfe7 100644 --- a/kv/route_history.go +++ b/kv/route_history.go @@ -20,7 +20,7 @@ import ( // kv.RouteHistory bypass the wrapper entirely and implement the kv // interface directly. // -// See docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md +// See docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md // §M2. func WrapDistributionEngine(e *distribution.Engine) RouteHistory { if e == nil { diff --git a/kv/sharded_coordinator.go b/kv/sharded_coordinator.go index ff3d0aecb..d91d3fd20 100644 --- a/kv/sharded_coordinator.go +++ b/kv/sharded_coordinator.go @@ -485,7 +485,7 @@ func (c *ShardedCoordinator) Dispatch(ctx context.Context, reqs *OperationGroup[ // dispatchTxnWithComposed1Retry runs the M4 Composed-1 retry loop // (design doc -// docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md +// docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md // §M4). Pins reqs.ObservedRouteVersion to the engine's current // catalog version on the FIRST attempt when the caller left it at the // zero sentinel — every txn that flows through ShardedCoordinator diff --git a/kv/transcoder.go b/kv/transcoder.go index 0e276b5c6..cce1a932f 100644 --- a/kv/transcoder.go +++ b/kv/transcoder.go @@ -46,7 +46,7 @@ type OperationGroup[T OP] struct { // "unpinned" — every existing caller leaves it at zero so this // is behaviour-neutral on the M1 plumbing PR. M3 of the // Composed-1 design - // (docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md) + // (docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md) // will gate the FSM apply path on this version so a route shift // between BeginTxn and Commit is caught before it can produce a // G1c anomaly across a cross-group MoveRange / SplitRange. diff --git a/main.go b/main.go index f460fc2c8..f13870401 100644 --- a/main.go +++ b/main.go @@ -846,7 +846,7 @@ func buildShardGroups( // verifyComposed1 apply-time gate can resolve the // observed-version owner-of-key without further plumbing // work. At M2 the FSM stores both but does not consult them; - // see docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md + // see docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md // §M2. sm := kv.NewKvFSMWithHLC(st, clock, kv.WithEncryption(applier), diff --git a/proto/internal.pb.go b/proto/internal.pb.go index 18e554c53..1d87a7e6e 100644 --- a/proto/internal.pb.go +++ b/proto/internal.pb.go @@ -203,7 +203,7 @@ type Request struct { // transaction's read set was captured at (set on BeginTxn from // distribution.Engine.Version()). Zero means "unpinned" (legacy // callers + read-only paths). M3 of the Composed-1 design - // (docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md) + // (docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md) // will gate the FSM apply path on it; M1 (this field's introduction) // is plumbing only — the FSM ignores the value, so all existing // callers see no behaviour change. diff --git a/proto/internal.proto b/proto/internal.proto index 05dde0d47..9a7d9db83 100644 --- a/proto/internal.proto +++ b/proto/internal.proto @@ -48,7 +48,7 @@ message Request { // transaction's read set was captured at (set on BeginTxn from // distribution.Engine.Version()). Zero means "unpinned" (legacy // callers + read-only paths). M3 of the Composed-1 design - // (docs/design/2026_05_29_proposed_composed1_cross_group_commit_guard.md) + // (docs/design/2026_05_29_partial_composed1_cross_group_commit_guard.md) // will gate the FSM apply path on it; M1 (this field's introduction) // is plumbing only — the FSM ignores the value, so all existing // callers see no behaviour change. diff --git a/scripts/run-jepsen-m5-local.sh b/scripts/run-jepsen-m5-local.sh index 4afd2653d..aa791ba65 100755 --- a/scripts/run-jepsen-m5-local.sh +++ b/scripts/run-jepsen-m5-local.sh @@ -5,7 +5,7 @@ # Why this script exists separately from run-jepsen-local.sh: the M5 # workload requires a multi-Raft-group cluster topology that the # existing 3-node single-group layout cannot provide. Per the design -# doc (docs/design/2026_06_02_proposed_composed1_m5_jepsen_route_shuffle.md +# doc (docs/design/2026_06_02_partial_composed1_m5_jepsen_route_shuffle.md # §3.3), today's `validateShardRanges` / `buildShardGroups` only # support a "single process hosts all groups" model — separate # processes per group fail validation or race on Raft listeners. From 39769a9cfe35d00295362ab3165a5e9636bb92eb Mon Sep 17 00:00:00 2001 From: "Yoshiaki Ueda (bootjp)" Date: Fri, 5 Jun 2026 01:39:24 +0900 Subject: [PATCH 3/3] tla(composed): missed _proposed_ -> _partial_ ref (claude[bot] on PR #927) claude[bot] review of b527dd5f flagged tla/composed/Composed.tla:93 as a missed reference. My earlier rg sweep was scoped to --include=*.go/*.clj/*.sh/*.md and did not match .tla files. $ grep -rn '2026_(05_29|06_02)_proposed_composed1' . -> 0 hits after this fix. --- tla/composed/Composed.tla | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tla/composed/Composed.tla b/tla/composed/Composed.tla index d15551c95..1f68bfcab 100644 --- a/tla/composed/Composed.tla +++ b/tla/composed/Composed.tla @@ -90,7 +90,7 @@ VARIABLES \* observed-version check and the apply-time \* current-version cross-version-read fence \* (Composed-1a; see docs/design/ - \* 2026_05_29_proposed_composed1_cross_group_commit_guard.md + \* 2026_05_29_partial_composed1_cross_group_commit_guard.md \* §4.4 and §5). opCount