From 036e4805cd0541b8e15130e0aa2ce15e4349c79e Mon Sep 17 00:00:00 2001 From: Alexander Nikolskiy Date: Wed, 5 Aug 2026 13:10:47 +0300 Subject: [PATCH] =?UTF-8?q?docs(roadmap):=20S4=20LLM-=D0=BA=D0=BE=D0=BD?= =?UTF-8?q?=D1=82=D1=83=D1=80=20=E2=80=94=20Reflexion-=D1=80=D0=B5=D1=82?= =?UTF-8?q?=D1=80=D0=B0=D0=B9=20=D0=B8=20smoke-=D0=B3=D0=B5=D0=B9=D1=82=20?= =?UTF-8?q?=D0=BF=D1=80=D0=B8=D0=B2=D1=8F=D0=B7=D0=B0=D0=BD=D1=8B=20=D0=BA?= =?UTF-8?q?=20=D0=B0=D0=BA=D1=82=D0=BE=D1=80-=D0=B8=D0=BD=D0=B8=D1=86?= =?UTF-8?q?=D0=B8=D0=B0=D1=82=D0=B8=D0=B2=D0=B5?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Двойное отражение фиксации из карточки shared-execution-engine (2026-08-05): lab-slice в docs/roadmaps/2026-08-05-actor-s4-llm-authoring.md; tech-debt строка «Builder Reflexion» получила якорь на инициативу. Аудит: BuildFeedback-шов и proof-петля есть, в боевой сборке ретрая нет; «ноль сделок» ловится только полной WFO-батареей. Co-Authored-By: Claude Fable 5 --- docs/conversational-operator-roadmap.md | 5 +++ .../2026-08-05-actor-s4-llm-authoring.md | 44 +++++++++++++++++++ 2 files changed, 49 insertions(+) create mode 100644 docs/roadmaps/2026-08-05-actor-s4-llm-authoring.md diff --git a/docs/conversational-operator-roadmap.md b/docs/conversational-operator-roadmap.md index fc9f264..56203a0 100644 --- a/docs/conversational-operator-roadmap.md +++ b/docs/conversational-operator-roadmap.md @@ -295,6 +295,11 @@ public Git history (see research §11). → `validateBundle` is a single pass — an invalid/failing bundle errors without self-correction. Add a bounded Reflexion loop (validation/sandbox errors → back into the builder context → capped retry) to lift codegen yield. Kill-switched on attempt count. (Agent-pattern checkup, 2026-06-19.) + **Anchored to the actor-contract initiative (2026-08-05):** this is S4 scope per the control-center + shared-execution-engine card (section «S4 / LLM-авторский контур») — see + [`docs/roadmaps/2026-08-05-actor-s4-llm-authoring.md`](roadmaps/2026-08-05-actor-s4-llm-authoring.md) + for the audit state (the `BuildFeedback` seam and the F2a proof-harness loop already exist) and the + companion semantic smoke gate. - **Token/cost kill-switch**: we cap *time* (retrieval soft/hard deadlines, reranker timeout) and *depth* (`MAX_CYCLE_DEPTH`), but not *tokens/$* per request. Add a cumulative token/cost budget guard (mirror `RetrievalBudget`) that aborts over-budget work. Phoenix/Mastra provide the usage *numbers*; diff --git a/docs/roadmaps/2026-08-05-actor-s4-llm-authoring.md b/docs/roadmaps/2026-08-05-actor-s4-llm-authoring.md new file mode 100644 index 0000000..86d5f65 --- /dev/null +++ b/docs/roadmaps/2026-08-05-actor-s4-llm-authoring.md @@ -0,0 +1,44 @@ +# Actor contract S4 — LLM authoring contour (lab-local roadmap entry, 2026-08-05) + +Canonical cross-repo status lives in the control-center +[shared-execution-engine card](../../../control-center/docs/delivery/initiatives/shared-execution-engine.md) +(section «S4 / LLM-авторский контур», recorded 2026-08-05); this file keeps only +lab's local slice (registry rule: no plan duplication). + +Full review: control-center `docs/analysis/25-event-driven-actor-contract-review.md` +(recommendation Р13) — currently on the control-center branch +`docs/event-driven-actor-contract-design`, next to the actor-contract spec itself. + +## Lab's part — `proposed`, behind Ф6/S4 (the epic's return trigger) + +Both items are S4 scope (event-driven bundle authoring). Audit of 2026-08-05: +the plumbing exists, the loop and the gate do not. + +- **Production Reflexion retry for bundle authoring.** Today the production + build path is single-shot: `authorStrategyBundleHandler` = build → assemble → + validate (fail-closed: `rejected` → return, the builder is never re-invoked); + the revision path validates the same way (its `MAX_RETRIES = 2` re-runs + candidate backtests, not the builder). The seam already exists: `BuildFeedback` + (`validation.violations` | `parity.diff`) in + `src/ports/strategy-builder.port.ts`, `buildStrategyUserMessage(profile, + feedback?)`, and a working build → validate → prove → feedback → loop lives in + the F2a proof harness (`src/proof/builder-proof-loop.ts`). S4 requirement: + production authoring of event-driven bundles runs through a capped Reflexion + loop with structured errors (industry-measured effect: QuantCode-Bench + 75.8 → 97.5 % valid generations within 1.5–2.4 iterations). The cycle-level + retry (`enqueueResearchRetry` + FAIL/MODIFY feedback through the + Outcome-Embargo sanitizer) already exists and is a different loop — it + iterates hypotheses after a backtest; it does not repair an invalid + generation before one. Related tech-debt entry: «Builder Reflexion» in + [`conversational-operator-roadmap.md`](../conversational-operator-roadmap.md) + — same mechanism, now anchored to this initiative. +- **Semantic smoke gate (lab ↔ backtester seam).** Missing everywhere: a + zero-trades strategy is only caught by the full WFO/holdout battery + (`minTradesTrain: 50` / `minTradesHoldout: 30` → `insufficient_history` / + FAIL) — i.e. after the most expensive step; lab's `make smoke` is + infrastructure-only, and the trade-preservation gate guards revisions, not + activation. Requirement: a cheap run of the freshly built bundle on a short + reference fixture with an activation check (≥1 trade / ≥N decisions) BEFORE + the full backtest is enqueued; the verdict feeds the same Reflexion loop as + `BuildFeedback`. Catches the largest measured class of LLM generation + failures (17.8 % — "signal conditions do not activate on data").