Skip to content
Merged
3 changes: 2 additions & 1 deletion ENV.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
Значения секретов здесь не появляются никогда — только имя и форма;
живые значения — в `.env.vps` на хостах и SOPS/age-контуре.

Переменных: 122. Точка чтения — `src/config/env.ts` (loadEnv) и явно перечисленные consumers.
Переменных: 123. Точка чтения — `src/config/env.ts` (loadEnv) и явно перечисленные consumers.

| Имя | Тип | Обяз. | Default | Secret | Flag | Описание |
| --- | --- | --- | --- | --- | --- | --- |
Expand Down Expand Up @@ -47,6 +47,7 @@
| `LAB_CONSOLIDATION_TOL_REL` | float | — | `0.001` | — | — | Относительный допуск parity-гейта консолидации |
| `LAB_HYPOTHESIS_HOLDOUT` | enum(off, log) | — | `off` | — | off/log → off | Флаг E4b-паттерна (R12a, research-validation-hardening item 5): режим раскатки лёгкого holdout-подтверждения проксистатуса PAPER_CANDIDATE (task hypothesis.holdout), запускающего break_battery@1 (R11) на уровне гипотезы. off — holdout не enqueue-ится; log — enqueue-ится, персистит и логирует, вердикты не меняет. Состояние enforce намеренно отклоняется резолвером до калибровки порогов battery-policy@1 |
| `LAB_MARKET_HISTORY_URL` | url | — | — | — | — | URL market-history-поверхности; при отсутствии — LAB_OPS_READ_URL, затем http://mock-platform:8839 |
| `LAB_ONBOARD_BATTERY_MODE` | enum(off, log) | — | `off` | — | off/log → off | Флаг E4b-паттерна (R13, research-validation-hardening item 6): режим детерминированной onboarding-батареи сеток вокруг baseline-значений перед первым WFO. off — батарея не запускается; log — прогоняет маленькую сетку через ParamGridRunner (все точки в trial ledger сервер-сайд), эмитит strategy.onboard_battery.completed/skipped со счётчиками и lonePeak-эвиденс, вердикты и цепочку baseline→wfo не меняет. Состояние enforce намеренно отклоняется резолвером: стадия log-only, калибровки порогов у неё нет (battery-policy@1) |
| `LAB_OPS_READ_FIXTURE_DIR` | string | — | — | — | — | Каталог фикстур для fixture-режима bot-results/trade-evidence; дефолт — встроенный каталог фикстур |
| `LAB_OPS_READ_TOKEN` | string | — | — | да | — | Bearer-токен ops-read-поверхности |
| `LAB_OPS_READ_URL` | url | — | — | — | — | Базовый URL ops-read-поверхности (bot-results/trade-evidence/market-history); фолбэк http://127.0.0.1:8839 в селекторах |
Expand Down
30 changes: 30 additions & 0 deletions docs/roadmaps/2026-07-23-research-validation-hardening.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,36 @@ Principle for every item: the verdict stays with deterministic versioned code
(b) register every hypothesis as a family trial in the backtester trial
ledger so FAIL/MODIFY retries and many-hypotheses selection get discounted
via DSR automatically.
- **R13 (2026-07-24) — done.** Typed axis catalog `SWEEP_AXIS_CATALOG`
(`hold_time`, `entry_thresholds`, `stops_takes`, `cooldown`, `sizing`,
`regime_as_axis`, plus `leverage` denylisted) with a prompt export; the "wide
plateau, not peak" and "include the expected degradation point" rules plus
the leverage denylist are wired into the sweep-designer prompts
(`SWEEP_DESIGNER_INSTRUCTIONS`) and researcher-capabilities. A deterministic
`denylisted_axis:*` gate lands in `validateSweepGrid` (the denylist check
runs first, the four pre-existing reason codes are untouched). Onboarding
battery: flag `LAB_ONBOARD_BATTERY_MODE` (`off` default → `log`, fail-closed)
hooks into `strategy-baseline.handler` right before `strategy.wfo` is
enqueued — a deterministic ±50% grid around the baseline values, built from
the axis catalog (no LLM, capped at 12 points, axes picked greedily by name),
runs through `ParamGridRunner`; every point lands server-side in the trial
ledger, and a `strategy.onboard_battery.completed` event (counts +
`lonePeak`, no magnitudes) plus an artifact summary close the run. Fail-soft:
the baseline→wfo chain never breaks on battery failure. The
`hold_time`/`sizing` matchers were narrowed to remove threshold/resize
collisions.

Decisions: the grid is built without an LLM (determinism over creativity at
this stage); at most 2 axes combine (3×3 ≤ 12 cap); the result surfaces via
event + artifact rather than a new migration; `enforce` is not implemented —
this is a log-only stage.

Tail: `trialFamilyHint` is absent from the strategy lane —
`StrategyExperimentRunRequest` carries no such field (the wire contract was
left untouched), so onboarding-battery points land in the trial ledger
without a profile hint and are grouped by `moduleRef.id` + window instead — a
separate task if that grouping turns out to matter. Merging the battery's
plateau points into the first WFO round is deferred as YAGNI.
- **R13 — sweep axis catalog + onboarding battery**: extend
`src/mastra/agents/sweep-designer.agent.ts` (and researcher-capabilities)
with a deterministically-checkable axis catalog (hold time, entry
Expand Down
13 changes: 13 additions & 0 deletions src/adapters/wfo/mastra-agents.prompt.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,19 @@ describe('WFO Mastra prompt builders — outcome embargo', () => {
expect(prompts[0]!).toContain('"sharpe":1.1');
});

// R13 (research-validation-hardening item 6): the axis catalog + plateau/degradation rules
// live in the agent's static instructions (sweep-designer.agent.ts), not the per-call prompt
// built here — asserts the two stay separate so the catalog isn't duplicated into every call.
it('sweep-designer per-call prompt does not duplicate the static axis-catalog instructions', async () => {
const { agent, prompts } = capturingAgent({ grid: {}, rationale: 'r' });
const input: SweepInput = {
profile, baselineTrainSummary: dirtyMetrics,
tunableParams: [], restrictToEntryParams: false, maxPoints: 4,
};
await new MastraSweepDesigner(agent, 'test').design(input);
expect(prompts[0]!).not.toMatch(/wide plateau, not (a )?peak/i);
});

it('result-interpreter prompt scrubs embargo keys nested inside topN', async () => {
const { agent, prompts } = capturingAgent({ decision: 'stop' });
const topN = [{
Expand Down
2 changes: 2 additions & 0 deletions src/composition.ts
Original file line number Diff line number Diff line change
Expand Up @@ -479,6 +479,8 @@ export function composeRuntime() {
paperEvidenceRequired: env.LAB_PAPER_EVIDENCE_REQUIRED,
cycleScorecards,
hypothesisHoldoutMode: env.LAB_HYPOTHESIS_HOLDOUT,
paramGridRunner,
onboardBatteryMode: env.LAB_ONBOARD_BATTERY_MODE,
};

const router = new WorkflowRouter();
Expand Down
16 changes: 16 additions & 0 deletions src/config/env-schema.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,19 @@ describe('env-schema: доменные инварианты lab', () => {
expect(v!.description).toMatch(/калибр/);
});

it('LAB_ONBOARD_BATTERY_MODE — флаг off|log, default off, enforce отклонён (стадия log-only)', () => {
const v = byName(doc, 'LAB_ONBOARD_BATTERY_MODE');
expect(v).toBeDefined();
expect(v!.flag).toBe(true);
expect(v!.type).toBe('enum');
expect(v!.enum_values).toEqual(['off', 'log']);
expect(v!.flag_states).toEqual(['off', 'log']);
expect(v!.default_state).toBe('off');
expect(v!.default).toBe('off');
expect(v!.description).toMatch(/enforce/);
expect(v!.description).toMatch(/log-only/);
});

it('все токены/ключи — secret с default null', () => {
const secretByPattern = doc.variables.filter((v) => /_(TOKEN|KEY)$/.test(v.name));
expect(secretByPattern.length).toBeGreaterThanOrEqual(10);
Expand Down Expand Up @@ -309,6 +322,9 @@ describe('env-schema: fail-fast loadEnv сохранён (негативы)', ()
it('enforce для LAB_BREAK_BATTERY_MODE отклоняется', () => {
expect(() => loadEnv({ LAB_BREAK_BATTERY_MODE: 'enforce' } as NodeJS.ProcessEnv)).toThrow(/enforce/);
});
it('enforce для LAB_ONBOARD_BATTERY_MODE отклоняется', () => {
expect(() => loadEnv({ LAB_ONBOARD_BATTERY_MODE: 'enforce' } as NodeJS.ProcessEnv)).toThrow(/enforce/);
});
it('неизвестные значения fail-closed осей бросают', () => {
expect(() => loadEnv({ TRADING_PLATFORM_INTEGRATION: 'backtestr' } as NodeJS.ProcessEnv)).toThrow();
expect(() => loadEnv({ LAB_AGENTS_ADAPTER: 'Mastra' } as NodeJS.ProcessEnv)).toThrow();
Expand Down
10 changes: 10 additions & 0 deletions src/config/env-schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,16 @@ const VARIABLES: EnvVariableSpec[] = [
default: 'off',
description: 'Флаг E4b-паттерна (R12a, research-validation-hardening item 5): режим раскатки лёгкого holdout-подтверждения проксистатуса PAPER_CANDIDATE (task hypothesis.holdout), запускающего break_battery@1 (R11) на уровне гипотезы. off — holdout не enqueue-ится; log — enqueue-ится, персистит и логирует, вердикты не меняет. Состояние enforce намеренно отклоняется резолвером до калибровки порогов battery-policy@1',
}),
v({
name: 'LAB_ONBOARD_BATTERY_MODE',
type: 'enum',
enum_values: ['off', 'log'],
flag: true,
flag_states: ['off', 'log'],
default_state: 'off',
default: 'off',
description: 'Флаг E4b-паттерна (R13, research-validation-hardening item 6): режим детерминированной onboarding-батареи сеток вокруг baseline-значений перед первым WFO. off — батарея не запускается; log — прогоняет маленькую сетку через ParamGridRunner (все точки в trial ledger сервер-сайд), эмитит strategy.onboard_battery.completed/skipped со счётчиками и lonePeak-эвиденс, вердикты и цепочку baseline→wfo не меняет. Состояние enforce намеренно отклоняется резолвером: стадия log-only, калибровки порогов у неё нет (battery-policy@1)',
}),
// --- селекторные оси (boot-safe селекторы читают свой env из composition.ts) ---
v({ name: 'LAB_SIGNED_EVIDENCE_SOURCE', type: 'enum', enum_values: ['none', 'fixture', 'http'], default: 'none', description: 'Источник подписанной backtest-evidence; fixture вне NODE_ENV=test требует LAB_ALLOW_FIXTURE_EVIDENCE=true (fail-closed)', consumers: ['src/adapters/platform/select-signed-evidence.ts'] }),
v({ name: 'LAB_ALLOW_FIXTURE_EVIDENCE', type: 'bool', default: 'false', description: 'Явное разрешение fixture-evidence вне NODE_ENV=test (self-signed, никогда для прод-гейтов)', consumers: ['src/adapters/platform/select-signed-evidence.ts'] }),
Expand Down
17 changes: 17 additions & 0 deletions src/config/env.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -669,3 +669,20 @@ describe('loadEnv — LAB_HYPOTHESIS_HOLDOUT (R12a hypothesis holdout)', () => {
expect(() => loadEnv({ LAB_HYPOTHESIS_HOLDOUT: 'bogus' } as NodeJS.ProcessEnv)).toThrow(/off\|log/);
});
});

describe('loadEnv — LAB_ONBOARD_BATTERY_MODE (R13 onboarding battery)', () => {
it("defaults to 'off' (battery never runs)", () => {
expect(loadEnv({} as NodeJS.ProcessEnv).LAB_ONBOARD_BATTERY_MODE).toBe('off');
expect(loadEnv({ LAB_ONBOARD_BATTERY_MODE: '' } as NodeJS.ProcessEnv).LAB_ONBOARD_BATTERY_MODE).toBe('off');
});

it("accepts 'off' and 'log'", () => {
expect(loadEnv({ LAB_ONBOARD_BATTERY_MODE: 'off' } as NodeJS.ProcessEnv).LAB_ONBOARD_BATTERY_MODE).toBe('off');
expect(loadEnv({ LAB_ONBOARD_BATTERY_MODE: 'log' } as NodeJS.ProcessEnv).LAB_ONBOARD_BATTERY_MODE).toBe('log');
});

it("fail-closed: 'enforce' (log-only stage) and unknown values throw", () => {
expect(() => loadEnv({ LAB_ONBOARD_BATTERY_MODE: 'enforce' } as NodeJS.ProcessEnv)).toThrow(/enforce/);
expect(() => loadEnv({ LAB_ONBOARD_BATTERY_MODE: 'bogus' } as NodeJS.ProcessEnv)).toThrow(/off\|log/);
});
});
6 changes: 6 additions & 0 deletions src/config/env.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { MODEL_PROVIDERS, type ModelProvider } from '../adapters/llm/model-provi
import { DEFAULT_PRESERVATION_THRESHOLDS, type PreservationThresholds } from '../validation/trade-preservation.ts';
import { resolveBreakBatteryMode, type BreakBatteryMode } from '../research/break-battery.ts';
import { resolveHypothesisHoldoutMode, type HypothesisHoldoutMode } from '../research/hypothesis-holdout.ts';
import { resolveOnboardBatteryMode, type OnboardBatteryMode } from '../research/onboard-battery.ts';

export interface Env {
DATABASE_URL?: string;
Expand Down Expand Up @@ -161,6 +162,10 @@ export interface Env {
* 'log' (enqueues the lightweight holdout + break battery, log-only, NEVER changes verdict).
* 'enforce' rejected until battery calibration closes (battery-policy@1). */
LAB_HYPOTHESIS_HOLDOUT: HypothesisHoldoutMode;
/** R13 onboarding-battery rollout mode: 'off' (default — battery never runs) | 'log' (runs the
* deterministic onboarding grid before the first WFO, seeds the trial ledger + logs lone-peak
* evidence, NEVER changes any verdict/chain). 'enforce' rejected — the stage is log-only. */
LAB_ONBOARD_BATTERY_MODE: OnboardBatteryMode;
}

function parseModelProvider(value: string | undefined): ModelProvider {
Expand Down Expand Up @@ -337,6 +342,7 @@ export function loadEnv(source: NodeJS.ProcessEnv = process.env): Env {
LAB_CONSOLIDATION_TOL_ABS: parseFloatOr(source.LAB_CONSOLIDATION_TOL_ABS, 0.01),
LAB_BREAK_BATTERY_MODE: resolveBreakBatteryMode(source.LAB_BREAK_BATTERY_MODE),
LAB_HYPOTHESIS_HOLDOUT: resolveHypothesisHoldoutMode(source.LAB_HYPOTHESIS_HOLDOUT),
LAB_ONBOARD_BATTERY_MODE: resolveOnboardBatteryMode(source.LAB_ONBOARD_BATTERY_MODE),
...loadRagEnv(source),
};
}
Expand Down
48 changes: 48 additions & 0 deletions src/domain/wfo.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,4 +97,52 @@ describe('validateSweepGrid', () => {
const r = validateSweepGrid({}, { tunableParamNames, restrictToEntryParams: false, entryAffecting });
expect(r).toEqual({ ok: false, reason: 'empty_grid' });
});

// R13 (research-validation-hardening item 6, report-13 G13/§3.5): leverage is denylisted from
// any sweep grid until the engine has a liquidation model — a systematic gate, independent of
// whether the profile happens to mark the param tunable.
it('a leverage-named key → denylisted_axis, even when it is a declared tunable param', () => {
const r = validateSweepGrid(
{ 'leverage.multiplier': [1, 2] },
{ tunableParamNames: ['leverage.multiplier'], restrictToEntryParams: false, entryAffecting: [] },
);
expect(r).toEqual({ ok: false, reason: 'denylisted_axis:leverage.multiplier' });
});

it('a margin-named key → denylisted_axis', () => {
const r = validateSweepGrid(
{ marginMode: [1, 2] },
{ tunableParamNames: ['marginMode'], restrictToEntryParams: false, entryAffecting: [] },
);
expect(r).toEqual({ ok: false, reason: 'denylisted_axis:marginMode' });
});

it('a leverage-named key that is NOT a declared tunable param → denylisted_axis takes priority over non_tunable_param', () => {
const r = validateSweepGrid(
{ 'leverage.multiplier': [1, 2] },
{ tunableParamNames, restrictToEntryParams: false, entryAffecting },
);
expect(r).toEqual({ ok: false, reason: 'denylisted_axis:leverage.multiplier' });
});

it('existing reasons are unaffected by the denylist check', () => {
expect(validateSweepGrid(
{ 'entry.fastBouncePct': [1, 2], 'tpLadder.tp1Pct': [3, 4] },
{ tunableParamNames, restrictToEntryParams: false, entryAffecting },
)).toEqual({ ok: true });
expect(validateSweepGrid(
{ 'entry.fastBouncePct': [1, 2], 'unknown.param': [1] },
{ tunableParamNames, restrictToEntryParams: false, entryAffecting },
)).toEqual({ ok: false, reason: 'non_tunable_param:unknown.param' });
expect(validateSweepGrid(
{ 'tpLadder.tp1Pct': [1, 2] },
{ tunableParamNames, restrictToEntryParams: true, entryAffecting },
)).toEqual({ ok: false, reason: 'non_entry_param_in_exploratory:tpLadder.tp1Pct' });
expect(validateSweepGrid(
{ 'entry.fastBouncePct': [] },
{ tunableParamNames, restrictToEntryParams: false, entryAffecting },
)).toEqual({ ok: false, reason: 'empty_values:entry.fastBouncePct' });
expect(validateSweepGrid({}, { tunableParamNames, restrictToEntryParams: false, entryAffecting }))
.toEqual({ ok: false, reason: 'empty_grid' });
});
});
9 changes: 9 additions & 0 deletions src/domain/wfo.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { z } from 'zod';

import type { StrategyParameter } from './strategy-profile.ts';
import { isDenylistedParam } from '../research/sweep-axis-catalog.ts';

/** Alias for the `StrategyProfile.profile.parameters[]` element type. */
export type ProfileParam = StrategyParameter;
Expand Down Expand Up @@ -72,6 +73,11 @@ export type SweepGridValidation = { ok: true } | { ok: false; reason: string };
* upstream confirms its keys are actual tunable params of the profile, nor that a 0-trade
* exploratory sweep stayed restricted to entry-affecting params. The Fake designer used in
* tests happens to respect both constraints; a real LLM is not guaranteed to.
*
* R13 (research-validation-hardening item 6, report-13 G13/§3.5): also rejects any key on a
* denylisted axis (today: leverage/margin — no liquidation model in the engine). Checked BEFORE
* the tunable-param check: the denylist is an absolute gate, independent of whether the profile
* happens to mark the param `tunable`.
*/
export function validateSweepGrid(
grid: Record<string, unknown[]>,
Expand All @@ -80,6 +86,9 @@ export function validateSweepGrid(
const keys = Object.keys(grid);
if (keys.length === 0) return { ok: false, reason: 'empty_grid' };
for (const key of keys) {
if (isDenylistedParam(key)) {
return { ok: false, reason: `denylisted_axis:${key}` };
}
if (!opts.tunableParamNames.includes(key)) {
return { ok: false, reason: `non_tunable_param:${key}` };
}
Expand Down
Loading
Loading