Skip to content

release: promote dev to main for v0.2.21 — forecast champion selector A–C#364

Merged
w7-mgfcode merged 46 commits into
mainfrom
dev
Jun 1, 2026
Merged

release: promote dev to main for v0.2.21 — forecast champion selector A–C#364
w7-mgfcode merged 46 commits into
mainfrom
dev

Conversation

@w7-mgfcode
Copy link
Copy Markdown
Owner

@w7-mgfcode w7-mgfcode commented Jun 1, 2026

Release: promote devmain for v0.2.21

Cuts the next pre-1.0 PATCH release (0.2.20 → 0.2.21). Merging this PR lets release-please open the chore(main): release 0.2.21 Release PR; merging that tags v0.2.21.

⚠️ Merge this PR via the GitHub web UI (default Merge pull request #N subject) or with an explicit non-conventional --subject. The release: title is non-conventional to release-please so it traverses the underlying feat: commits and bumps — but do not retitle it to a chore:/docs: subject or the bump will be skipped (see RUNBOOKS → "release-please skipped the bump").

Headline — 🏆 Forecast Champion Selector (A → B → C)

Agents / Ollama hardening

RAG / Showcase

API / Ops / UI fixes

Validation

All five gates green on dev HEAD (ruff + format, mypy --strict, pyright --strict, pytest, migration-check). Alembic head: e4f5a6b7c8d9.

Summary by CodeRabbit

  • New Features

    • Added Champion Selector: compare forecasting models, run deterministic ranking, and select a winner for any store/product pair.
    • Async model comparison with live polling and cancellation support.
    • Training and forecasting with safety-stock decision heuristics.
    • Approval-gated promotion of winners to registry aliases.
  • Bug Fixes

    • Improved embedding provider authentication failure handling with machine-readable error classification.
  • Chores

    • Added configuration options for async runner concurrency limits and drain timeouts.

w7-mgfcode and others added 30 commits May 31, 2026 21:05
chore(release): back-merge v0.2.20 into dev
…ful-skip-401

fix(api): skip showcase knowledge phase on embedding auth failure
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
…hots

docs(docs): complete showcase dogfood screenshots
…tion

fix(agents): persist pending_action for gated tool calls
fix(api): honor feature_frame_version >= 3 in ops + registry
fix(api): showcase agent step runs on local Ollama (no API key needed)
…lback

fix(agents): non-streaming fallback for Ollama agent chat
Ollama's OpenAI-compatible /v1/chat/completions rejects any message whose content is JSON null and carries no tool_calls (400 'invalid message content type: <nil>'). A weak local model emits that shape for an empty assistant turn and PydanticAI replays it on retry, so every retry 400s and the run dies with FallbackExceptionGroup. Inject a sanitizing httpx transport into the OllamaProvider client that coerces outgoing content:null to content:"".
…344)

A gated tool (create_alias/archive_run/save_scenario) records deps.pending_action the moment it fires, but does not halt the run. A weak model can ramble past the gate and exhaust its retry budget, so agent.run raises UnexpectedModelBehavior before returning and the post-run approval-surfacing path never runs. The captured approval is valid, so surface it: chat() and stream_chat()'s UnexpectedModelBehavior handlers now check deps.pending_action first and emit the approval (ChatResponse pending_approval / approval_required event) before falling back to the generic error.
)

The chat Approve/Reject handlers awaited POST /approve but discarded the ApprovalResponse and only cleared the card, so a click produced no visible result — and a failed action execution (e.g. create_alias 'Run not found', returned as status=rejected + result.error) was silent. Capture the response and append a one-line report for every outcome (executed / approved-but-failed / rejected / expired) via a pure, unit-tested formatApprovalReport helper.

Closes #346.
fix(agents,ui): Ollama chat HITL — null-content, pending-approval salvage, and approval report
fix(agents): constrain experiment read-only queries
fix(agents): stop experiment read-only tool-call loop
w7-mgfcode and others added 16 commits June 1, 2026 05:24
fix(agents): salvage experiment answer when weak model fails structured output
…or-backend

feat(api,db): add forecast champion selector backend (#353)
…te-in-effect

fix(ui): avoid setState-in-effect in RunHistoryStrip to unblock lint (#357)
feat(ui): add forecast champion selector foundation
feat(api,db,ui): forecast champion selector slice B — async comparison & results
feat(api,db,ui): forecast champion selector slice C — forecast decision, business summary & promotion (#362)
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @w7-mgfcode, your pull request is larger than the review limit of 150000 diff characters

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 1, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 16211643-a0ba-4dc3-b960-5083194045a2

📥 Commits

Reviewing files that changed from the base of the PR and between d4fe362 and 0602a86.

⛔ Files ignored due to path filters (5)
  • docs/user-guide/img/agent-hitl-approve.png is excluded by !**/*.png
  • docs/user-guide/img/inspect-artifacts-panel.png is excluded by !**/*.png
  • docs/user-guide/img/kpi-strip.png is excluded by !**/*.png
  • docs/user-guide/img/ops-snapshot-tiles.png is excluded by !**/*.png
  • docs/user-guide/img/run-history-strip.png is excluded by !**/*.png
📒 Files selected for processing (118)
  • .env.example
  • PRPs/ai_docs/forecast-champion-selector-backend-research.md
  • PRPs/forecast-champion-selector-backend.md
  • PRPs/forecast-champion-selector-slice-a-selection-capability.md
  • PRPs/forecast-champion-selector-slice-b-async-comparison-results.md
  • PRPs/forecast-champion-selector-slice-c-forecast-decision-operationalization.md
  • alembic/env.py
  • alembic/versions/b667d321603c_create_model_selection_run.py
  • alembic/versions/d3e4f5a6b7c8_add_model_selection_candidate_and_progress.py
  • alembic/versions/e4f5a6b7c8d9_add_model_selection_decision_promotion.py
  • app/core/config.py
  • app/core/exceptions.py
  • app/core/problem_details.py
  • app/core/tests/test_config.py
  • app/features/agents/agents/base.py
  • app/features/agents/agents/experiment.py
  • app/features/agents/deps.py
  • app/features/agents/service.py
  • app/features/agents/tests/test_base.py
  • app/features/agents/tests/test_read_only_guard.py
  • app/features/agents/tests/test_service.py
  • app/features/demo/pipeline.py
  • app/features/demo/tests/test_pipeline.py
  • app/features/model_selection/__init__.py
  • app/features/model_selection/capabilities.py
  • app/features/model_selection/decision.py
  • app/features/model_selection/explanations.py
  • app/features/model_selection/models.py
  • app/features/model_selection/ranking.py
  • app/features/model_selection/routes.py
  • app/features/model_selection/runner.py
  • app/features/model_selection/schemas.py
  • app/features/model_selection/service.py
  • app/features/model_selection/tests/__init__.py
  • app/features/model_selection/tests/conftest.py
  • app/features/model_selection/tests/test_async_routes.py
  • app/features/model_selection/tests/test_capabilities.py
  • app/features/model_selection/tests/test_decision.py
  • app/features/model_selection/tests/test_explanations.py
  • app/features/model_selection/tests/test_models.py
  • app/features/model_selection/tests/test_ranking.py
  • app/features/model_selection/tests/test_routes.py
  • app/features/model_selection/tests/test_routes_integration.py
  • app/features/model_selection/tests/test_runner.py
  • app/features/model_selection/tests/test_schemas.py
  • app/features/model_selection/tests/test_service.py
  • app/features/ops/service.py
  • app/features/ops/tests/test_service.py
  • app/features/rag/embeddings.py
  • app/features/rag/routes.py
  • app/features/rag/tests/test_embeddings.py
  • app/features/rag/tests/test_routes.py
  • app/features/registry/service.py
  • app/features/registry/tests/test_service.py
  • app/features/scenarios/agent_tools.py
  • app/features/scenarios/tests/conftest.py
  • app/features/scenarios/tests/test_agent_tools.py
  • app/main.py
  • docs/_base/RUNBOOKS.md
  • docs/user-guide/champion-selector-guide.md
  • docs/user-guide/feature-reference.md
  • docs/user-guide/showcase-walkthrough.md
  • frontend/src/App.tsx
  • frontend/src/components/champion-selector/availability-panel.test.tsx
  • frontend/src/components/champion-selector/availability-panel.tsx
  • frontend/src/components/champion-selector/backtest-settings-form.test.tsx
  • frontend/src/components/champion-selector/backtest-settings-form.tsx
  • frontend/src/components/champion-selector/candidate-model-picker.test.tsx
  • frontend/src/components/champion-selector/candidate-model-picker.tsx
  • frontend/src/components/champion-selector/copy.ts
  • frontend/src/components/champion-selector/decision/business-interpretation-panel.test.tsx
  • frontend/src/components/champion-selector/decision/business-interpretation-panel.tsx
  • frontend/src/components/champion-selector/decision/constants.ts
  • frontend/src/components/champion-selector/decision/daily-forecast-table.test.tsx
  • frontend/src/components/champion-selector/decision/daily-forecast-table.tsx
  • frontend/src/components/champion-selector/decision/decision-section.tsx
  • frontend/src/components/champion-selector/decision/forecast-chart.test.tsx
  • frontend/src/components/champion-selector/decision/forecast-chart.tsx
  • frontend/src/components/champion-selector/decision/forecast-summary-card.test.tsx
  • frontend/src/components/champion-selector/decision/forecast-summary-card.tsx
  • frontend/src/components/champion-selector/decision/promote-champion-dialog.test.tsx
  • frontend/src/components/champion-selector/decision/promote-champion-dialog.tsx
  • frontend/src/components/champion-selector/decision/safety-stock-panel.test.tsx
  • frontend/src/components/champion-selector/decision/safety-stock-panel.tsx
  • frontend/src/components/champion-selector/decision/train-forecast-actions.test.tsx
  • frontend/src/components/champion-selector/decision/train-forecast-actions.tsx
  • frontend/src/components/champion-selector/decision/winner-decision-panel.test.tsx
  • frontend/src/components/champion-selector/decision/winner-decision-panel.tsx
  • frontend/src/components/champion-selector/results/cancel-run-dialog.test.tsx
  • frontend/src/components/champion-selector/results/cancel-run-dialog.tsx
  • frontend/src/components/champion-selector/results/comparison-charts.test.tsx
  • frontend/src/components/champion-selector/results/comparison-charts.tsx
  • frontend/src/components/champion-selector/results/constants.ts
  • frontend/src/components/champion-selector/results/model-detail-drawer.test.tsx
  • frontend/src/components/champion-selector/results/model-detail-drawer.tsx
  • frontend/src/components/champion-selector/results/ranking-table.test.tsx
  • frontend/src/components/champion-selector/results/ranking-table.tsx
  • frontend/src/components/champion-selector/results/run-progress-panel.test.tsx
  • frontend/src/components/champion-selector/results/run-progress-panel.tsx
  • frontend/src/components/champion-selector/results/winner-card.test.tsx
  • frontend/src/components/champion-selector/results/winner-card.tsx
  • frontend/src/components/champion-selector/run-request.test.ts
  • frontend/src/components/champion-selector/run-request.ts
  • frontend/src/components/champion-selector/searchable-entity-select.test.tsx
  • frontend/src/components/champion-selector/searchable-entity-select.tsx
  • frontend/src/components/champion-selector/split-config.ts
  • frontend/src/components/demo/RunHistoryStrip.tsx
  • frontend/src/hooks/index.ts
  • frontend/src/hooks/use-model-selection.test.ts
  • frontend/src/hooks/use-model-selection.ts
  • frontend/src/lib/approval-report.test.ts
  • frontend/src/lib/approval-report.ts
  • frontend/src/lib/constants.ts
  • frontend/src/pages/chat.tsx
  • frontend/src/pages/visualize/champion.test.tsx
  • frontend/src/pages/visualize/champion.tsx
  • frontend/src/types/api.ts
  • tests/test_e2e_demo.py

📝 Walkthrough

Walkthrough

Adds Champion Selector backend and frontend (Slices A–C): migrations, ORM, strict schemas, async runner, service, routes, UI, hooks, and types. Introduces config/env settings. Enhances agents HITL recovery and RAG embedding auth error handling. Updates docs and extensive unit/integration tests.

Changes

Champion Selector End-to-End

Layer / File(s) Summary
Backend data model and migrations
alembic/versions/*, app/features/model_selection/models.py
Creates run/candidate tables, enums, indexes; adds Slice C columns; ORM models reflect schema for runs and per-candidate progress.
Service, runner, and routes
app/features/model_selection/{service,runner,routes}.py, app/core/config.py, .env.example, app/main.py
Implements availability, sync/async runs, cancellation, settlement, training, predicting, promotion; bounded-concurrency runner; FastAPI endpoints; settings and app wiring.
Contracts and pure logic
app/features/model_selection/{schemas,ranking,explanations,decision,capabilities}.py
Strict request/response schemas; catalog builder; deterministic ranking and chart data; winner/business explanations; heuristic decision computations.
Frontend flows and UI
frontend/src/pages/visualize/champion.tsx, components/champion-selector/**, hooks/use-model-selection.ts, types/api.ts, lib/constants.ts, App.tsx
Adds champion page with selection, async progress/results, decision/promote flows; new components and hooks; API types; route/nav entries.
Agents HITL and RAG auth handling
app/features/agents/**, frontend/src/lib/approval-report.ts, frontend/src/pages/chat.tsx, app/features/rag/{embeddings,routes}.py, app/core/{exceptions,problem_details}.py
Improves misbehavior salvage and pending-approval propagation; formats approval reports; classifies embedding auth failures with RFC7807 mapping.
Docs and tests
PRPs/*, docs/**, app/**/tests/**, frontend/**.test.tsx, tests/test_e2e_demo.py
Adds PRPs and user docs; comprehensive unit/integration tests across backend/frontend; e2e demo adjustments.

Sequence Diagram(s)

sequenceDiagram
  participant UI (Champion Page)
  participant Hooks
  participant API
  participant Service
  participant DB
  UI (Champion Page)->>Hooks: useSubmitSelectionRun(request)
  Hooks->>API: POST /model-selection/runs
  API->>Service: submit_run
  Service->>DB: insert run+candidates (RUNNING/PENDING)
  Service-->>API: 202 + monitor_url
  API-->>Hooks: 202
  Hooks-->>UI (Champion Page): selectionId
  UI (Champion Page)->>Hooks: useSelectionRun(selectionId)
  Hooks->>API: GET /model-selection/{id}
  API->>Service: get_selection
  Service->>DB: load run+children
  Service-->>API: run with progress/ranking
  API-->>Hooks: 200
  Hooks-->>UI (Champion Page): render progress/results
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related issues

Possibly related PRs

Suggested labels

autorelease: pending

Suggested reviewers

  • w7-learn

Poem

A rabbit compares with nimble care,
Models race in async air;
Charts and ranks, a champion crowned,
Safety stock and bias found.
If creds say “no,” we hop and skip—
Approvals logged, no fumbled quip.
Thump! Promotion seals the trip.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dev

@w7-mgfcode w7-mgfcode merged commit c839217 into main Jun 1, 2026
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant