feat: studies-list convergence visibility + real demo data (feat_studies_convergence_visibility)

[SoundMindsAI]

Summary

Epic 2 only — Epic 1 already on main via PR #421's squash-merge bundle. This PR completes feat_studies_convergence_visibility by shipping the demo-data enrichment + max_trials bump + headroom test scaffold.

Context

The earlier push (5dab2a5e) included both epics, but during CI watch I discovered Epic 1 had already merged to main as part of PR #421 e5c3b8b9 — a squash-merge that bundled the complementary-architecture-onepager docs PR with the entire Epic 1 backend (count_trials_for_studies / resolve_list_convergence_verdicts / StudySummary.trial_count + convergence_verdict) + frontend (Trials + Convergence columns + glossary entry + E2E spec). I rebased this branch onto e5c3b8b9 and force-pushed; this PR now carries only the Epic 2 commits on top.

What this PR ships

• Story 2.1 — enrich docs + judgments (FR-5) — rewrote all 5 small SCENARIOS in scripts/seed_meaningful_demos.py using the decoy-by-title pattern: each query has a "best answer" doc whose query terms appear in description/body/bullet_points (NOT in title) and a "decoy" doc with query terms densely in title only with shallow description. The deterministic engine-backed headroom harness (backend/tests/integration/test_demo_scenarios_headroom.py) asserts the FR-5 bounds (0.40 ≤ baseline ≤ 0.70, lift ≥ 0.10, better < 0.99) for each scenario:

  Scenario	Baseline NDCG@10	Better NDCG@10	Lift
  acme-products-prod	0.597	0.851	+0.254
  corp-docs-search	0.633	0.863	+0.230
  news-search-staging	0.561	0.799	+0.238
  jobs-marketplace-prod	0.690	0.985	+0.295
  acme-kb-docs-solr	0.644	0.878	+0.234

• Story 2.2 — max_trials 12 → 50 via shared constant (FR-6, D-11) — new DEMO_SMALL_STUDY_MAX_TRIALS exported from scripts/seed_meaningful_demos.py and aliased by backend/app/services/demo_seeding.py:_REAL_STUDY_MAX_TRIALS, single-sourcing the CLI seed and home-button reseed paths so they cannot drift. Pinned at STUDIES_TPE_WARMUP_FLOOR so the small-scenario LLM + UBI studies clear the warmup floor and convergence reads converged / still_improving instead of too_few_trials.

• Story 2.3 scaffold — engine-backed headroom harness — new fixture module backend/tests/integration/fixtures/headroom_harness.py provides ES/OS bulk-index + Solr configset-upload + collection-create helpers; the test renders each scenario's template with the baseline (midpoint) params and a hand-picked "better" param set, evaluates NDCG@10 via the shipped eval engine, and asserts the FR-5 bounds. ES/OS scenarios are hard CI gates (via _require_es_or_fail() / _require_opensearch_or_fail() — CI=true → pytest.fail, local → pytest.skip); Solr skip-gates per D-18.

• Story 2.3 finalize — shape invariants + AC-7/AC-8 — backend/tests/unit/scripts/test_scenarios_judgment_density.py (21 parametrized invariants enforcing doc-count floor, judgment density, valid doc_id/query_idx refs, and the FULL {0,1,2,3} rubric per query). The heavy-lane backend/tests/integration/test_demo_seeding_ubi_full.py gains an AC-7/AC-8 assertion block reading persisted Study.baseline_metric + Study.best_metric via the live count_trials_for_studies + resolve_list_convergence_verdicts path (matches the StudySummary.convergence_verdict pipeline). AC-8 wall-clock ceiling raised 1140s → 3600s per D-9.

Tangential inline fix

• backend/tests/integration/test_health_integration.py: the contract test was asserting the /healthz subsystems set didn't include solr despite Solr being live since 2026-05-31. Fixed inline per CLAUDE.md fix-inline-by-default.

Test coverage

Layer	File	New tests
Unit	backend/tests/unit/scripts/test_scenarios_judgment_density.py	21
Unit	backend/tests/unit/scripts/test_demo_max_trials_single_source.py	4
Integration	backend/tests/integration/test_demo_scenarios_headroom.py	6 (5 scenarios + resolver-parity guard)
Integration	backend/tests/integration/test_demo_seeding_ubi_full.py	+ extended with AC-7/AC-8 block
Integration	backend/tests/integration/test_health_integration.py	+ tangential solr subsystem fix

All Epic 2 tests pass locally: 6 headroom + 21 shape + 4 single-source = 31 new tests green; 2187 unit + 330 contract (existing) green.

Test plan

• make test-unit (2187 passed)
• make test-contract (330 passed, 66 pre-existing skipped on Postgres-not-reachable)
• make test-integration headroom + healthz suite (8 passed against live ES/OS/Solr)
• make lint && make typecheck (clean)
• ./.venv/bin/ruff format --check backend/ scripts/ (CI parity)
• Epic 2 phase-gate GPT-5.5 cycle 1: 6 findings (4 accepted+fixed, 1 accepted as comment, 1 deferred to docs step)
• Epic 2 phase-gate GPT-5.5 cycle 2: clean pass {"findings": []}
• Final GPT-5.5 cross-model review (full diff): 2 findings — both rejected as out-of-scope/UX-non-regression (see adjudication comment)
• Gemini Code Assist (pre-rebase): 2 findings — both rejected as hunk-isolated false positives in Epic 1 code that no longer appears in this PR after the rebase
• CI pr.yml green (running)
• Gemini Code Assist re-review on rebased SHA
• Manual make seed-demo FORCE=1 confirming FR-5/FR-6 ranges (operator-path verification — deferred to release gate; headroom test is the deterministic CI guard)

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces completed-trial counts and convergence verdicts to the studies list endpoint and UI, utilizing new batched database queries to maintain a strict performance budget. It also enriches the five small demo scenarios with denser graded judgments and raises their trial budget to 50, allowing the optimizer to demonstrate real lift and meaningful convergence states. Feedback on the changes suggests explicitly coercing study_id to a string when mapping trial counts and grouping complete trials to prevent potential type mismatch issues if the database returns UUID objects.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

To ensure robustness and prevent potential type mismatch issues (e.g., if study_id is stored or returned as a uuid.UUID object rather than a string), it is safer to explicitly coerce row.study_id to a string when constructing the result dictionary keys. This aligns with the type annotation dict[str, TrialCounts] and prevents silent failures where trial_counts.get(str(r.id)) fails to find the key.

Suggested change

	result: dict[str, TrialCounts] = {
	row.study_id: TrialCounts(total=int(row.total), complete=int(row.complete)) for row in rows
	}
	result: dict[str, TrialCounts] = {
	str(row.study_id): TrialCounts(total=int(row.total), complete=int(row.complete)) for row in rows
	}

[gemini-code-assist]

Similarly, to prevent a potential KeyError if trial.study_id is returned as a uuid.UUID object (while study_ids are passed as strings), explicitly coerce trial.study_id to a string when indexing into the grouped dictionary. This ensures defensive programming and compatibility across different database column types.

Suggested change

	grouped: dict[str, list[Trial]] = {sid: [] for sid in study_ids}
	for trial in (await db.execute(stmt)).scalars().all():
	grouped[trial.study_id].append(trial)
	return grouped
	grouped: dict[str, list[Trial]] = {sid: [] for sid in study_ids}
	for trial in (await db.execute(stmt)).scalars().all():
	grouped[str(trial.study_id)].append(trial)
	return grouped

[SoundMindsAI]

Review adjudication (Gemini Code Assist)

Commits landing fixes: none — both findings rejected with counter-evidence.

Gemini Code Assist (2 findings)

#	Sev	Location	Verdict	Notes
1	Medium	backend/app/db/repo/trial.py:167	Rejected — hunk-isolated	Counter-evidence: backend/app/db/models/trial.py:73 declares study_id: Mapped[str] = mapped_column(String(36), ForeignKey(...), nullable=False) — the column type is a fixed-width VARCHAR(36) UUIDv7 string per the project's data-model convention (UUIDv7 strings, not native UUID type). row.study_id at runtime is already str; explicit str(...) coercion would be speculative future-proofing for a migration that isn't planned. Gemini reviewed the diff hunk without the model definition above it. The reciprocal list_studies caller already uses str(r.id) defensively, so the keys/lookups match.
2	Medium	backend/app/db/repo/trial.py:203	Rejected — same root cause as #1	Same counter-evidence: Trial.study_id is Mapped[str] (VARCHAR(36)). The grouped[trial.study_id].append(trial) lookup is already str-keyed against study_ids (also strings on the call sites). No coercion needed.

Outcomes

• Applied fixes (0): —
• Rejected with cited counter-evidence (2): Gemini docs: reorganize into numbered IA, add pipeline skills and feature templates #1 + docs: MVP1 architecture + 12 feature specs + GPT-5.5 review fixes #2 (both hunk-isolated false positives — model column type pins study_id to str).
• Deferred as non-regression follow-ups (0): —

The cross-model phase-gate review (GPT-5.5 cycle 2) returned {"findings": []} — no other open items from the model-side review. Ready for human review + CI green.

[SoundMindsAI]

Final cross-model review (GPT-5.5, post-Gemini-adjudication)

GPT-5.5 reviewed the full PR diff (git diff main..HEAD, 366 KB across both epics) with the phase-gate + Gemini rejection log in the system prompt. 2 findings raised, both rejected with counter-evidence.

GPT-5.5 final review (2 findings)

#	Sev	Location	Verdict	Notes
1	High	scripts/seed_meaningful_demos.py	Rejected — out of scope, pre-existing behavior	The Solr CLI scenario being study-less in make seed-demo is pre-existing behavior from infra_adapter_solr Story A13 (PR #336, 2026-05-31) and intentionally documented in the script: _seed_solr_scenario_minimum only registers cluster + template, with the comment "operators kick that off via the UI once they've verified the collection contents". The home-button reseed at backend/app/services/demo_seeding.py DOES create the full Solr study + UBI study (8 studies without Solr, 10 with — asserted by test_demo_seeding_ubi_full.py). This PR's spec (feature_spec.md §14) explicitly scopes the FR-5 verification to the headroom test (covers all 5 directly, no optimizer run) + an @pytest.mark.slow end-to-end seed test for ONE representative scenario (acme-products-prod). Expanding the CLI Solr seed path would conflict with the existing make seed-solr separation discipline — that's a separate scope from infra_adapter_solr.
2	Low	ui/src/components/studies/studies-table.column-config.tsx	Rejected — UX non-regression	The convergence column uses tooltipKey: 'convergence_verdict' which the DataTable primitive renders as an <InfoTooltip> in the column HEADER (per the inline comment at studies-table.column-config.tsx:135-137). This is the established convention for multi-row list columns where every row shares the same semantic — the sibling trial_count column uses the same pattern. Per-row badge tooltips would be visual noise (operator scans down the column; the per-row badge color + label IS the cue). Epic 1's own phase-gate GPT-5.5 review accepted this pattern (per the implementation plan tracker §9 — "Epic 1 phase gate — GPT-5.5 review: 5 findings (4 accepted+fixed a0c40d37, F4 rejected with counter-evidence); re-review clean"). The plan's "adjacent" wording (line 115 of implementation_plan.md) is satisfied by the DataTable's header InfoTooltip rendered at data-table.tsx:435.

Outcomes

• Applied fixes (0): —
• Rejected with cited counter-evidence (2): Final-F1 (Solr CLI scope) + Final-F2 (header-tooltip UX convention) — both phase-gate-accepted patterns, no regression in this PR.
• Deferred as non-regression follow-ups (0): —

Convergence summary

• Epic 2 phase-gate cycle 1: 6 findings — 4 accepted + fixed (6e7934a7), 1 accepted as comment, 1 deferred to docs step (applied in 5dab2a5e).
• Epic 2 phase-gate cycle 2: clean pass {"findings": []}.
• Gemini Code Assist (on the PR): 2 findings — both rejected as hunk-isolated false positives (model column type is Mapped[str]); see adjudication summary above.
• Final GPT-5.5 review: 2 findings — both rejected as out-of-scope/UX non-regression (this comment).

Ready for human review + merge once CI is green.

[SoundMindsAI]

Branch rebased — Epic 2 only

Discovered during CI watch that Epic 1 of this feature already merged to main as part of PR #421 e5c3b8b9 (a squash-merge that bundled complementary-architecture.md with the entire Epic 1 backend + frontend code). The original push here carried both epics and conflicted with main on the Epic 1 files.

Rebased onto e5c3b8b9 and force-pushed (5dab2a5e...e0742e71). This PR now carries only the 6 Epic 2 commits on top of main. PR description updated to reflect the Epic 2-only scope.

Gemini findings on the prior push (2 medium, both about backend/app/db/repo/trial.py:167 / :203) are now moot — that file is no longer modified in this PR (it shipped with Epic 1 via #421). Both were rejected on review as hunk-isolated false positives (the Trial.study_id column is Mapped[str] per backend/app/db/models/trial.py:73); they will not surface against the rebased SHA.

The final GPT-5.5 cross-model review's adjudication comment above (re: Solr CLI scope + header-tooltip UX) still applies — both findings reference Epic 2 content unchanged by the rebase.