feat(prd): stress-test web UI — trigger + streaming progress (#561)#595
Conversation
Add a Stress Test button to the PRD page that runs the existing recursive-decomposition stress test over SSE and streams progress into a modal. Backend: - core: stress_test_prd_stream() async generator wraps the headless stress_test_prd pipeline, yielding goals_extracted / goal_analyzed / complete / error events (provider.complete offloaded via asyncio.to_thread) - GET /api/v2/prd/stress-test SSE endpoint streams those events; missing PRD / missing ANTHROPIC_API_KEY surface as in-stream error events. GET (not POST) for browser EventSource compatibility, matching the existing task stream endpoint. Frontend: - useStressTestStream hook (built on useEventSource, direct SSE connect) parses events into an idle->streaming->complete|error state machine - StressTestModal renders streaming lines, completion summary, and a graceful error state with Retry - Stress Test button wired through PRDHeader -> PRDView -> /prd page, enabled only when a PRD exists Results rendering / answer flow is out of scope (tracked in #562); the hook retains tech_spec_markdown + ambiguity_report for that work. Closes #561
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✅ Files skipped from review due to trivial changes (3)
WalkthroughAdds an async backend generator that streams PRD stress-test progress via an SSE router endpoint; a React hook subscribes, accumulates human-readable lines, and a modal UI renders streaming, completion, and error states with button wiring on the PRD page and tests across layers. ChangesPRD Stress-Test Streaming Feature
Sequence DiagramsequenceDiagram
participant User
participant PRDPage
participant StressTestModal
participant useStressTestStream
participant EventSource
participant Backend
User->>PRDPage: Click "Stress Test" button
PRDPage->>PRDPage: setStressTestOpen(true)
PRDPage->>StressTestModal: mount open=true
StressTestModal->>useStressTestStream: start()
useStressTestStream->>EventSource: new EventSource(url)
EventSource->>Backend: GET /api/v2/prd/stress-test
Backend->>Backend: stress_test_prd_stream(prd, provider)
loop emit events
Backend-->>EventSource: data: {type:goals_extracted, goals:[...]}
EventSource-->>useStressTestStream: goals_extracted
useStressTestStream->>useStressTestStream: append lines, status=streaming
useStressTestStream-->>StressTestModal: render progress
Backend-->>EventSource: data: {type:goal_analyzed, classification, ambiguity_count}
EventSource-->>useStressTestStream: goal_analyzed
end
Backend-->>EventSource: data: {type:complete, tech_spec_markdown, ambiguity_report}
EventSource-->>useStressTestStream: complete
useStressTestStream->>useStressTestStream: status=complete, close EventSource
useStressTestStream-->>StressTestModal: status/result updated
User->>StressTestModal: Click "Close"
StressTestModal->>useStressTestStream: reset()
StressTestModal->>PRDPage: onOpenChange(false)
PRDPage->>PRDPage: setStressTestOpen(false)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
test |
|
Review: feat(prd): stress-test web UI - see follow-up comments for full review |
|
High: Multi-provider support bypassed
|
|
High: Missing rate limiting on an expensive endpoint Every other router in |
|
Medium: Redundant scroll containers in StressTestModal In |
|
Low: No server-side disconnect detection When the user clicks Cancel mid-stream, the EventSource closes client-side. However |
|
What's well done
|
Address cross-family (codex) review: - Frontend: useStressTestStream now observes useEventSource onError, so a transport-level failure (server down, 404/CORS, dropped connection) with no data frame transitions the modal to the error state instead of hanging on 'Analyzing PRD...'. Guarded to ignore transient (non-CLOSED) reconnect errors and to not clobber a backend-sent error event. - Backend: extract the SSE generator to _stress_test_event_stream and check request.is_disconnected() between events so an abandoned stream stops issuing further (billable) LLM calls — mirroring event_stream_generator.
Cross-family review (codex) — addressed in 18ef219codex flagged two P2 streaming edge cases; both fixed with tests:
Tests added: transport-failure / transient-error / no-clobber (hook) and disconnect-abort / stays-connected (router). Full suite: 908 frontend + 33 backend stress-test tests green. |
Follow-up review — after 18ef219Addressed since last reviewThe two P2 streaming edge cases flagged by codex are properly resolved:
Still unresolved — both block merge1. Multi-provider support bypassed (High) 2. Missing rate limiting (High) Still present — mediumRedundant scroll containers in StressTestModal No new issuesThe rest of the implementation is clean. The |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@codeframe/ui/routers/prd_v2.py`:
- Around line 244-249: The stress-test SSE endpoint function
stress_test_prd_stream_endpoint is missing the standard rate limiting decorator;
add the `@rate_limit_standard`() decorator immediately above the function
definition (matching how other endpoints in this router are protected) so the
LLM-backed streaming path uses the same rate-limiting policy and prevents burst
abuse.
In `@tests/ui/test_prd_stress_test_router.py`:
- Around line 197-200: The assertion on mock_provider.complete.call_count is too
loose (uses "< 6") and can miss regressions; tighten it to guard against a full
non-aborted run (which makes 4 provider calls) by changing the assertion to
assert mock_provider.complete.call_count < 4 so the test fails if abort behavior
regresses to a full run.
In `@web-ui/src/hooks/useStressTestStream.ts`:
- Around line 101-104: The hook sets the streaming state and calls start() even
when workspacePath is null, so ensure you guard against missing workspacePath
before initiating the EventSource: in useStressTestStream.ts check workspacePath
(and that url !== null) before calling start() or flipping to the "streaming"
state where the const url is computed, and return/abort early (or set state to
idle/error) if workspacePath is absent; apply the same guard to the other
start/streaming block around the code referenced at lines 182-190 to prevent
entering an analyzing/streaming state without a valid EventSource URL.
- Around line 8-65: The exported Stress Test UI types (StressTestEventType,
StressTestGoalsExtractedEvent, StressTestGoalAnalyzedEvent,
StressTestCompleteEvent, StressTestErrorEvent, StressTestEvent,
StressTestStatus, StressTestResultData, UseStressTestStreamReturn) should be
relocated into the shared types module (types/index) and this hook file should
import them instead of redeclaring them; remove the local exports, add imports
for each moved type, and ensure the hook implementation and any consumers
reference the imported types to keep the public type surface consistent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 977f6039-e5ef-466e-ab73-c8bce7fb2caa
📒 Files selected for processing (16)
codeframe/core/prd_stress_test.pycodeframe/ui/routers/prd_v2.pytests/core/test_prd_stress_test.pytests/ui/test_prd_stress_test_router.pyweb-ui/__mocks__/@hugeicons/react.jsweb-ui/src/__tests__/components/prd/PRDHeader.test.tsxweb-ui/src/__tests__/components/prd/PrdPage.test.tsxweb-ui/src/__tests__/components/prd/StressTestModal.test.tsxweb-ui/src/__tests__/hooks/useStressTestStream.test.tsweb-ui/src/app/prd/page.tsxweb-ui/src/components/prd/PRDHeader.tsxweb-ui/src/components/prd/PRDView.tsxweb-ui/src/components/prd/StressTestModal.tsxweb-ui/src/components/prd/index.tsweb-ui/src/hooks/index.tsweb-ui/src/hooks/useStressTestStream.ts
- Add @rate_limit_standard() to the stress-test SSE endpoint (it runs recursive LLM decomposition — throttle burst abuse), matching the rest of prd_v2. (coderabbit Major / claude High) - Move SSE event payload types to web-ui/src/types/prd.ts (re-exported from types/index.ts) per the repo's type-location guideline; the hook imports them. (coderabbit Major) - Guard useStressTestStream.start() when workspacePath is null — fail fast to the error state instead of hanging in 'streaming'. (coderabbit) - Drop the redundant ScrollArea wrapper in StressTestModal so the auto-scroll ref and the visible scrollbar are the same element. (claude Medium) - Tighten the disconnect-abort test assertion to <= 2 calls (this fixture's full run is 4). (coderabbit)
…561) Address coderabbit Major: the SSE endpoint hardcoded AnthropicProvider. Now resolve provider via the documented chain (CODEFRAME_LLM_PROVIDER env → .codeframe/config.yaml → default anthropic) using the shared get_provider factory, mirroring runtime.py. The ANTHROPIC_API_KEY gate only applies when the resolved provider is anthropic, so OpenAI-compatible and local (ollama/vllm) providers work. Unknown provider types surface as in-stream error events.
Review feedback addressed (802ecdd, d4d9267)CodeRabbit
claude-review
Suites green: 909 frontend + 6 stress-test router + core tests; ruff + mypy clean. |
|
FULL REVIEW BODY |
Follow-up review -- current HEADResolved since last reviewRate limiting (High -- now fixed) Redundant scroll containers (Medium -- now fixed) Still unresolved -- blocks mergeMulti-provider support bypassed (High) No new issuesEverything else in the current diff is clean. The transport-level error handling, disconnect guard, runId retry pattern, and test coverage are all solid. Once the provider-factory issue is addressed, this is ready to merge. |
Mark Phase 5.4 trigger+streaming shipped in CLAUDE.md, PRODUCT_ROADMAP, and the CLI↔API mapping; note results rendering (#562) still pending.
|
test comment - please ignore |
|
deleting test |
Code Review: feat(prd): stress-test web UIThis is a clean, well-structured PR. The layering is sound: headless core generator → thin SSE router → React hook → modal, mirroring existing conventions throughout. What is working well
Issues1. In # current (wrong order):
base_url_override = (llm_cfg.base_url if llm_cfg else None) or os.getenv("OPENAI_BASE_URL")
# should be (env var wins):
base_url_override = os.getenv("OPENAI_BASE_URL") or (llm_cfg.base_url if llm_cfg else None)This matches the documented priority: env var > .codeframe/config.yaml > default. 2. Async tests may be missing
3. No SSE heartbeat for long-running decompositions (low) A deep decomposition ( 4.
Minor observations (no action needed)
SummaryThe feature is production-ready as written. The one meaningful correctness issue is the |
1 participant
Summary
Adds a Stress Test button to the PRD page that runs the existing recursive-decomposition stress test over SSE and streams progress into a modal. Closes #561.
This is the web surface for
cf prd stress-test— web-only users can now run the most valuable part of the THINK phase (surfacing PRD ambiguities) without the CLI.What changed
Backend (
codeframe/)core/prd_stress_test.py: newstress_test_prd_stream()async generator wrapping the headlessstress_test_prdpipeline, yieldinggoals_extracted/goal_analyzed/complete/errorevents. Each blockingprovider.complete()is offloaded viaasyncio.to_threadso the event loop stays responsive. Stays headless (no FastAPI imports).ui/routers/prd_v2.py: newGET /api/v2/prd/stress-testSSE endpoint that streams those events from the latest PRD. Missing PRD / missingANTHROPIC_API_KEYsurface as in-streamerrorevents.Frontend (
web-ui/)hooks/useStressTestStream.ts: hook built on the existinguseEventSource, parsing events into anidle → streaming → complete | errorstate machine with human-readable progress lines.components/prd/StressTestModal.tsx: streaming log, completion summary (ambiguity count), and a graceful error state with Retry.Stress Testbutton wired throughPRDHeader → PRDView → /prd page, enabled only when a PRD exists.Deviations from the issue's plan (and why)
EventSourceonly issues GET requests; this matches the existingGET /api/v2/tasks/{id}/streamSSE endpoint. (The plan's own sequence diagram shows a GET.)useStressTestStreamhook instead of a rawEventSourcefactory inapi.ts. Matches the establisheduseTaskStreamconvention and connects directly toNEXT_PUBLIC_SSE_URL— the Next.js rewrite proxy buffers chunked responses and would break SSE streaming.useTaskStream), re-exported fromhooks/index.ts.errorevents, not HTTP errors, so the browserEventSourcerenders them via the modal's error state.Testing
uv run pytest tests/core/test_prd_stress_test.py tests/ui/test_prd_stress_test_router.py— green; ruff + mypy clean.npm run buildsucceeds. New code ~98% covered./tmp/demo-561-*.png. The LLM provider was the only mock (no API key in the demo env); the endpoint, core generator, SSE framing, hook, and modal all ran for real.Acceptance criteria
/prdshows the button;PRDHeadertest asserts disabled when no PRD✓ Extracted 3 goals→ per-goal lines →✓ Analysis complete — 1 ambiguity foundStress test failed+ backend message + Retry buttonnpm testanduv run pytestpassKnown limitations
tech_spec_markdown+ambiguity_reportfor that follow-up.Summary by CodeRabbit
New Features
UI
Client
Tests & Docs