Conversation
Full-bleed deploy preview UI for reviewing GitHub PRs visually. - GitHub OAuth sign-in with serverless callback - Dashboard listing open PRs (authored + review-requested) - Review mode: iframe preview, collapsible sidebar with affected areas checklist, inline code diffs, and review controls - AI-powered diff analysis (Anthropic API) with heuristic fallback - Preview URL auto-detection from PR comments/checks (Vercel, Netlify) - Line-level comments and review submission synced to GitHub Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
QA Report — BlockedStatus: Cannot test in preview. No preview environment exists for this PR. What I checked
Likely causes
To unblock QA
Happy to re-run the full QA pass (OAuth sign-in → dashboard → review mode → iframe preview → AI affected-areas → inline comments → approve/request-changes round-trip to GitHub) as soon as a preview is live. |
There was a problem hiding this comment.
Automated Review — feat/factory-preview-driven-pr-review
Result: BLOCKED
Mode: review-only
✅ code-quality
Level: moderate · Tolerance: medium · Status: PASS
The OAuth callback passes the access token as a plain URL query parameter (?token=...), which is a poor pattern — tokens appear in browser history, server logs, and referrer headers. No tests are present anywhere in the diff. The analyze.ts endpoint passes raw PR diff content directly to an external AI API with minimal sanitization, and error handling swallows the original error (catch (e) without logging).
✅ ux
Level: none · Tolerance: medium · Status: PASS
The PR description explains a full-bleed iframe layout with a collapsible sidebar, and the tech stack choices (Tailwind v4, TanStack Router) are appropriate for the described UX. No red flags are visible in the diff fragments shown; UX concerns (iframe blocking, sidebar layout) are already acknowledged in the PR description.
❌ security
Level: serious · Tolerance: medium · Status: BLOCKED
The GitHub OAuth access token is transmitted via a URL redirect query parameter (?token=data.access_token), exposing it in browser history, server access logs, and referrer headers — a well-known OAuth security anti-pattern. Additionally, the /api/analyze endpoint accepts arbitrary file content from the client request body and forwards it to Anthropic without any authentication check on the caller, meaning any unauthenticated user could abuse the AI API key. No CSRF state parameter is validated in the OAuth callback flow.
✅ scope
Level: moderate · Tolerance: medium · Status: PASS
This is a greenfield application with ~3,500 lines added (mostly package-lock.json) and several new source files. The functional diff itself is reasonably scoped for the described feature, but the PR introduces an entire new app with multiple integrations (GitHub OAuth, Anthropic AI, Vercel serverless) in a single change with no incremental breakdown.
✅ breaking
Level: none · Tolerance: medium · Status: PASS
This appears to be a new standalone application with no existing API contracts, schemas, or consumers to break. No breaking changes are present.
✅ performance
Level: minor · Tolerance: medium · Status: PASS
The /api/analyze endpoint truncates each file patch to 2000 characters but concatenates all files without a total token budget cap, so a PR with many files could still send a very large prompt. No caching of AI analysis results is implemented, so every page load re-calls the Anthropic API.
❌ spec-completeness
Level: critical · Tolerance: medium · Status: BLOCKED
QA was completely blocked — no preview environment exists, no env vars are provisioned, and OAuth sign-in (the entry point) cannot be tested. All 7 verification steps (OAuth sign-in, dashboard, iframe preview, AI checklist, inline comments, PR approval, sidebar collapse) remain unverified. The diff shows server-side OAuth and AI analysis handlers but the frontend components implementing steps 2–7 are not visible in the diff, so it cannot be confirmed they are implemented correctly. This PR should not merge until a working preview is available and all spec steps are validated.
Reviewed by Slop
Summary
Tech Stack
Needs Input
GITHUB_CLIENT_ID/GITHUB_CLIENT_SECRET: Need a GitHub OAuth App provisioned and env vars set in VercelANTHROPIC_API_KEY: Needed for AI-powered diff analysis (falls back to heuristic grouping without it)VITE_GITHUB_CLIENT_ID: Must be set as a public env var for the frontend OAuth redirectX-Frame-Optionson some preview hosts — Vercel previews allow iframing by default, but others may notGenerated with Slop