Skip to content

feat: preview-driven PR review app#1

Open
failingbuild2[bot] wants to merge 1 commit intomainfrom
feat/factory-preview-driven-pr-review
Open

feat: preview-driven PR review app#1
failingbuild2[bot] wants to merge 1 commit intomainfrom
feat/factory-preview-driven-pr-review

Conversation

@failingbuild2
Copy link
Copy Markdown

@failingbuild2 failingbuild2 bot commented Apr 16, 2026

Summary

  • Full-bleed deploy preview UI for reviewing GitHub PRs — iframe loads the preview URL (~90% viewport), collapsible sidebar for review controls
  • GitHub OAuth sign-in, dashboard listing open PRs, review mode with AI-generated affected areas checklist, inline code diffs with line-level comments, and approve/request-changes/comment synced back to GitHub
  • Auto-detects preview URLs from PR comments and check runs (Vercel, Netlify patterns); fallback to manual URL entry

Tech Stack

  • Vite + React + TanStack Router + Tailwind CSS v4
  • Vercel serverless functions for OAuth callback and AI diff analysis
  • GitHub API for all PR data and review operations

Needs Input

  • GITHUB_CLIENT_ID / GITHUB_CLIENT_SECRET: Need a GitHub OAuth App provisioned and env vars set in Vercel
  • ANTHROPIC_API_KEY: Needed for AI-powered diff analysis (falls back to heuristic grouping without it)
  • VITE_GITHUB_CLIENT_ID: Must be set as a public env var for the frontend OAuth redirect
  • Preview iframe will be blocked by X-Frame-Options on some preview hosts — Vercel previews allow iframing by default, but others may not

Generated with Slop

Full-bleed deploy preview UI for reviewing GitHub PRs visually.
- GitHub OAuth sign-in with serverless callback
- Dashboard listing open PRs (authored + review-requested)
- Review mode: iframe preview, collapsible sidebar with affected areas checklist, inline code diffs, and review controls
- AI-powered diff analysis (Anthropic API) with heuristic fallback
- Preview URL auto-detection from PR comments/checks (Vercel, Netlify)
- Line-level comments and review submission synced to GitHub

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
marcusglowe added a commit that referenced this pull request Apr 16, 2026
@failingbuild2
Copy link
Copy Markdown
Author

failingbuild2 bot commented Apr 17, 2026

QA Report — Blocked

Status: Cannot test in preview. No preview environment exists for this PR.

What I checked

  • GET /deployments → empty array (no GitHub deployments created)
  • Check suites for 0d19495 → all 10 integrations (Vercel, Cloudflare, Sentry, Mintlify, Cursor, Claude, Graphite, etc.) stuck in status: queued, conclusion: null — none have actually started after polling for ~4 minutes
  • No bot comments on the PR with a preview URL
  • funpr-git-feat-factory-preview-driven-pr-review.vercel.app → 404
  • funpr.vercel.app → 404

Likely causes

  1. The Vercel GitHub App may not be installed on Create-Inc/funpr, or the project isn't linked — checks stay queued forever when no runner picks them up.
  2. Per the PR description's "Needs Input" section, GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET, ANTHROPIC_API_KEY, and VITE_GITHUB_CLIENT_ID are not yet provisioned in Vercel. Even if a deployment ran, OAuth sign-in (the entry point to the whole feature) would fail.

To unblock QA

  • Link the repo to a Vercel project (or whichever preview host is intended) so PR deployments actually run
  • Provision the four env vars listed in the PR body
  • Register a GitHub OAuth App with the preview URL as the callback
  • Re-request QA once a preview URL is posted on the PR

Happy to re-run the full QA pass (OAuth sign-in → dashboard → review mode → iframe preview → AI affected-areas → inline comments → approve/request-changes round-trip to GitHub) as soon as a preview is live.

@failingbuild2 failingbuild2 bot marked this pull request as ready for review April 17, 2026 23:59
Copy link
Copy Markdown

@slop-reviewer slop-reviewer bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Review — feat/factory-preview-driven-pr-review

Result: BLOCKED
Mode: review-only

✅ code-quality

Level: moderate · Tolerance: medium · Status: PASS

The OAuth callback passes the access token as a plain URL query parameter (?token=...), which is a poor pattern — tokens appear in browser history, server logs, and referrer headers. No tests are present anywhere in the diff. The analyze.ts endpoint passes raw PR diff content directly to an external AI API with minimal sanitization, and error handling swallows the original error (catch (e) without logging).

✅ ux

Level: none · Tolerance: medium · Status: PASS

The PR description explains a full-bleed iframe layout with a collapsible sidebar, and the tech stack choices (Tailwind v4, TanStack Router) are appropriate for the described UX. No red flags are visible in the diff fragments shown; UX concerns (iframe blocking, sidebar layout) are already acknowledged in the PR description.

❌ security

Level: serious · Tolerance: medium · Status: BLOCKED

The GitHub OAuth access token is transmitted via a URL redirect query parameter (?token=data.access_token), exposing it in browser history, server access logs, and referrer headers — a well-known OAuth security anti-pattern. Additionally, the /api/analyze endpoint accepts arbitrary file content from the client request body and forwards it to Anthropic without any authentication check on the caller, meaning any unauthenticated user could abuse the AI API key. No CSRF state parameter is validated in the OAuth callback flow.

✅ scope

Level: moderate · Tolerance: medium · Status: PASS

This is a greenfield application with ~3,500 lines added (mostly package-lock.json) and several new source files. The functional diff itself is reasonably scoped for the described feature, but the PR introduces an entire new app with multiple integrations (GitHub OAuth, Anthropic AI, Vercel serverless) in a single change with no incremental breakdown.

✅ breaking

Level: none · Tolerance: medium · Status: PASS

This appears to be a new standalone application with no existing API contracts, schemas, or consumers to break. No breaking changes are present.

✅ performance

Level: minor · Tolerance: medium · Status: PASS

The /api/analyze endpoint truncates each file patch to 2000 characters but concatenates all files without a total token budget cap, so a PR with many files could still send a very large prompt. No caching of AI analysis results is implemented, so every page load re-calls the Anthropic API.

❌ spec-completeness

Level: critical · Tolerance: medium · Status: BLOCKED

QA was completely blocked — no preview environment exists, no env vars are provisioned, and OAuth sign-in (the entry point) cannot be tested. All 7 verification steps (OAuth sign-in, dashboard, iframe preview, AI checklist, inline comments, PR approval, sidebar collapse) remain unverified. The diff shows server-side OAuth and AI analysis handlers but the frontend components implementing steps 2–7 are not visible in the diff, so it cannot be confirmed they are implemented correctly. This PR should not merge until a working preview is available and all spec steps are validated.


Reviewed by Slop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant