NextFlow - AI-Powered Workflow Automation Platform

A production-grade, pixel-perfect clone of Krea.ai's workflow builder. NextFlow enables users to compose, execute, and monitor complex multi-step workflows involving LLM calls, media processing, and custom logic through a visual DAG-based canvas interface.

Architecture Overview

Core Components

React Flow Canvas: Visual workflow builder with drag-and-drop node creation, real-time execution visualization, and type-safe connections between compatible node types.

DAG Engine: Implements cycle detection, topological sorting, and dependency-aware batching for parallel execution of independent nodes. Supports convergence nodes that wait for all upstream dependencies.

Trigger.dev Integration: All computationally intensive operations (LLM calls, image cropping, video frame extraction) are executed as isolated Trigger.dev tasks, ensuring scalability and reliability.

Zustand State Management: Separated into three stores:

• Canvas store: workflow structure, undo/redo history, zoom/pan state
• Execution store: real-time node execution state, progress tracking
• History store: workflow run records and node-level execution details

Prisma ORM: PostgreSQL persistence for workflows, execution runs, and node-level history with user-scoped isolation via Clerk authentication.

Node Types

1. Text Node: Input text for workflows
2. Upload Image Node: Image file upload via Transloadit
3. Upload Video Node: Video file upload via Transloadit
4. LLM Node: Google Gemini API with multimodal support (text + images)
5. Crop Image Node: FFmpeg-based image cropping with percentage parameters
6. Extract Frame Node: FFmpeg-based video frame extraction at timestamp or percentage

Execution Model

The DAG engine uses Kahn's algorithm for topological sorting and groups nodes into execution batches based on dependency levels. Nodes at the same level execute in parallel via Promise.all, while convergence nodes automatically wait for all upstream dependencies.

Failure propagation prevents downstream execution when required dependencies fail, but still records partial branch success in the workflow run history.

Setup Instructions

Prerequisites

• Node.js 18+
• PostgreSQL (Neon recommended for serverless)
• Clerk account for authentication
• Google Gemini API key
• Trigger.dev account
• Transloadit account for file uploads

1. Environment Configuration

Create a .env.local file with the following variables:

# Database
DATABASE_URL=postgresql://user:password@host:port/nextflow

# Clerk Authentication
CLERK_SECRET_KEY=your_clerk_secret_key
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=your_clerk_publishable_key

# Trigger.dev
TRIGGER_API_KEY=your_trigger_api_key
TRIGGER_API_URL=https://api.trigger.dev

# Google Gemini API
GEMINI_API_KEY=your_gemini_api_key

# Transloadit
TRANSLOADIT_AUTH_KEY=your_transloadit_auth_key
TRANSLOADIT_SECRET=your_transloadit_secret
TRANSLOADIT_TEMPLATE_ID=your_transloadit_template_id

# Application
NODE_ENV=development
NEXT_PUBLIC_APP_URL=http://localhost:3000

2. Database Setup (Neon)

1. Create a new Neon project at https://console.neon.tech
2. Copy the connection string to DATABASE_URL
3. Run migrations:

pnpm run db:push

3. Clerk Setup

1. Create a new application at https://dashboard.clerk.com
2. Configure OAuth providers (optional)
3. Copy CLERK_SECRET_KEY and NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY
4. Add http://localhost:3000 to allowed redirect URIs

4. Trigger.dev Setup

1. Create account at https://trigger.dev
2. Create a new project
3. Copy TRIGGER_API_KEY
4. Register the three tasks:
   • llm-task (Gemini API calls)
   • crop-task (FFmpeg image cropping)
   • extract-frame-task (FFmpeg video frame extraction)

5. Google Gemini API

1. Go to https://ai.google.dev/
2. Create API key
3. Enable Gemini API
4. Copy key to GEMINI_API_KEY

6. Transloadit Setup

1. Create account at https://transloadit.com
2. Create an upload template for image/video processing
3. Copy auth key, secret, and template ID

7. Installation

# Install dependencies
pnpm install

# Run development server
pnpm dev

# Open http://localhost:3000

API Endpoints

Workflow Management

• GET /api/workflows - List user's workflows
• POST /api/workflows - Create new workflow
• GET /api/workflows/[id] - Get workflow details
• PUT /api/workflows/[id] - Update workflow
• DELETE /api/workflows/[id] - Delete workflow

Execution

• POST /api/workflows/[id]/run - Execute workflow (full, partial, or single node)
• GET /api/workflows/[id]/runs - Get workflow run history

Workflow Execution Modes

Full Execution

Executes all nodes in the workflow respecting dependencies:

POST /api/workflows/[id]/run
{
  "scope": "full"
}

Partial Execution

Executes from a selected node and all downstream dependencies:

POST /api/workflows/[id]/run
{
  "scope": "partial",
  "startNodeId": "node-id"
}

Single Node Execution

Executes only a specific node:

POST /api/workflows/[id]/run
{
  "scope": "single",
  "startNodeId": "node-id"
}

Type-Safe Connections

The connection validation system enforces strict type compatibility:

• Text → Text (Text Node output)
• Image → Image (Upload Image, Crop Image, Extract Frame outputs)
• Video → Video (Upload Video output)
• LLM accepts Text + Image inputs
• Crop Image requires Image input
• Extract Frame requires Video input

Invalid connections are visually prevented in the UI and validated server-side.

DAG Engine Features

Cycle Detection

Uses depth-first search to detect cycles before execution. Prevents workflows with circular dependencies.

Topological Sorting

Kahn's algorithm ensures nodes are executed in valid dependency order.

Parallel Execution

Nodes with no dependencies or completed dependencies execute concurrently via Promise.all.

Convergence Nodes

LLM nodes automatically wait for all upstream dependencies (images and text) before executing.

Failure Propagation

Failed nodes block downstream execution but allow sibling branches to continue, resulting in "partial" workflow status.

Deployment (Vercel)

Prerequisites

• GitHub repository with NextFlow code
• Vercel account
• Environment variables configured

Steps

1. Push code to GitHub
2. Import project in Vercel
3. Configure environment variables in Vercel dashboard
4. Deploy

# Vercel automatically runs:
pnpm build
pnpm start

Database Connection

For production, use Neon's serverless PostgreSQL:

1. Create Neon project
2. Set DATABASE_URL in Vercel environment
3. Vercel automatically handles connection pooling

Trigger.dev in Production

1. Create production Trigger.dev project
2. Update TRIGGER_API_KEY in Vercel
3. Register tasks in production environment

Testing

Run TypeScript type checking:

pnpm check

Run ESLint:

pnpm lint

Run tests:

pnpm test

Project Structure

nextflow/
├── app/
│   ├── api/
│   │   └── workflows/          # API routes
│   ├── dashboard/              # Workflow list page
│   ├── workflow/[id]/          # Workflow editor page
│   ├── sign-in/                # Clerk sign-in
│   ├── layout.tsx              # Root layout
│   ├── page.tsx                # Root redirect
│   └── globals.css             # Global styles
├── components/
│   ├── nodes/                  # Node components
│   │   ├── BaseNode.tsx
│   │   ├── NodeComponents.tsx
│   │   └── CustomEdge.tsx
│   └── workflow/               # Workflow UI
│       ├── WorkflowEditor.tsx
│       ├── LeftSidebar.tsx
│       ├── RightSidebar.tsx
│       └── TopToolbar.tsx
├── lib/
│   ├── dag-engine.ts           # DAG execution
│   ├── connection-validation.ts # Type-safe connections
│   ├── connection-handler.ts   # Edge creation
│   ├── node-executor.ts        # Node execution
│   ├── trigger-client.ts       # Trigger.dev wrapper
│   ├── validation.ts           # Zod schemas
│   └── stores/                 # Zustand stores
│       ├── canvas-store.ts
│       ├── execution-store.ts
│       └── history-store.ts
├── trigger/
│   ├── llm-task.ts            # Gemini API task
│   ├── crop-task.ts           # FFmpeg crop task
│   └── extract-frame-task.ts  # FFmpeg extract task
├── drizzle/
│   └── schema.ts              # Prisma schema
├── server/
│   └── db.ts                  # Database client
├── middleware.ts              # Clerk middleware
└── package.json

Error Handling

Graceful Failures

• Gemini API: Returns user-friendly error messages; partial workflow continues
• FFmpeg: Handles missing files, invalid parameters; logs detailed errors
• Database: Connection pooling with automatic retry; user sees "Database unavailable"
• Trigger.dev: Task timeout after 60s (LLM) or 120s (video); returns error to client

User Feedback

• Node-level error display with red border and error message
• Workflow status badges: success (green), failed (red), partial (yellow), running (blue)
• Real-time progress in right sidebar with node execution details

Performance Considerations

• Parallel Execution: Independent nodes execute simultaneously, reducing total workflow time
• Connection Pooling: Neon provides serverless connection pooling for database
• Task Isolation: Trigger.dev tasks run in isolated environments, preventing resource contention
• Caching: React Flow memoizes node components; Zustand prevents unnecessary re-renders
• Lazy Loading: Workflow runs loaded on demand; history limited to 100 most recent

Security

• User Isolation: All workflows scoped to authenticated user via Clerk
• API Validation: Zod schemas validate all inputs; server-side authorization checks
• Environment Variables: All secrets stored in environment, never committed
• CORS: API routes protected by Clerk middleware
• SQL Injection: Drizzle ORM prevents SQL injection via parameterized queries

Future Enhancements

• Workflow templates and sharing
• Advanced scheduling (cron, webhooks)
• Custom node types via plugins
• Workflow versioning and rollback
• Advanced monitoring and analytics
• Batch execution and job queues
• Real-time collaboration

Support

For issues or questions:

1. Check documentation in this README
2. Review API error messages in browser console
3. Check Trigger.dev logs for task execution issues
4. Verify environment variables are correctly set

UI Inspiration Acknowledgment

The visual design patterns and interaction behaviors in NextFlow were inspired by publicly available Krea.ai frontend references:

• https://github.com/DavidOG03/krea-ai-frontend
• https://github.com/krea-ai

However, all components have been fully re-implemented from scratch using our required technology stack (Next.js 14, React Flow, Zustand, Prisma, Trigger.dev, Clerk, Transloadit, TailwindCSS). We extracted only visual design tokens (spacing, colors, animations, interaction patterns) while building a completely original architecture with:

• Custom DAG execution engine with cycle detection and topological sorting
• Zustand-based state management (canvas, execution, history stores)
• Trigger.dev task integration for LLM, image processing, and video operations
• Prisma ORM with PostgreSQL for workflow persistence
• Clerk authentication with user-scoped workflow isolation
• Type-safe connection validation between nodes
• Custom React Flow node implementations

No code, business logic, backend logic, or API integrations were copied from the reference repositories.

License

MIT